Android-based cloud desktop connection multi-protocol automatic switching method and system

Abstract

This invention relates to the field of cloud desktop transmission protocol technology, and discloses an Android-based method for automatic switching of multiple protocols for cloud desktop connections. The method includes: identifying the cloud desktop transmission protocol corresponding to a target quality transmission curve segment; establishing a pixel change interval-protocol lookup table between the pixel change interval and the cloud desktop transmission protocol; if the current cloud desktop frame to be transmitted is not a video frame, then the Spice protocol is used for cloud desktop image transmission; if the current cloud desktop frame to be transmitted is a video frame, then the current cloud desktop frame to be transmitted is transmitted according to the target transmission protocol; determining whether the image transmission quality is qualified; if not qualified, then the PCoIP protocol is used for image transmission; if qualified, then returning to obtain the current cloud desktop frame to be transmitted. This invention also proposes an Android-based system, electronic device, and computer-readable storage medium for automatic switching of multiple protocols for cloud desktop connections. This invention can solve the problems of poor transmission effect and low flexibility in current cloud desktop transmission methods.

Description

Technical Field

[0001] This invention relates to the field of cloud desktop transmission protocol technology, and in particular to an Android-based cloud desktop connection multi-protocol automatic switching method, system, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of computer technology, internet virtualization technology has been widely applied. Desktop virtualization refers to the virtualization of computer systems, which separates the desktop operating environment from the hardware operating environment, thereby achieving both security and flexibility for the client desktop.

[0003] Currently, cloud desktop transmission protocols mainly include ICA (Independent Computing Architecture), PcoIP (PC over IP), and Spice (Simple Protocol for Independent Computing Environment). Each of these three protocols has its own advantages and applicable areas. However, most cloud desktop connections can only use one specific protocol. Therefore, current cloud desktop transmission methods suffer from poor transmission performance and lack of flexibility. Summary of the Invention

[0004] This invention provides a method, system, and computer-readable storage medium for automatic switching of multiple protocols for cloud desktop connection based on Android. Its main purpose is to solve the problems of poor transmission effect and poor flexibility in current cloud desktop transmission methods.

[0005] To achieve the above objectives, this invention provides a method for automatic switching of multiple protocols for cloud desktop connection based on Android, comprising:

[0006] Obtain test video frames and dynamic region size delineation sequences, and sequentially delineate dynamically changing regions on the test video frames according to the dynamic region size delineation sequences to obtain an initial video frame sequence;

[0007] The pixel positions in the dynamically changing regions of the initial video frame sequence are shuffled to obtain the target video frame sequence;

[0008] The target video frame sequence was transmitted to the cloud desktop using the pre-built ICA and Spice protocols, and the time-to-image transmission quality curves of the ICA and Spice protocols were measured.

[0009] The pixel change value of each target video frame in the target video frame sequence is calculated using a pre-constructed pixel change formula. The pixel change value is then used to position the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves for the ICA and Spice protocols.

[0010] The target quality transmission curve segments are extracted sequentially from the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol, and the cloud desktop transmission protocol corresponding to the target quality transmission curve segments is identified.

[0011] Obtain the pixel variation range corresponding to the target quality transmission curve segment, establish the correspondence between the pixel variation range and the cloud desktop transmission protocol, and obtain the pixel variation range-protocol lookup table;

[0012] Get the current cloud desktop frame to be transmitted;

[0013] Determine whether the cloud desktop frame to be transmitted is a video frame;

[0014] If the cloud desktop frame to be transmitted is not a video frame, then the Spice protocol is used to transmit the cloud desktop image and the image transmission quality is measured.

[0015] If the current cloud desktop frame to be transmitted is a video frame, then calculate the current pixel change value of the current cloud desktop frame to be transmitted;

[0016] Based on the current pixel change value, the target transmission protocol is looked up using the pixel change range-protocol lookup table. The current cloud desktop frame to be transmitted is then transmitted according to the target transmission protocol, and the image transmission quality is measured.

[0017] Determine whether the image transmission quality is up to standard;

[0018] If the current image transmission quality is substandard, then the pre-built PCoIP protocol will be used for image transmission.

[0019] If the current screen transmission quality is acceptable, then return to the steps described above for obtaining the current cloud desktop frame to be transmitted, thus completing the Android-based cloud desktop connection multi-protocol automatic switching method.

[0020] Optionally, obtaining the test video frames and dynamic region size delineation sequence includes:

[0021] Obtain an initial video frame, and set the pixels in the initial video frame to preset pixel values ​​to obtain a test video frame;

[0022] Obtain the dynamic size sequence, and construct the dynamic region size delineation sequence based on the dynamic size sequence.

[0023] Optionally, the determination of the time-to-frame transmission quality curves of the ICA protocol and the Spice protocol includes:

[0024] The screen transmission index score sets of the ICA protocol and Spice protocol in cloud desktop screen transmission were determined respectively;

[0025] The image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission is calculated based on the image transmission index score set.

[0026] Obtain the measurement time, and plot the time-image transmission quality curves of the ICA protocol and Spice protocol based on the correspondence between the image transmission quality and the measurement time.

[0027] Optionally, the step of calculating the image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission based on the image transmission index score set includes:

[0028] Obtain the weights of the image transmission metrics;

[0029] The image transmission index score set is weighted according to the image transmission index weight to obtain the image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission.

[0030] Optionally, the formula for calculating pixel change is as follows:

[0031]

[0032] Where μ represents the pixel change value, i represents the index of the pixel in the target video frame, I represents the total number of pixels in the target video frame, and x′ i x represents the pixel value of the i-th pixel in the next video frame of the target video frame. i represents the pixel value of the i-th pixel in the target video frame, and f represents the transmission frame rate of the target video frame sequence.

[0033] Optionally, the step of using the pixel change value to positionally calibrate the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves for the ICA and Spice protocols includes:

[0034] Sequentially identify the image transmission quality coordinates of the target video frames in the target video frame sequence on the time-image transmission quality curve;

[0035] Identify the measurement time point corresponding to the coordinate point of the image transmission quality;

[0036] Obtain the pixel change value of the target video frame, replace the measurement time point with the pixel change value, and obtain the pixel change-image transmission quality curve of the ICA protocol and Spice protocol.

[0037] Optionally, the step of sequentially identifying the image transmission quality coordinates of the target video frames in the target video frame sequence within the time-image transmission quality curve includes:

[0038] Obtain the video frame number of the target video frame in the target video frame sequence;

[0039] Using a pre-constructed calibration formula, the measurement time of the target video frame is calculated based on the video frame number and the transmission frame rate.

[0040] Based on the measurement time of the target video frame, the image transmission quality coordinate point of the target video frame is identified in the time-image transmission quality curve.

[0041] Optionally, the calibration formula is as follows:

[0042]

[0043] Where t represents the measurement time of the target video frame, t0 represents the measurement start time, j represents the video frame number, and f represents the transmission frame rate.

[0044] Optionally, the step of sequentially extracting the target quality transmission curve segment from the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol includes:

[0045] Determine whether the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol intersect;

[0046] If the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol do not intersect, then the upper pixel change-image transmission quality curve is obtained, and the upper pixel change-image transmission quality curve is used as the target quality transmission curve segment.

[0047] If the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol intersect, the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol are segmented by the intersection point, and the upper pixel change-image transmission quality curve between each two intersection points is identified to obtain the upper pixel change-image transmission quality curve set.

[0048] The target quality transmission curve segments are extracted sequentially from the set of pixel change-image transmission quality curves on the upper side.

[0049] To address the aforementioned problems, the present invention also provides an Android-based cloud desktop connection multi-protocol automatic switching system, the system comprising:

[0050] The target video frame sequence acquisition module is used to acquire test video frames and dynamic region size delineation sequences, sequentially delineate dynamically changing regions on the test video frames according to the dynamic region size delineation sequence to obtain an initial video frame sequence, and shuffle the pixel positions in the dynamically changing regions of the initial video frame sequence to obtain the target video frame sequence.

[0051] The pixel change-image transmission quality curve calibration module is used to transmit the target video frame sequence to the cloud desktop using pre-built ICA and Spice protocols, and to measure the time-image transmission quality curves of the ICA and Spice protocols. It calculates the pixel change value of each target video frame in the target video frame sequence using a pre-built pixel change formula, and uses the pixel change value to calibrate the position of the time-image transmission quality curve, thus obtaining the pixel change-image transmission quality curves of the ICA and Spice protocols.

[0052] The pixel variation range-protocol lookup table construction module is used to sequentially extract target quality transmission curve segments from the pixel variation-screen transmission quality curves of the ICA protocol and Spice protocol, identify the cloud desktop transmission protocol corresponding to the target quality transmission curve segment, obtain the pixel variation range corresponding to the target quality transmission curve segment, establish the correspondence between the pixel variation range and the cloud desktop transmission protocol, and obtain the pixel variation range-protocol lookup table.

[0053] The video frame recognition module is used to acquire the cloud desktop frame to be transmitted and determine whether the cloud desktop frame to be transmitted is a video frame.

[0054] The current cloud desktop frame classification and transmission module is used to: if the current cloud desktop frame to be transmitted is not a video frame, then use the Spice protocol to transmit the cloud desktop image and measure the image transmission quality; if the current cloud desktop frame to be transmitted is a video frame, then calculate the current pixel change value of the current cloud desktop frame to be transmitted; based on the current pixel change value, use the pixel change range-protocol lookup table to look up the target transmission protocol, transmit the current cloud desktop frame to be transmitted according to the target transmission protocol, and measure the image transmission quality; determine whether the image transmission quality is qualified; if the current image transmission quality is unqualified, then use the pre-built PCoIP protocol to transmit the image; if the current image transmission quality is qualified, then return to the above steps of obtaining the current cloud desktop frame to be transmitted.

[0055] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0056] At least one processor; and,

[0057] A memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores instructions that can be executed by the at least one processor, which execute the instructions to implement the above-described method for automatic switching of multiple protocols for connecting cloud desktops based on Android.

[0059] To address the aforementioned issues, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned method for automatic switching of multiple protocols for Android-based cloud desktop connections.

[0060] Compared to the background technology, current cloud desktop transmission methods suffer from poor transmission quality and low flexibility. This invention first requires determining the optimal pixel change values ​​for the ICA and Spice protocols. Before measurement, test video frames and dynamic region size delineation sequences are obtained. Then, dynamic change regions are sequentially delineated on the test video frames according to the dynamic region size delineation sequence to obtain an initial video frame sequence. Finally, the pixel positions in the dynamic change regions of the initial video frame sequence are shuffled to obtain the target video frame sequence. At this point, the pre-constructed ICA and Spice protocols can be used to transmit the target video frame sequence to the cloud desktop. The time-to-image transmission quality curves of the ICA and Spice protocols are measured. Since a correspondence between pixel change values ​​and image transmission quality needs to be established, the pixel change value of each target video frame in the target video frame sequence needs to be calculated using a pre-constructed pixel change formula. The pixel change values ​​are then used to calibrate the position of the time-to-image transmission quality curves to obtain the pixel change-image transmission quality curves of the ICA and Spice protocols. The target quality transmission curve segment is extracted sequentially from the pixel change-image transmission quality curve of the protocol. Then, the cloud desktop transmission protocol corresponding to the target quality transmission curve segment is identified to obtain the pixel change range corresponding to the target quality transmission curve segment. Finally, the correspondence between the pixel change range and the cloud desktop transmission protocol is established to obtain the pixel change range-protocol lookup table. At this point, the current cloud desktop frame to be transmitted can be classified and transmitted. If the current cloud desktop frame to be transmitted is not a video frame, it indicates that the Spice protocol can be used for cloud desktop image transmission. If the current cloud desktop frame to be transmitted is a video frame, the current pixel change value of the current cloud desktop frame to be transmitted needs to be calculated. Then, based on the current pixel change value, the target transmission protocol is looked up using the pixel change range-protocol lookup table to achieve the purpose of transmitting the current cloud desktop frame to be transmitted according to the target transmission protocol. At the same time, during the image transmission process, it is necessary to determine whether the image transmission quality is qualified. If the current image transmission quality is unqualified, the pre-built PCoIP protocol is used for image transmission. If the current image transmission quality is qualified, the steps of obtaining the current cloud desktop frame to be transmitted are returned until the transmission is completed. Therefore, the Android-based cloud desktop connection multi-protocol automatic switching method, system, electronic device and computer-readable storage medium proposed in this invention can solve the problems of poor transmission effect and poor flexibility in the current cloud desktop transmission method. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating an embodiment of the Android-based cloud desktop connection multi-protocol automatic switching method provided by the present invention.

[0062] Figure 2 This is a functional module diagram of an Android-based cloud desktop connection system with automatic multi-protocol switching, provided in an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the Android-based cloud desktop connection multi-protocol automatic switching method according to an embodiment of the present invention.

[0064] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] This application provides an Android-based method for automatic multi-protocol switching in cloud desktop connections. The executing entity of this Android-based method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the Android-based method for automatic multi-protocol switching in cloud desktop connections can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0067] Example 1:

[0068] Reference Figure 1 The diagram shown is a flowchart illustrating an Android-based cloud desktop connection multi-protocol automatic switching method according to an embodiment of the present invention. In this embodiment, the Android-based cloud desktop connection multi-protocol automatic switching method includes:

[0069] S1. Obtain test video frames and dynamic region size delineation sequence, and delineate dynamically changing regions on the test video frames in sequence according to the dynamic region size delineation sequence to obtain an initial video frame sequence.

[0070] Explained, the test video frame refers to the video frame used for testing, and the pixel values ​​of all pixels in the test video frame are not fixed values. The dynamic region size delineation sequence refers to a pre-constructed sequence of rectangles of different sizes.

[0071] Furthermore, the initial video frame sequence refers to a sequence of test video frames defined by a dynamically changing region. The dynamically changing region refers to a region where pixel changes are possible.

[0072] In this embodiment of the invention, obtaining the test video frames and the dynamic region size delineation sequence includes:

[0073] Obtain an initial video frame, and set the pixels in the initial video frame to preset pixel values ​​to obtain a test video frame;

[0074] Obtain the dynamic size sequence, and construct the dynamic region size delineation sequence based on the dynamic size sequence.

[0075] S2. Disrupt the pixel positions in the dynamically changing regions of the initial video frame sequence to obtain the target video frame sequence.

[0076] Furthermore, the pixels in the dynamically changing region are scrambled to form video frames with a different dynamically changing region than the test video frame. The target video frame sequence refers to the video frame sequence obtained after scrambling the pixels in the dynamically changing region of the initial video frame sequence. For example, the first target video frame in the target video frame sequence is 100*100, while the pixels in the 1*1 dynamically changing region at the top left corner are scrambled, and the pixels in the 2*2 dynamically changing region at the top left corner of the second target video frame are scrambled.

[0077] S3. Use the pre-built ICA protocol and Spice protocol to transmit the target video frame sequence to the cloud desktop, and measure the time-to-image transmission quality curves of the ICA protocol and Spice protocol.

[0078] Understandably, the ICA protocol refers to the Independent Computing Architecture protocol, and the Spice protocol refers to the Simple Protocol for Independent Computing Environment protocol. The time-to-image transmission quality curve refers to the curve showing how image transmission quality changes over time.

[0079] In this embodiment of the invention, the determination of the time-to-frame transmission quality curves of the ICA protocol and the Spice protocol includes:

[0080] The screen transmission index score sets of the ICA protocol and Spice protocol in cloud desktop screen transmission were determined respectively;

[0081] The image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission is calculated based on the image transmission index score set.

[0082] Obtain the measurement time, and plot the time-image transmission quality curves of the ICA protocol and Spice protocol based on the correspondence between the image transmission quality and the measurement time.

[0083] Furthermore, the image transmission index score set refers to the score set of various image transmission indicators, which can be indicators such as latency, packet loss rate, and jitter.

[0084] Specifically, the step of calculating the image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission based on the image transmission index score set includes:

[0085] Obtain the weights of the image transmission metrics;

[0086] The image transmission index score set is weighted according to the image transmission index weight to obtain the image transmission quality of the ICA protocol and Spice protocol in cloud desktop image transmission.

[0087] Furthermore, the image transmission index weight refers to the weight of each index. For example, the image transmission index weight for latency is 0.3, the image transmission index weight for packet loss rate is 0.5, and the image transmission index weight for jitter is 0.2. The higher the image transmission quality, the better the image transmission effect.

[0088] S4. Calculate the pixel change value of each target video frame in the target video frame sequence using the pre-constructed pixel change formula, and use the pixel change value to calibrate the position of the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves of the ICA protocol and the Spice protocol.

[0089] Explained, the pixel change-image transmission quality curve refers to the image transmission quality curve that changes with the pixel change value.

[0090] In this embodiment of the invention, the formula for calculating pixel change is as follows:

[0091]

[0092] Where μ represents the pixel change value, i represents the index of the pixel in the target video frame, I represents the total number of pixels in the target video frame, and x′ i x represents the pixel value of the i-th pixel in the next video frame of the target video frame. i represents the pixel value of the i-th pixel in the target video frame, and f represents the transmission frame rate of the target video frame sequence.

[0093] In this embodiment of the invention, the step of using the pixel change value to positionally calibrate the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves of the ICA and Spice protocols includes:

[0094] Sequentially identify the image transmission quality coordinates of the target video frames in the target video frame sequence on the time-image transmission quality curve;

[0095] Identify the measurement time point corresponding to the coordinate point of the image transmission quality;

[0096] Obtain the pixel change value of the target video frame, replace the measurement time point with the pixel change value, and obtain the pixel change-image transmission quality curve of the ICA protocol and Spice protocol.

[0097] Further, the step of sequentially identifying the image transmission quality coordinates of the target video frames in the target video frame sequence within the time-image transmission quality curve includes:

[0098] Obtain the video frame number of the target video frame in the target video frame sequence;

[0099] Using a pre-constructed calibration formula, the measurement time of the target video frame is calculated based on the video frame number and the transmission frame rate.

[0100] Based on the measurement time of the target video frame, the image transmission quality coordinate point of the target video frame is identified in the time-image transmission quality curve.

[0101] The calibration formula can be explained as follows:

[0102]

[0103] Where t represents the measurement time of the target video frame, t0 represents the measurement start time, j represents the video frame number, and f represents the transmission frame rate.

[0104] Furthermore, each target video frame corresponds to a measurement time and a pixel change value. Therefore, the pixel change-picture transmission quality curve can be constructed based on the correspondence between the picture transmission quality of the target video frame and the pixel change value.

[0105] S5. Extract the target quality transmission curve segment sequentially from the pixel change-image transmission quality curve of the ICA protocol and Spice protocol, and identify the cloud desktop transmission protocol corresponding to the target quality transmission curve segment.

[0106] Explained, the target quality transmission curve segment refers to the transmission protocol that exhibits better image transmission quality within a specific measurement period. By comparing the image transmission quality represented by the pixel change-image transmission quality curves of the ICA and Spice protocols at the same time, the optimal transmission protocol corresponding to different pixel change values ​​can be identified. For example, if the entire measurement period is from 10:00 to 11:00, and the pixel change-image transmission quality curve value corresponding to the ICA protocol is larger than that of the Spice protocol during the 10:00-10:10 time period, then the pixel change value during the 10:00-10:10 time period is suitable for image transmission using the ICA protocol.

[0107] In this embodiment of the invention, the step of sequentially extracting the target quality transmission curve segment from the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol includes:

[0108] Determine whether the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol intersect;

[0109] If the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol do not intersect, then the upper pixel change-image transmission quality curve is obtained, and the upper pixel change-image transmission quality curve is used as the target quality transmission curve segment.

[0110] If the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol intersect, the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol are segmented by the intersection point, and the upper pixel change-image transmission quality curve between each two intersection points is identified to obtain the upper pixel change-image transmission quality curve set.

[0111] The target quality transmission curve segments are extracted sequentially from the set of pixel change-image transmission quality curves on the upper side.

[0112] S6. Obtain the pixel change range corresponding to the target quality transmission curve segment, establish the correspondence between the pixel change range and the cloud desktop transmission protocol, and obtain the pixel change range-protocol lookup table.

[0113] S7. Obtain the current cloud desktop frame to be transmitted.

[0114] Explained, the currently pending cloud desktop frame refers to a specific video frame in the set of currently pending cloud desktop video frames.

[0115] S8. Determine whether the cloud desktop frame to be transmitted is a video frame.

[0116] In this embodiment of the invention, the determination can be made by judging whether the frame rate of the video frame containing the cloud desktop frame to be transmitted is greater than 20 frames / s. If the frame rate of the video frame is greater than 20 frames / s, it indicates that it is a video frame; otherwise, it is not.

[0117] If the cloud desktop frame to be transmitted is not a video frame, then execute S9, use the Spice protocol to transmit the cloud desktop screen, and measure the screen transmission quality.

[0118] Furthermore, the Spice protocol is applicable to office environments, so when the cloud desktop frame to be transmitted is not a video frame, the Spice protocol is used by default.

[0119] If the current cloud desktop frame to be transmitted is a video frame, then execute S10 and calculate the current pixel change value of the current cloud desktop frame to be transmitted.

[0120] In this embodiment of the invention, when the cloud desktop frame to be transmitted is a video frame, the ICA protocol, which is more suitable for video, Flash playback, and 3D applications, is adopted.

[0121] S11. Based on the current pixel change value, use the pixel change range-protocol lookup table to find the target transmission protocol, transmit the current cloud desktop frame to be transmitted according to the target transmission protocol, and measure the image transmission quality.

[0122] S12. Determine whether the image transmission quality is qualified.

[0123] Furthermore, a threshold can be set to determine whether the image transmission quality is up to standard.

[0124] If the current image transmission quality is unqualified, then execute S13 and use the pre-built PCoIP protocol to transmit the image.

[0125] Understandably, the PCoIP protocol refers to the PC over IP protocol. Since the PCoIP protocol is more suitable for use in low-bandwidth networks, it can be used when the video transmission quality is consistently substandard.

[0126] If the current screen transmission quality is acceptable, then return to the steps described above for obtaining the current cloud desktop frame to be transmitted, thus completing the Android-based cloud desktop connection multi-protocol automatic switching method.

[0127] Compared to the background technology, current cloud desktop transmission methods suffer from poor transmission quality and low flexibility. This invention first requires determining the optimal pixel change values ​​for the ICA and Spice protocols. Before measurement, test video frames and dynamic region size delineation sequences are obtained. Then, dynamic change regions are sequentially delineated on the test video frames according to the dynamic region size delineation sequence to obtain an initial video frame sequence. Finally, the pixel positions in the dynamic change regions of the initial video frame sequence are shuffled to obtain the target video frame sequence. At this point, the pre-constructed ICA and Spice protocols can be used to transmit the target video frame sequence to the cloud desktop. The time-to-image transmission quality curves of the ICA and Spice protocols are measured. Since a correspondence between pixel change values ​​and image transmission quality needs to be established, the pixel change value of each target video frame in the target video frame sequence needs to be calculated using a pre-constructed pixel change formula. The pixel change values ​​are then used to calibrate the position of the time-to-image transmission quality curves to obtain the pixel change-image transmission quality curves of the ICA and Spice protocols. The target quality transmission curve segment is extracted sequentially from the pixel change-image transmission quality curve of the protocol. Then, the cloud desktop transmission protocol corresponding to the target quality transmission curve segment is identified to obtain the pixel change range corresponding to the target quality transmission curve segment. Finally, the correspondence between the pixel change range and the cloud desktop transmission protocol is established to obtain the pixel change range-protocol lookup table. At this point, the current cloud desktop frame to be transmitted can be classified and transmitted. If the current cloud desktop frame to be transmitted is not a video frame, it indicates that the Spice protocol can be used for cloud desktop image transmission. If the current cloud desktop frame to be transmitted is a video frame, the current pixel change value of the current cloud desktop frame to be transmitted needs to be calculated. Then, based on the current pixel change value, the target transmission protocol is looked up using the pixel change range-protocol lookup table to achieve the purpose of transmitting the current cloud desktop frame to be transmitted according to the target transmission protocol. At the same time, during the image transmission process, it is necessary to determine whether the image transmission quality is qualified. If the current image transmission quality is unqualified, the pre-built PCoIP protocol is used for image transmission. If the current image transmission quality is qualified, the steps of obtaining the current cloud desktop frame to be transmitted are returned until the transmission is completed. Therefore, the Android-based cloud desktop connection multi-protocol automatic switching method, system, electronic device and computer-readable storage medium proposed in this invention can solve the problems of poor transmission effect and poor flexibility in the current cloud desktop transmission method.

[0128] Example 2:

[0129] like Figure 2 The diagram shown is a functional block diagram of an Android-based cloud desktop connection multi-protocol automatic switching system provided in an embodiment of the present invention.

[0130] The Android-based cloud desktop connection multi-protocol automatic switching system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the Android-based cloud desktop connection multi-protocol automatic switching system 100 may include a target video frame sequence acquisition module 101, a pixel change-image transmission quality curve calibration module 102, a pixel change interval-protocol lookup table construction module 103, a video frame recognition module 104, and a current cloud desktop frame classification and transmission module 105. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0131] The target video frame sequence acquisition module 101 is used to acquire test video frames and dynamic region size delineation sequences, sequentially delineate dynamically changing regions on the test video frames according to the dynamic region size delineation sequence to obtain an initial video frame sequence; and shuffle the pixel positions in the dynamically changing regions of the initial video frame sequence to obtain the target video frame sequence.

[0132] The pixel change-image transmission quality curve calibration module 102 is used to transmit the target video frame sequence to the cloud desktop using the pre-built ICA protocol and Spice protocol respectively, and to measure the time-image transmission quality curve of the ICA protocol and Spice protocol; to calculate the pixel change value of each target video frame in the target video frame sequence using the pre-built pixel change formula, and to use the pixel change value to calibrate the position of the time-image transmission quality curve to obtain the pixel change-image transmission quality curve of the ICA protocol and Spice protocol;

[0133] The pixel change interval-protocol lookup table construction module 103 is used to sequentially extract target quality transmission curve segments from the pixel change-screen transmission quality curves of the ICA protocol and the Spice protocol, identify the cloud desktop transmission protocol corresponding to the target quality transmission curve segment, obtain the pixel change interval corresponding to the target quality transmission curve segment, establish the correspondence between the pixel change interval and the cloud desktop transmission protocol, and obtain the pixel change interval-protocol lookup table.

[0134] The video frame recognition module 104 is used to acquire the current cloud desktop frame to be transmitted and determine whether the current cloud desktop frame to be transmitted is a video frame.

[0135] The current cloud desktop frame classification and transmission module 105 is used to: if the current cloud desktop frame to be transmitted is not a video frame, then use the Spice protocol to transmit the cloud desktop image and measure the image transmission quality; if the current cloud desktop frame to be transmitted is a video frame, then calculate the current pixel change value of the current cloud desktop frame to be transmitted; based on the current pixel change value, use the pixel change range-protocol lookup table to find the target transmission protocol, transmit the current cloud desktop frame to be transmitted according to the target transmission protocol, and measure the image transmission quality; determine whether the image transmission quality is qualified; if the current image transmission quality is unqualified, then use the pre-built PCoIP protocol to transmit the image; if the current image transmission quality is qualified, then return to the above steps of obtaining the current cloud desktop frame to be transmitted.

[0136] In detail, the modules in the Android-based cloud desktop connection multi-protocol automatic switching system 100 described in this embodiment of the invention adopt the same approach as described above. Figure 1 The method described above uses the same technical means as the Android-based cloud desktop connection multi-protocol automatic switching method and can produce the same technical effect, so it will not be repeated here.

[0137] Example 3:

[0138] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements an Android-based cloud desktop connection multi-protocol automatic switching method according to an embodiment of the present invention.

[0139] The electronic device 1 may include a processor 10, a memory 11, a bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as an Android-based cloud desktop connection multi-protocol automatic switching program.

[0140] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code for an Android-based cloud desktop connection multi-protocol automatic switching program, but also to temporarily store data that has been output or will be output.

[0141] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., an Android-based cloud desktop connection multi-protocol automatic switching program) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0142] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0143] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0144] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management system, thereby enabling functions such as charging management, discharging management, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0145] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0146] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0147] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0148] The Android-based cloud desktop connection multi-protocol automatic switching program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0149] Obtain test video frames and dynamic region size delineation sequences, and sequentially delineate dynamically changing regions on the test video frames according to the dynamic region size delineation sequences to obtain an initial video frame sequence;

[0150] The pixel positions in the dynamically changing regions of the initial video frame sequence are shuffled to obtain the target video frame sequence;

[0151] The target video frame sequence was transmitted to the cloud desktop using the pre-built ICA and Spice protocols, and the time-to-image transmission quality curves of the ICA and Spice protocols were measured.

[0152] The pixel change value of each target video frame in the target video frame sequence is calculated using a pre-constructed pixel change formula. The pixel change value is then used to position the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves for the ICA and Spice protocols.

[0153] The target quality transmission curve segments are extracted sequentially from the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol, and the cloud desktop transmission protocol corresponding to the target quality transmission curve segments is identified.

[0154] Obtain the pixel variation range corresponding to the target quality transmission curve segment, establish the correspondence between the pixel variation range and the cloud desktop transmission protocol, and obtain the pixel variation range-protocol lookup table;

[0155] Obtain the current cloud desktop frame to be transmitted, and determine whether the current cloud desktop frame to be transmitted is a video frame;

[0156] If the cloud desktop frame to be transmitted is not a video frame, then the Spice protocol is used to transmit the cloud desktop image and the image transmission quality is measured.

[0157] If the current cloud desktop frame to be transmitted is a video frame, then calculate the current pixel change value of the current cloud desktop frame to be transmitted;

[0158] Based on the current pixel change value, the target transmission protocol is looked up using the pixel change range-protocol lookup table. The current cloud desktop frame to be transmitted is then transmitted according to the target transmission protocol, and the image transmission quality is measured.

[0159] Determine whether the image transmission quality is up to standard;

[0160] If the current image transmission quality is substandard, then the pre-built PCoIP protocol will be used for image transmission.

[0161] If the current screen transmission quality is acceptable, then return to the steps described above for obtaining the current cloud desktop frame to be transmitted, thus completing the Android-based cloud desktop connection multi-protocol automatic switching method.

[0162] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 2 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0163] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0164] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0165] Obtain test video frames and dynamic region size delineation sequences, and sequentially delineate dynamically changing regions on the test video frames according to the dynamic region size delineation sequences to obtain an initial video frame sequence;

[0166] The pixel positions in the dynamically changing regions of the initial video frame sequence are shuffled to obtain the target video frame sequence;

[0167] The target video frame sequence was transmitted to the cloud desktop using the pre-built ICA and Spice protocols, and the time-to-image transmission quality curves of the ICA and Spice protocols were measured.

[0168] The pixel change value of each target video frame in the target video frame sequence is calculated using a pre-constructed pixel change formula. The pixel change value is then used to position the time-to-image transmission quality curve to obtain the pixel change-to-image transmission quality curves for the ICA and Spice protocols.

[0169] The target quality transmission curve segments are extracted sequentially from the pixel change-image transmission quality curves of the ICA protocol and the Spice protocol, and the cloud desktop transmission protocol corresponding to the target quality transmission curve segments is identified.

[0170] Obtain the pixel variation range corresponding to the target quality transmission curve segment, establish the correspondence between the pixel variation range and the cloud desktop transmission protocol, and obtain the pixel variation range-protocol lookup table;

[0171] Obtain the current cloud desktop frame to be transmitted, and determine whether the current cloud desktop frame to be transmitted is a video frame;

[0172] If the cloud desktop frame to be transmitted is not a video frame, then the Spice protocol is used to transmit the cloud desktop image and the image transmission quality is measured.

[0173] If the current cloud desktop frame to be transmitted is a video frame, then calculate the current pixel change value of the current cloud desktop frame to be transmitted;

[0174] Based on the current pixel change value, the target transmission protocol is looked up using the pixel change range-protocol lookup table. The current cloud desktop frame to be transmitted is then transmitted according to the target transmission protocol, and the image transmission quality is measured.

[0175] Determine whether the image transmission quality is up to standard;

[0176] If the current image transmission quality is substandard, then the pre-built PCoIP protocol will be used for image transmission.

[0177] If the current screen transmission quality is acceptable, then return to the steps described above for obtaining the current cloud desktop frame to be transmitted, thus completing the Android-based cloud desktop connection multi-protocol automatic switching method.

[0178] In the several embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0179] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0181] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for automatically switching protocols for Android-based cloud desktop connection, characterized in that, The method comprises: acquiring a test video frame and a dynamic region size demarcation sequence, demarcating a dynamic change region in the test video frame in sequence according to the dynamic region size demarcation sequence to obtain an initial video frame sequence; disordering pixel positions in the dynamic change region in the initial video frame sequence to obtain a target video frame sequence; performing cloud desktop picture transmission on the target video frame sequence by using a pre-constructed ICA protocol and a Spice protocol respectively, and measuring time-picture transmission quality curves of the ICA protocol and the Spice protocol; calculating a pixel change value of each target video frame in the target video frame sequence by using a pre-constructed pixel change formula, and performing position demarcation on the time-picture transmission quality curves by using the pixel change value to obtain pixel change-picture transmission quality curves of the ICA protocol and the Spice protocol; extracting target quality transmission curve segments in the pixel change-picture transmission quality curves of the ICA protocol and the Spice protocol in sequence, and identifying cloud desktop transmission protocols corresponding to the target quality transmission curve segments; acquiring a pixel change interval corresponding to the target quality transmission curve segment, establishing a corresponding relationship between the pixel change interval and the cloud desktop transmission protocol, and obtaining a pixel change interval-protocol comparison table; acquiring a current to-be-transmitted cloud desktop frame; judging whether the current to-be-transmitted cloud desktop frame is a video frame; if the current to-be-transmitted cloud desktop frame is not a video frame, performing cloud desktop picture transmission by using the Spice protocol, and measuring picture transmission quality; if the current to-be-transmitted cloud desktop frame is a video frame, calculating a current pixel change value of the current to-be-transmitted cloud desktop frame; according to the current pixel change value, querying a target transmission protocol from the pixel change interval-protocol comparison table, transmitting the current to-be-transmitted cloud desktop frame according to the target transmission protocol, and measuring picture transmission quality; judging whether the picture transmission quality is qualified; if the current picture transmission quality is not qualified, performing picture transmission by using a pre-constructed PCoIP protocol; if the current picture transmission quality is qualified, returning to the step of acquiring the current to-be-transmitted cloud desktop frame, and completing an Android-based cloud desktop connection multi-protocol automatic switching method. 2.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 1, wherein, The acquiring of the test video frame and the dynamic region size demarcation sequence comprises: acquiring an initial video frame, setting pixel points in the initial video frame to a preset pixel value to obtain a test video frame; acquiring a dynamic size sequence, and constructing the dynamic region size demarcation sequence according to the dynamic size sequence. 3.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 1, wherein, The measuring of the time-picture transmission quality curves of the ICA protocol and the Spice protocol comprises: respectively measuring picture transmission index score sets of the ICA protocol and the Spice protocol in cloud desktop picture transmission; calculating picture transmission quality of the ICA protocol and the Spice protocol in cloud desktop picture transmission according to the picture transmission index score sets; acquiring a measurement time, and drawing the time-picture transmission quality curves of the ICA protocol and the Spice protocol according to a corresponding relationship between the picture transmission quality and the measurement time. 4.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 3, wherein, The picture transmission quality of the ICA protocol and the Spice protocol in cloud desktop picture transmission is calculated according to the picture transmission index score set, and the picture transmission quality of the ICA protocol and the Spice protocol in cloud desktop picture transmission is calculated according to the picture transmission index score set, including: The picture transmission index weight is obtained; The picture transmission index score set is weighted and calculated according to the picture transmission index weight, and the picture transmission quality of the ICA protocol and the Spice protocol in cloud desktop picture transmission is obtained. 5.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 1, wherein, The pixel change calculation formula is as follows: ； wherein, represents a pixel change value, represents a serial number of a pixel point in a target video frame, represents a total number of pixel points in the target video frame, represents a pixel value of a pixel point in a next video frame of the target video frame, represents a pixel value of a pixel point in a target video frame, represents a pixel value of a pixel point in a target video frame, represents a pixel value of a pixel point in a target video frame, represents a transmission frame rate of the target video frame sequence. 6.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 1, wherein, The pixel change value is used to calibrate the position of the time-picture transmission quality curve, and the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol is obtained, including: The picture transmission quality coordinate points of the target video frame in the time-picture transmission quality curve are identified in sequence; The measurement time point corresponding to the picture transmission quality coordinate point is identified; The pixel change value of the target video frame is obtained, and the measurement time point is replaced by the pixel change value to obtain the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol. 7.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 6, wherein, The picture transmission quality coordinate points of the target video frame in the time-picture transmission quality curve are identified in sequence, including: The video frame serial number of the target video frame in the target video frame sequence is obtained; The measurement time of the target video frame is calculated by using the transmission frame rate according to the video frame serial number by using the pre-constructed calibration formula; The picture transmission quality coordinate points of the target video frame in the time-picture transmission quality curve are identified according to the measurement time of the target video frame. 8.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 7, wherein, The calibration formula is as follows: ； wherein, represents a measurement time of a target video frame, represents a measurement start time, represents a video frame number, represents a transmission frame rate. 9.The Android-based cloud desktop connection multi-protocol automatic switching method of claim 1, wherein, The target quality transmission curve segment is extracted in the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol in sequence, including: It is judged whether the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol has an intersection point; If the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol does not have an intersection point, the upper pixel change-picture transmission quality curve is obtained, and the upper pixel change-picture transmission quality curve is taken as the target quality transmission curve segment; If the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol has an intersection point, the pixel change-picture transmission quality curve of the ICA protocol and the Spice protocol is segmented at the intersection point, and the upper pixel change-picture transmission quality curve between each two intersection points is identified to obtain an upper pixel change-picture transmission quality curve set; The target quality transmission curve segment is extracted in the upper pixel change-picture transmission quality curve set in sequence.

10. An Android-based cloud desktop connection multi-protocol automatic switching system, characterized in that, The system comprises: A target video frame sequence acquisition module is used to obtain a test video frame and a dynamic region size demarcation sequence, demarcate a dynamic change region for the test video frame in sequence according to the dynamic region size demarcation sequence, obtain an initial video frame sequence, and obtain a target video frame sequence by disordering the pixel positions in the dynamic change region in the initial video frame sequence. A pixel change-picture transmission quality curve calibration module is configured to perform cloud desktop picture transmission on the target video frame sequence respectively by using pre-constructed ICA protocol and Spice protocol, and measure time-picture transmission quality curves of the ICA protocol and the Spice protocol; calculate pixel change values of each target video frame in the target video frame sequence by using a pre-constructed pixel change formula, calibrate positions of the time-picture transmission quality curves by using the pixel change values, and obtain pixel change-picture transmission quality curves of the ICA protocol and the Spice protocol; A pixel change interval-protocol correspondence table construction module is configured to extract target quality transmission curve segments from the pixel change-picture transmission quality curves of the ICA protocol and the Spice protocol in sequence, identify cloud desktop transmission protocols corresponding to the target quality transmission curve segments, obtain pixel change intervals corresponding to the target quality transmission curve segments, establish a correspondence between the pixel change intervals and the cloud desktop transmission protocols, and obtain a pixel change interval-protocol correspondence table; A video frame identification module is configured to obtain a current to-be-transmitted cloud desktop frame, and determine whether the current to-be-transmitted cloud desktop frame is a video frame. A current to-be-transmitted cloud desktop frame classification transmission module is configured to, if the current to-be-transmitted cloud desktop frame is not a video frame, perform cloud desktop picture transmission by using the Spice protocol and measure picture transmission quality; if the current to-be-transmitted cloud desktop frame is a video frame, calculate a current pixel change value of the current to-be-transmitted cloud desktop frame; according to the current pixel change value, query a target transmission protocol from the pixel change interval-protocol correspondence table, transmit the current to-be-transmitted cloud desktop frame according to the target transmission protocol, and measure picture transmission quality; determine whether the picture transmission quality is qualified; if the current picture transmission quality is not qualified, perform picture transmission by using a pre-constructed PCoIP protocol; and if the current picture transmission quality is qualified, return to the step of obtaining the current to-be-transmitted cloud desktop frame.

Images