Remote desktop access method, device and equipment, medium and program product

Abstract

The invention discloses a remote desktop access method, device and equipment, a medium and a program product, and relates to the field of remote desktops, accessed equipment encodes a desktop image and then transmits the desktop image to a remote desktop agent, and the remote desktop agent directly transmits the acquired image data to a user side. And after the user side decodes the image data by itself, through a local browser of the user side, calling a graphic rendering tool of the user side to carry out image rendering so as to obtain a remote desktop image of the accessed equipment. The remote desktop agent no longer undertakes complex image rendering work in remote control and only forwards and transmits the image data, and the user side performs image rendering automatically, so that the load of the remote desktop agent is reduced, the session concurrency of the remote desktop agent is improved, and the user experience is improved. Therefore, the remote desktop agent can process the remote desktop requests of the multiple users at the same time, the anomalies such as use lagging caused by user concurrence are avoided, and meanwhile the hardware cost of the remote desktop agent is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote desktop, in particular to a remote desktop access method, device, equipment, medium and program product. BACKGROUND

[0002] With the continuous development of network technology, remote desktop technology and RDP (Remote Desktop Protocol) are applied more and more widely, and at present, the remote control of the local user to the remote server mainly depends on the RDP agent, and the core point is that the RDP agent obtains the desktop image data of the remote server through the communication between the RDP agent and the remote server, and then the RDP agent renders the desktop image data of the remote server into an image, and then transmits the rendered image to the user end, and the user end directly displays the rendered image to the user. However, this way requires the RDP agent to undertake the image rendering work, and the image rendering needs to process a large amount of data and has high complexity, so the RDP agent is difficult to process the remote desktop requests of multiple users at the same time; when multiple users concurrently request remote desktop, the RDP agent will have abnormal situations such as use lag, which affects the user experience. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a remote desktop access method, device, equipment, medium and program product, which reduces the load of the remote desktop agent and improves the session concurrency of the remote desktop agent. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a remote desktop access method applied to a user end, comprising:

[0005] obtaining target image data sent by a remote desktop agent; the target image data is image data obtained by encoding a desktop image of an accessed device based on a preset encoding mode;

[0006] decoding the target image data by using a preset decoding mode to obtain decoded data; wherein the preset decoding mode is a decoding mode matched with the preset encoding mode;

[0007] rendering the decoded data by calling a graphics rendering tool through a local browser, and displaying the rendered image, so as to complete the remote desktop access to the accessed device.

[0008] Optionally, before obtaining the target image data sent by the remote desktop agent, the method further comprises:

[0009] acquire first data sent by the remote desktop agent; wherein the first data is data obtained by encrypting the target image data by the remote desktop agent based on a preset encryption mode;

[0010] Correspondingly, acquiring target image data sent by a remote desktop agent comprises:

[0011] decrypting the first data by using a preset decryption mode to obtain the target image data; wherein the preset decryption mode is a decryption mode matched with the preset encryption mode.

[0012] Optionally, displaying the rendered image comprises:

[0013] displaying the rendered image on a preset page of a local browser; wherein the preset page supports adjustment of the size of a displayed picture.

[0014] Correspondingly, rendering the decoded data by invoking a graphics rendering tool of a local browser comprises:

[0015] invoking a web graphics library integrated in the browser through the preset page;

[0016] controlling the web graphics library to run on the local browser to render the decoded data based on the web graphics library after running.

[0017] Optionally, rendering the decoded data based on the web graphics library after running comprises:

[0018] processing the decoded data based on the web graphics library after running and by using a graphics processing unit in the user terminal to obtain a rendered image.

[0019] wherein the graphics processing unit comprises a first processing unit for executing a target computing task in parallel and a second processing unit for executing an image rendering operation; the target computing task is a computing task other than the image rendering operation in the process of processing the decoded data based on the web graphics library.

[0020] Optionally, decoding and processing the target image data by using a preset decoding mode comprises:

[0021] invoking a graphics processing unit of the user terminal through an application programming interface;

[0022] controlling the graphics processing unit to decode and process the target image data by using a preset decoding mode.

[0023] In a second aspect, the application further discloses another remote desktop access method, applied to a remote desktop agent, comprising:

[0024] obtaining target image data sent by the accessed device; wherein the target image data is image data obtained by encoding a desktop image of the accessed device based on a preset encoding mode;

[0025] sending the target image data to a user terminal, so that the user terminal decodes the target image data by using a preset decoding mode, renders the decoded data by using a local browser to call a graphics rendering tool, and displays the rendered image, so as to complete remote desktop access to the accessed device;

[0026] wherein the preset decoding mode is a decoding mode matched with the preset encoding mode.

[0027] Optionally, obtaining the target image data sent by the accessed device comprises:

[0028] establishing a communication connection with the accessed device based on a remote desktop protocol;

[0029] calling a preset function to enable a graphics acceleration transmission function of a graphics module in the communication connection; wherein the graphics acceleration transmission function is a graphics acceleration transmission function based on the preset encoding mode;

[0030] obtaining the target image data sent by the accessed device through the communication connection.

[0031] Optionally, establishing the communication connection with the accessed device based on the remote desktop protocol comprises:

[0032] loading a preset protocol library; wherein the preset protocol library is a protocol library using the remote desktop protocol;

[0033] configuring configuration parameters of the preset protocol library to a preset state; wherein the preset state indicates that the preset protocol library uses the preset encoding mode to transmit data in a session;

[0034] negotiating with the accessed device through the configured preset protocol library, so as to establish the communication connection with the accessed device.

[0035] Optionally, before sending the target image data to the user terminal, the method further comprises:

[0036] obtaining a format identifier of the target image data;

[0037] determining whether the target image data is data encoded based on the preset encoding mode according to the format identifier;

[0038] If the target image data is data processed based on the preset encoding mode, the step of sending the target image data to the user terminal is performed.

[0039] Optionally, before the target image data is sent to the user terminal, the method further includes:

[0040] The target image data is encapsulated by using a preset private protocol to obtain encapsulated data.

[0041] The target image data is sent to the user terminal, including:

[0042] The encapsulated data is transmitted to the user terminal.

[0043] Optionally, the encapsulated data is transmitted to the user terminal, including:

[0044] The encapsulated data between the remote desktop agent and the user terminal is encrypted by using an encryption protocol integrated with Nginx reverse proxy, so as to transmit the encapsulated data to the user terminal by an encrypted transmission mode.

[0045] In a third aspect, the application further discloses a remote desktop access device applied to a user terminal and including:

[0046] An image acquisition unit is configured to acquire target image data sent by a remote desktop agent, wherein the target image data is image data obtained by encoding a desktop image of an accessed device based on a preset encoding mode.

[0047] A decoding unit is configured to decode the target image data by using a preset decoding mode to obtain decoded data, wherein the preset decoding mode is a decoding mode matched with the preset encoding mode.

[0048] A rendering unit is configured to render the decoded data by calling a graphics rendering tool through a local browser, display the rendered image, and complete remote desktop access to the accessed device.

[0049] In a fourth aspect, the application further discloses an electronic device including a processor and a memory, wherein the memory is configured to store a computer program, the computer program is loaded and executed by the processor to implement the remote desktop access method as described above.

[0050] In a fifth aspect, the application further discloses a computer readable storage medium configured to store a computer program, the computer program is executed by a processor to implement the remote desktop access method as described above.

[0051] In a sixth aspect, the present application also discloses a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the remote desktop access method described above.

[0052] The present application discloses a remote desktop access method, device, equipment, medium and program product. A desktop image is encoded by an accessed equipment and then transmitted to a remote desktop agent. The remote desktop agent directly transmits the obtained target image data to a user end. The target image data is decoded by the user end, and then the remote desktop image of the accessed equipment is obtained by calling a graphic rendering tool of a browser of the user end to perform image rendering. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.

[0054] Figure 1 A flowchart of a remote desktop access method provided by the present application is shown in FIG. 1.

[0055] Figure 2 An assembly interaction diagram during remote desktop access provided by the present application is shown in FIG. 2.

[0056] Figure 3 A flowchart of another remote desktop access method provided by the present application is shown in FIG. 3.

[0057] Figure 4 A structural diagram of a remote desktop access device provided by the present application is shown in FIG. 4.

[0058] Figure 5 A structural diagram of an electronic equipment provided by the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0059] The core of the present application is to provide a remote desktop access method, device, equipment, medium and program product, so that the remote desktop agent can process multiple user remote desktop requests at the same time, avoid the use of cardon and other abnormalities caused by user concurrency, and reduce the hardware cost of the remote desktop agent.

[0060] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0061] Referring to Figure 1 As shown in the figure, Figure 1 A flowchart of a remote desktop access method provided by the present application is shown. In a first aspect, the present application discloses a remote desktop access method applied to a user terminal, which comprises the following steps:

[0062] S11: Obtain target image data sent by a remote desktop agent; the target image data is image data obtained by encoding a desktop image of the accessed device based on a preset encoding mode;

[0063] It can be understood that, in order to improve the security during remote desktop access, the user terminal needs to use the remote desktop agent to realize remote desktop access to the accessed device. The remote desktop agent, as an intermediate node, can hide the real IP (Internet Protocol) and port of the user terminal and the accessed device, avoid security risks such as being attacked caused by directly opening the port of the user terminal and the accessed device, and realize cross-network access between the user terminal and the accessed device by means of the remote desktop agent. Therefore, if the user terminal wants to realize remote desktop access to the accessed device, it needs to use the remote desktop agent to obtain the target image data of the accessed device.

[0064] S12: Decode the target image data by using a preset decoding mode to obtain decoded data; wherein the preset decoding mode is a decoding mode matched with the preset encoding mode;

[0065] Meanwhile, in order to ensure the cross-device compatibility of the remote desktop, the data transmission of the remote desktop needs to be performed by using the encoding and decoding operations, and thus after the accessed device establishes the remote access connection with the user end through the remote desktop agent, the accessed device encodes the desktop image thereof in a preset encoding mode, and transmits the desktop screen content and screen content changes of the accessed device to the remote desktop agent in real time through the target image data obtained by the encoding processing. In order to reduce the load and hardware cost of the remote desktop agent, after the remote desktop agent obtains the target image data transmitted by the accessed device, the remote desktop agent directly forwards the target image data to the user end. After the user end receives the target image data, the user end decodes the target image data by using a preset decoding mode matched with the preset encoding mode, to obtain the decoded data representing the desktop image. By using the encoding and decoding mode, the data transmission amount can be effectively reduced, and the transmission efficiency during the remote desktop access can be improved. The preset decoding mode is matched with the preset encoding mode, which means that the decoding process based on the preset decoding mode can accurately identify and reversely execute the algorithm, format and rules adopted by the encoding process corresponding to the preset encoding mode, and finally the compressed data after the encoding can be completely restored to the original data.

[0066] S13: rendering the decoded data by using a graphic rendering tool through the local browser, and displaying the rendered image, to complete the remote desktop access to the accessed device.

[0067] Further, the decoded data is still in the form of data, and thus the user end still needs to further convert the decoded data into visual images by using the rendering, and thus after the user end completes the decoding of the target image data, the user end still needs to further render the decoded data to obtain the rendered visual images. Considering that the user end in the present application is a client allowing a user to directly perform the remote access and remote control through the local web browser of the user end, the remote desktop access to the accessed device is also a WebRDP implemented based on the local browser of the user end, and thus the user end needs to call a graphic rendering tool through the local browser to complete the rendering operation. The WebRDP is implemented in the Web (webpage) interface of the browser to access the remote device, and does not need to use the terminal software such as the RDP client software of the local computer of the user. The specific implementation mode of the local browser of the user end is not particularly limited in the present application, and can be a Chrome browser.

[0068] It should be noted that the specific types and implementation manners of the remote desktop agent, the accessed device and the user terminal are not particularly limited in the present application, and the present application is mainly illustrated based on the remote desktop based on the RDP protocol. Correspondingly, the remote desktop agent is implemented by the RDP agent, the accessed device can be various types of servers or computers, and the user terminal can be determined according to the actual device used by the user. The specific types and implementation manners of the preset encoding mode and the preset decoding mode also have multiple choices, and the present application does not particularly limit them, and they can be implemented by H. 264 encoding and corresponding H. 264 decoding. The specific types and implementation manners of the image rendering tool are not particularly limited in the present application, and can be selected according to the actual situation of the user terminal. The specific display mode of the rendered image is not particularly limited in the present application. The communication mode between the remote desktop agent and the accessed device and the communication mode between the remote desktop agent and the user terminal are not particularly limited in the present application. The remote desktop agent can establish communication with the user terminal by the websocket protocol or the webtransport protocol.

[0069] The present application provides a high-performance WebRDP implementation scheme, which solves the performance bottleneck of the RDP agent caused by image rendering in the implementation of the traditional remote desktop. The remote desktop agent no longer undertakes complex image rendering work, and does not need to set the rendering component and the data conversion component in the remote desktop agent, thereby effectively reducing the load and hardware cost of the RDP agent. The decoding and rendering of the image data are implemented by the local browser of the user terminal, which can accelerate the decoding and / or rendering by means of the GPU (Graphics Processing Unit, graphics processor) of the user terminal, so that the user terminal can use the WebRDP more smoothly, and the concurrent quantity of the RDP session is improved. By reconstructing the image rendering mode in the implementation of the WebRDP, the concurrent user number and the use experience of the WebRDP are greatly improved, and the user terminal can smoothly play the video and the scrolling picture of the remote desktop.

[0070] As an optional embodiment, before obtaining the target image data sent by the remote desktop agent, the method further comprises:

[0071] obtaining first data sent by the remote desktop agent; wherein the first data is obtained by encrypting the target image data by the remote desktop agent based on a preset encryption mode;

[0072] Correspondingly, obtaining the target image data sent by the remote desktop agent comprises:

[0073] The first data is decrypted by using a preset decryption mode to obtain target image data; wherein the preset decryption mode is a decryption mode matched with the preset encryption mode.

[0074] It is understandable that, in order to ensure the security in the process of data transmission, especially when the accessed device and the user end are in different network environments, it is necessary to ensure the security of data transmission. Therefore, when the remote desktop agent sends the target image data to the user end, the target image data will be further encrypted to obtain the first data. After the user end receives the first data, the corresponding decryption mode is used to decrypt the first data to obtain the decrypted target image data, and then further decoding processing is performed. The preset decryption mode is matched with the preset encryption mode, which means that the decryption process based on the preset decryption mode can accurately reverse the algorithm, key and rule used in the encryption process corresponding to the preset encryption mode, so as to correctly restore the encrypted ciphertext (first data) to the original plaintext (target image data). The specific type and implementation of the preset encryption mode and the preset decryption mode are not particularly limited in this application, and can be implemented by using TLS (Transport Layer Security) encryption and the like.

[0075] As a specific embodiment, refer to Figure 2 shown in the figure, Figure 2A schematic diagram of component interaction in remote desktop access provided by the application; the accessed device is an RDP server in an intranet, an RDP agent is used as a remote desktop agent, and the user end is a user's local client browser in a public network. The RDP agent uses a FreeRDP library, enables GFX connection based on the RDP protocol, and obtains target image data based on the GFX connection. Meanwhile, a websocket connection is established between the RDP agent and the local browser of the user end, thereby establishing a remote access connection between the user end and the accessed device. In order to ensure the security of subsequent data transmission in the public network, the RDP agent performs encryption at the transmission layer through Nginx reverse proxy, and implements a preset encryption mode by combining websocket with TLS encryption to encrypt websocket communication at the application layer. The websocket combined with TLS encryption adds a TLS / SSL (Secure Sockets Layer) layer based on the standard websocket protocol, ensuring the confidentiality and integrity of the data transmission process. In the implementation scheme of WebRDP provided by the application, taking H.264 encoding as an example of a preset encoding mode, the target image data is specifically H.264 encoded video stream, and the RDP agent transmits the screen content of the RDP server to the local browser of the user end through the H.264 encoded video stream mode, and then the user end calls the local GPU for hardware accelerated decoding and rendering by using the front-end code on the browser.

[0076] Specifically, by adding a data encryption process in the data transmission process between the remote desktop agent and the user end, the security and reliability of the data transmission process are ensured, and in particular, the data security when using WebRDP in a public network is ensured.

[0077] As an optional embodiment, the rendered image is displayed, including:

[0078] The rendered image is displayed on a preset page of the local browser; wherein the preset page supports adjustment of the display size of the picture;

[0079] Correspondingly, the decoded data is rendered by calling a graphics rendering tool through the local browser, including:

[0080] The internal integrated web graphics library of the browser is called through the preset page;

[0081] The web graphics library is controlled to run in the local browser, so as to render the decoded data based on the running web graphics library.

[0082] It can be understood that, in order to realize the WebRDP, a preset page is pre-configured in the browser of the user terminal to display the rendered image representing the remote desktop image. After the target image data is decoded to obtain the decoded data, the user terminal can call the WebGL (Web Graphics Library) integrated in the browser through the preset page to perform image rendering using the WebGL. The browser of the user terminal integrates a WebGL engine. When the browser calls the WebGL by loading and executing corresponding WebGL code, the WebGL engine runs in the browser and renders the decoded data into an image, thereby realizing access to the remote desktop image. The specific type and implementation of the preset page and the web graphics library are not particularly limited in the present application. The preset page can be configured to support a page that supports display size adjustment, and can also be configured to support various display functions such as full screen display.

[0083] Specifically, the WebGL can effectively realize the rendering of the encoded data, and does not require a plug-in. The WebGL can be directly run as long as the browser of the user terminal supports the WebGL, has strong compatibility, and can directly realize graphic rendering in the web page of the browser in an efficient and cross-platform manner, and is easy to implement.

[0084] As an optional embodiment, the rendering of the decoded data based on the running web graphics library includes:

[0085] rendering the decoded data based on the running web graphics library and using a graphic processing unit in the user terminal to obtain a rendered image.

[0086] In the graphic processing unit, a first processing unit for parallel execution of a target computing task and a second processing unit for execution of an image rendering operation are included. The target computing task is a computing task other than the image rendering operation in the process of processing the decoded data based on the web graphics library.

[0087] It is understandable that WebGL can directly call GPU for calculation and / or rendering, so as to utilize the GPU hardware of the user side to realize acceleration of the rendering operation. Specifically, WebGL can call the first processing unit in the GPU to complete some other calculation tasks except the image rendering operation when the renderer in WebGL renders, such as mathematical operation of data or code calculation, so as to improve the rendering efficiency of the renderer in WebGL by utilizing the parallel computing capability of the GPU. On the other hand, WebGL can call the second processing unit in the GPU to complete part of the image rendering operation. For example, WebGL divides the rendering operation of the target image data into several rendering sub-operations, wherein a part of the rendering sub-operations are completed by the renderer in WebGL, and the other part is directly completed by the second processing unit in the GPU. The specific type and implementation manner of the first processing unit and the second processing unit are not particularly limited in the present application. The first processing unit refers to the computing core of the GPU, that is, the processor core in the GPU responsible for executing the calculation task, which is used to realize the target calculation task assigned by WebGL to the GPU. The second processing unit mainly includes the fixed function hardware for image rendering, such as rasterizer or texture unit, etc.

[0088] Specifically, when WebGL is used for image rendering, the rendering and / or calculation can be further performed by calling the GPU of the user side, so as to effectively improve the efficiency of image rendering, accelerate the rendering completion, realize high-efficiency rendering performance by utilizing the GPU acceleration, and ensure the fluency during remote desktop access.

[0089] As an optional embodiment, the target image data is decoded by using a preset decoding manner, including:

[0090] The application programming interface is used to call the GPU of the user side;

[0091] The GPU is controlled to decode the target image data by using the preset decoding manner.

[0092] It can be understood that after the encrypted first data is received by the browser of the user end, the target image data is first decrypted, and then the target image data can be decoded. When decoding, the front-end code on the local browser of the user end can be run to call the GPU of the user end through the WebCodecs API (Web Codecs Application Programming Interface) or other types of application programming interfaces, send the received target image data to the decoder in the GPU using the preset decoding mode, and thus utilize the GPU to realize hardware decoding acceleration. The specific type and implementation of the application programming interface are not particularly limited by the present application, and are not limited to the WebCodecs API. The WebCodecs API is a set of APIs provided by the browser for directly operating the encoding and decoding process of audio and video.

[0093] Specifically, by calling the local GPU of the user end for hardware decoding acceleration, the decoding efficiency can be greatly improved by utilizing the special hardware modules in the GPU and the parallel processing capability of the GPU, while reducing the CPU (Central Processing Unit) load of the user end, reducing the delay and overhead caused by the participation of the CPU and data transfer.

[0094] Referring to Figure 3 As shown in the drawings, Figure 3 The present application provides another remote desktop access method flow chart; secondly, the present application also discloses another remote desktop access method, applied to remote desktop agent; comprising:

[0095] S21: obtaining target image data sent by the accessed device; wherein the target image data is image data obtained by encoding the desktop image of the accessed device based on a preset encoding mode;

[0096] S22: sending the target image data to the user end, so that the user end decodes the target image data using a preset decoding mode, calling a graphics rendering tool through a local browser to render the decoded data, and displaying the rendered image to complete the remote desktop access to the accessed device; wherein the preset decoding mode is a decoding mode matched with the preset encoding mode.

[0097] For another remote desktop access method provided by the present application, please refer to the above-mentioned remote desktop access method, and the present application will not be repeated here.

[0098] As an optional embodiment, obtaining the target image data sent by the accessed device comprises:

[0099] establishing a communication connection with the accessed device based on a remote desktop protocol;

[0100] calling a preset function to enable a graphics acceleration transmission function of a graphics module in the communication connection; wherein the graphics acceleration transmission function is a graphics acceleration transmission function based on a preset encoding mode;

[0101] obtaining target image data sent by the accessed device through the communication connection.

[0102] It can be understood that the remote desktop agent first needs to establish a communication connection with the accessed device based on the RDP protocol in order to remotely access the accessed device from the user end. On this basis, a preset function can be further called to start the graphics acceleration transmission function in the RDP protocol. Specifically, the preset function can use the freerdp_settings_set_bool function to start the GFX (RemoteFX graphics) extension module (i.e. the graphics module) in the RDP protocol by configuring a specific configuration item in the freerdp_settings_set_bool function, thereby establishing a GFX connection between the remote desktop agent and the accessed device, enabling the graphics acceleration transmission function in the RDP protocol to achieve the acquisition of target image data.

[0103] Specifically, by enabling the graphics acceleration transmission function in the RDP protocol, efficient graphics instructions can be transmitted using the GFX connection, which can greatly reduce bandwidth occupation, reduce delay, improve the response speed of remote desktop access, and improve user experience compared to the bitmap transmission method in the traditional scheme.

[0104] As an optional embodiment, establishing a communication connection with the accessed device based on a remote desktop protocol comprises:

[0105] loading a preset protocol library; wherein the preset protocol library is a protocol library using a remote desktop protocol;

[0106] configuring configuration parameters of the preset protocol library to a preset state; wherein the preset state indicates that the preset protocol library uses a preset encoding mode to transmit data in a session;

[0107] negotiating with the accessed device through the completed preset protocol library to establish a communication connection with the accessed device.

[0108] It can be understood that the remote desktop agent can specifically establish a communication connection with the accessed device by loading a preset protocol library, such as Figure 2As shown, the RDP agent uses the FreeRDP library to establish a GFX connection with the RDP server. Then the setting of the preset encoding mode is realized by configuring the configuration parameters of the preset protocol library, and finally a communication connection is established between the accessed device based on the configured preset protocol library. The configuration of the configuration parameters of the preset protocol library can also be implemented by reusing the freerdp_settings_set_bool function, which is an API function for configuring the FreeRDP library. For example, in the case of using H.264 encoding as the preset encoding mode, the freerdp_settings_set_bool function is configured as freerdp_settings_set_bool (settings, FreeRDP_GfxH264, GfxH264) through the API interface provided by the FreeRDP library to enable the H.264 codec, so as to configure the preset encoding mode as H.264 encoding. After successful configuration, the changes of the desktop screen content of the RDP server will be transmitted to the RDP agent in the form of H.264 encoded video stream. The specific negotiation method between the remote desktop agent and the accessed device is not particularly limited in the present application, and can be realized by handshaking and the like.

[0109] Specifically, the establishment of the communication connection with the accessed device and the configuration of the preset encoding mode can be realized by loading and configuring the preset protocol library, which is simple, effective and easy to implement.

[0110] As an optional embodiment, before the target image data is sent to the user end, the method further comprises:

[0111] Obtaining a format identifier of the target image data;

[0112] Determining whether the target image data is data processed based on the preset encoding mode according to the format identifier;

[0113] If the target image data is data processed based on the preset encoding mode, jumping to the step of sending the target image data to the user end.

[0114] It is not difficult to understand that, in order to ensure the accuracy and reliability of the target image data, the remote desktop agent can further obtain the format identifier carried by the target image data after receiving the target image data, and further determine whether the target image data is encoded data obtained by using the preset encoding mode through the format identifier, and determine whether the format identifier carried by the target image data represents the preset encoding mode through the comparison of the format identifiers. The specific implementation of the format identifier is not particularly limited in the present application, such as Figure 2As shown, the RDP agent can parse the H.264 data sent by the RDP server (the accessed device) through the FreeRDP library, the library code of the FreeRDP library parses the public header of the data packet of the target image data, and then identifies the codecId field in the RDPGFX_SURFACE_COMMAND* cmd type (a structure instance). The codecId field is a parameter for identifying the encoding format of the graphic data. If the codecId field is of the RDPGFX_CODECID_AVC420 type, it indicates that the target image data is data processed based on the preset encoding manner.

[0115] Specifically, the target image data is further ensured to be data processed based on the preset encoding manner before being sent to the user end, so that the user end can accurately and effectively decode the decoded data representing the desktop image of the accessed device by using the preset decoding manner, and the accuracy and reliability of the remote desktop access are improved.

[0116] As an optional embodiment, before the target image data is sent to the user end, the method further includes:

[0117] The target image data is encapsulated by using a preset private protocol to obtain encapsulated data.

[0118] The target image data is sent to the user end, including:

[0119] The encapsulated data is transmitted to the user end.

[0120] It can be understood that after it is determined that the target image data is data processed based on the preset encoding manner, the target image data can be further encapsulated before being sent to the user end. On the basis of the public protocol (such as the websocket protocol) used when the communication between the remote desktop agent and the user end is established, a layer of self-defined communication rule with non-public protocol format is further established, and the security of the transmission is further improved. The remote desktop agent encapsulates the target image data into a private protocol, and then further transmits it to the user end through the websocket protocol and the like. The specific implementation manner of the preset private protocol is not particularly limited in the present application, and can be set according to actual application scenarios and the like.

[0121] Specifically, by further encapsulating to enhance the transmission security, the compatibility of the transmission data can also be effectively improved, and the stability and reliability of the entire remote desktop access are ensured.

[0122] As an optional embodiment, the encapsulated data is transmitted to the user end, including:

[0123] The encrypted protocol integrated by the Nginx reverse proxy is used to encrypt the encapsulated data between the remote desktop agent and the user end, so that the encapsulated data is transmitted to the user end through the encrypted transmission mode.

[0124] It is understandable that, after the remote desktop agent completes the secondary encapsulation of the target image data, the remote desktop agent can further use the Nginx reverse proxy to encrypt and protect the TCP (Transmission Control Protocol) connection between the remote desktop agent and the user end at the transmission layer in addition to using the preset encryption mode to encrypt the target image data. The preset encryption mode can be used to realize the comprehensive encryption protection of the transmission layer and the application layer. The specific type of the Nginx reverse proxy and the specific implementation mode of the integrated encryption protocol are not particularly limited in the application.

[0125] Specifically, the encryption processing flow of the transmission layer is further increased by the Nginx reverse proxy, so as to ensure the security and reliability of the data transmission process, and in particular, the data security when the WebRDP is used in the public network.

[0126] Referring to Figure 4 , Figure 4 A structure schematic diagram of a remote desktop access device provided by the application is shown; in a third aspect, the application further discloses a remote desktop access device applied to a user end, which comprises:

[0127] An image acquisition unit 11 is configured to acquire target image data sent by a remote desktop agent; the target image data is image data obtained by encoding a desktop image of an accessed device based on a preset encoding mode;

[0128] A decoding unit 12 is configured to decode the target image data by using a preset decoding mode, so as to obtain decoded data; wherein the preset decoding mode is a decoding mode matched with the preset encoding mode;

[0129] A rendering unit 13 is configured to render the decoded data by calling a graphic rendering tool through a local browser, and display the rendered image, so as to complete the remote desktop access to the accessed device.

[0130] As an optional embodiment, the application further comprises:

[0131] A first encryption unit is configured to acquire first data sent by the remote desktop agent; wherein the first data is data obtained by encrypting the target image data based on a preset encryption mode by the remote desktop agent;

[0132] Correspondingly, the image acquisition unit 11 comprises:

[0133] The first decryption unit is configured to decrypt the first data by using a preset decryption mode to obtain target image data, wherein the preset decryption mode is a decryption mode matched with the preset encryption mode.

[0134] As an optional embodiment, the rendering unit 13 comprises:

[0135] The display unit is configured to display the rendered image on a preset page of the local browser, wherein the preset page supports adjustment of the display screen size.

[0136] The rendering calling unit is configured to call a web graphics library integrated in the browser through the preset page.

[0137] The rendering sub-unit is configured to control the web graphics library to run in the local browser, and to render the decoded data based on the running web graphics library.

[0138] As an optional embodiment, the rendering sub-unit comprises:

[0139] The rendering acceleration unit is configured to process the decoded data based on the running web graphics library and by using a graphics processing unit in the user terminal to obtain the rendered image.

[0140] In the graphics processing unit, a first processing unit is configured to execute a target computing task in parallel, and a second processing unit is configured to execute an image rendering operation; the target computing task is other computing task than the image rendering operation in the process of processing the decoded data based on the web graphics library.

[0141] As an optional embodiment, the decoding unit 12 comprises:

[0142] The decoding calling unit is configured to call a graphics processing unit of the user terminal through an application programming interface.

[0143] The decoding acceleration unit is configured to control the graphics processing unit to decode and process the target image data by using a preset decoding mode.

[0144] On the other hand, the application further discloses another remote desktop access device applied to a remote desktop agent, comprising:

[0145] The intermediate acquisition unit is configured to acquire target image data sent by an accessed device, wherein the target image data is image data obtained by encoding and processing a desktop image of the accessed device based on a preset encoding mode.

[0146] The sending unit is configured to send the target image data to the user terminal, so that the user terminal decodes the target image data by using a preset decoding mode, calls a graphic rendering tool through a local browser to render the decoded data, and displays the rendered image, so as to complete the remote desktop access to the accessed device. The preset decoding mode is a decoding mode matched with the preset encoding mode.

[0147] As an optional embodiment, the intermediate obtaining unit comprises:

[0148] The connection establishing unit is configured to establish a communication connection with the accessed device based on the remote desktop protocol.

[0149] The acceleration configuring unit is configured to call a preset function to enable a graphic acceleration transmission function of the graphic module in the communication connection. The graphic acceleration transmission function is a graphic acceleration transmission function based on the preset encoding mode.

[0150] The intermediate obtaining subunit is configured to obtain the target image data sent by the accessed device through the communication connection.

[0151] As an optional embodiment, the connection establishing unit comprises:

[0152] The loading unit is configured to load a preset protocol library. The preset protocol library is a protocol library using the remote desktop protocol.

[0153] The encoding configuring unit is configured to configure a configuration parameter of the preset protocol library into a preset state. The preset state indicates that the preset protocol library uses the preset encoding mode to transmit data in a session.

[0154] The negotiation unit is configured to negotiate with the accessed device through the configured preset protocol library, so as to establish the communication connection with the accessed device.

[0155] As an optional embodiment, the method further comprises:

[0156] The identification obtaining unit is configured to obtain a format identification of the target image data before the target image data is sent to the user terminal.

[0157] The data format determining unit is configured to determine whether the target image data is data encoded by using the preset encoding mode according to the format identification. If the target image data is the data encoded by using the preset encoding mode, the sending unit is triggered.

[0158] As an optional embodiment, the method further comprises:

[0159] The secondary packaging unit is configured to package the target image data by using a preset private protocol before the target image data is sent to the user terminal, to obtain packaged data.

[0160] The transmitting unit includes:

[0161] The transmitting subunit is used to transmit the encapsulated data to the user terminal.

[0162] As an optional embodiment, the transmitting subunit includes:

[0163] The second encryption unit is used to encrypt the encapsulated data between the remote desktop agent and the user terminal using the encryption protocol integrated with Nginx reverse proxy, so as to transmit the encapsulated data to the user terminal through encrypted transmission.

[0164] For a description of the remote desktop access device provided by the present invention, please refer to the embodiments of the remote desktop access method described above; the present invention will not be described again here.

[0165] See Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided by the present invention. In a fourth aspect, this application also discloses an electronic device, which includes a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program 222, which is loaded and executed by the processor 21 to implement the remote desktop access method as described above.

[0166] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0167] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the remote desktop access method disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be an electronic computer.

[0168] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0169] In addition, the memory 22 can be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like, and can store operating systems 221, computer programs 222, and the like. The memory 221 can be temporarily stored or permanently stored.

[0170] The operating system 221 is used to manage and control each hardware device on the electronic device and the computer program 222, and can be Windows Server, Netware, Unix, Linux, or the like. In addition to the computer program capable of completing the remote desktop access method disclosed in any of the above embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0171] The electronic device provided by the present application is described above with reference to the embodiments of the remote desktop access method, and the present application will not be described again.

[0172] In a fifth aspect, the present application further discloses a computer readable storage medium for storing a computer program, and the computer program is executed by a processor to implement the remote desktop access method as described above.

[0173] It can be understood that if the method in the above embodiments is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and executes all or part of the steps of the method described in each embodiment of the present application. Specifically, the computer readable storage medium can include, but is not limited to, any type of disk, including floppy disks, optical disks, and mobile hard disks, or any type of medium or device suitable for storing instructions, data, and the like. The present application does not make special limitations here.

[0174] The computer readable storage medium provided by the present application is described above with reference to the embodiments of the remote desktop access method, and the present application will not be described again.

[0175] In a sixth aspect, the present application further discloses a computer program product, including computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the remote desktop access method as described above.

[0176] The computer program product provided by the present application is described above with reference to the embodiments of the remote desktop access method, and the present application will not be described again.

[0177] The various embodiments described in the specification are progressive in nature, and each embodiment highlights the differences from other embodiments. The same or similar parts among the various embodiments can be mutually referred to. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method.

[0178] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or both. To clearly illustrate the interchangeability of hardware and software, the description has generally been described in terms of exemplary components and steps that can comprise functionality implemented in hardware and / or software. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation should not be interpreted to change the scope of the application.

[0179] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can be located in random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0180] Finally, it should be noted that the terms such as first and second, etc. are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a list of elements does not exclude other elements not explicitly listed, or other elements inherent in such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device that includes the element.

[0181] The above describes the technical solutions provided by the present application in detail. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In view of the above, the content of the specification should not be understood as limiting the present application.

Claims

1. A method of remote desktop access, characterized by, The application is applied to a user terminal and comprises the following steps: obtaining target image data sent by a remote desktop agent, wherein the target image data is image data obtained by encoding a desktop image of an accessed device based on a preset encoding mode; decoding the target image data based on a preset decoding mode to obtain decoded data, wherein the preset decoding mode is a decoding mode matched with the preset encoding mode; rendering the decoded data by calling a graphics rendering tool through a local browser, and displaying the rendered image to complete remote desktop access to the accessed device.

2. The remote desktop access method of claim 1, wherein, Before the step of obtaining the target image data sent by the remote desktop agent, the method further comprises the following steps: obtaining first data sent by the remote desktop agent, wherein the first data is data obtained by encrypting the target image data based on a preset encryption mode; correspondingly, the step of obtaining the target image data sent by the remote desktop agent comprises the following steps: decrypting the first data based on a preset decryption mode to obtain the target image data, wherein the preset decryption mode is a decryption mode matched with the preset encryption mode.

3. The remote desktop access method of claim 1, wherein, The step of displaying the rendered image comprises the following steps: displaying the rendered image on a preset page of the local browser, wherein the preset page supports adjustment of the size of a displayed picture; correspondingly, the step of rendering the decoded data by calling the graphics rendering tool through the local browser comprises the following steps: calling a web graphics library integrated in the browser through the preset page; controlling the web graphics library to run in the local browser to render the decoded data based on the running web graphics library.

4. The remote desktop access method of claim 3, wherein, The step of rendering the decoded data based on the running web graphics library comprises the following steps: processing the decoded data based on the running web graphics library and by using a graphics processing unit in the user terminal to obtain the rendered image; wherein the graphics processing unit comprises a first processing unit for executing a target computing task in parallel and a second processing unit for executing an image rendering operation, and the target computing task is a computing task other than the image rendering operation in the process of processing the decoded data based on the web graphics library.

5. The remote desktop access method according to any one of claims 1 to 4, wherein, The step of decoding the target image data based on the preset decoding mode comprises the following steps: calling the graphics processing unit of the user terminal through an application programming interface; controlling the graphics processing unit to decode the target image data based on the preset decoding mode.

6. A remote desktop access method characterized by, The application is applied to a remote desktop agent and comprises the following steps: obtaining target image data sent by an accessed device, wherein the target image data is image data obtained by encoding a desktop image of the accessed device based on a preset encoding mode; sending the target image data to a user terminal, so that the user terminal decodes the target image data based on a preset decoding mode, renders decoded data by calling a graphics rendering tool through a local browser, and displays the rendered image to complete remote desktop access to the accessed device. The preset decoding mode is matched with the preset encoding mode.

7. The remote desktop access method of claim 6, wherein, The target image data sent by the accessed device is acquired, including: A communication connection between the accessed device is established based on a remote desktop protocol; A preset function is called to enable a graphics acceleration transmission function of a graphics module in the communication connection; the graphics acceleration transmission function is based on the preset encoding mode; The target image data sent by the accessed device is acquired through the communication connection.

8. The remote desktop access method of claim 7, wherein, A communication connection between the accessed device is established based on a remote desktop protocol, including: A preset protocol library is loaded; the preset protocol library is a protocol library using a remote desktop protocol; Configuration parameters of the preset protocol library are configured to a preset state; the preset state indicates that the preset protocol library uses a preset encoding mode to transmit data in a session; The preset protocol library is negotiated with the accessed device to establish the communication connection between the accessed device.

9. The remote desktop access method of claim 7, wherein, Before the target image data is sent to the user end, the method further includes: An identification of a format of the target image data is acquired; It is determined whether the target image data is data processed based on the preset encoding mode according to the identification of the format; If the target image data is data processed based on the preset encoding mode, the step of sending the target image data to the user end is jumped to.

10. The remote desktop access method according to any one of claims 7 to 9, wherein, Before the target image data is sent to the user end, the method further includes: The target image data is data encapsulated using a preset private protocol to obtain encapsulated data; The target image data is sent to the user end, including: The encapsulated data is transmitted to the user end.

11. The remote desktop access method of claim 10, wherein, The encapsulated data is transmitted to the user end, including: The encapsulated data between the remote desktop agent and the user end is encrypted using an encryption protocol integrated with an Nginx reverse proxy to transmit the encapsulated data to the user end through an encrypted transmission mode.

12. A remote desktop access device, characterized by Applied to a user end, including: An image acquisition unit is configured to acquire target image data sent by a remote desktop agent; the target image data is image data obtained by encoding a desktop image of an accessed device based on a preset encoding mode; A decoding unit is configured to decode the target image data using a preset decoding mode to obtain decoded data; the preset decoding mode is matched with the preset encoding mode; A rendering unit is configured to render the decoded data using a local browser to call a graphics rendering tool and display a rendered image to complete remote desktop access to the accessed device.

13. An electronic device, comprising: The electronic device includes a processor and a memory; the memory is configured to store a computer program; the computer program is loaded and executed by the processor to implement the remote desktop access method of any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program is executed by a processor to implement the remote desktop access method of any one of claims 1 to 11.

15. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the remote desktop access method of any one of claims 1 to 11.

Images