Multi-remote desktop unified display method based on Web mode

Abstract

The invention relates to a multi-remote desktop unified display method based on a Web mode, and the method comprises the steps: constructing a remote desktop gateway based on Guacamole, and carrying out the unified packaging of different remote desktop protocols; constructing a remote protocol proxy service for converting the request parameter into a unified encapsulation protocol required by a remote desktop gateway; configuring a connection parameter and a canvas display position parameter of each remote desktop instance needing to be accessed in the browser; when each remote desktop needs to be accessed, the browser forwards the configured connection parameters of each remote desktop instance to the remote desktop gateway through the remote protocol proxy service; and after the remote desktop gateway establishes connection with the corresponding remote desktop according to the received protocol, picture frame data of the remote desktop are returned. According to the invention, remote desktop cross-platform access is realized, the working efficiency is improved, and various requirement scenes of global monitoring and interlocking operation can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of remote desktops, and in particular to a method for uniformly displaying multiple remote desktops based on a Web mode. Background Art

[0002] With the continuous development of computer and network technology, remote desktop technology has been widely used in the fields of enterprise collaboration, industrial control, intelligent operation and maintenance, etc. Through remote desktop technology, users can remotely access and operate the remote computer desktop on the local computer, realizing functions such as remote office, remote control and remote support.

[0003] At present, most remote desktop solutions on the market adopt C / S (client / server) architecture, where users need to install specific client software on their local computers and then use the client software to connect to the remote server to access and operate the remote desktop. This approach not only increases the user's installation and maintenance costs, but also limits the access methods and scenarios of the remote desktop.

[0004] In recent years, with the continuous development and popularization of Web technology, more and more applications have begun to adopt B / S (browser / server) architecture. B / S architecture has the advantages of cross-platform, easy deployment, and easy maintenance. Combining remote desktop technology with Web technology to achieve Web-based remote desktop access has become an important direction for the development of current remote desktop technology. However, most of the existing Web-based remote desktop technologies can only access and display a single remote desktop on a single page, and cannot achieve unified and integrated display of multiple remote desktops. This limits the user's management and operation of multiple remote desktops at the same time, reducing work efficiency. Therefore, how to design a technology and method based on the Web that can simultaneously display multiple remote desktops has become a problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a method for uniformly displaying multiple remote desktops based on a Web mode.

[0006] The specific plan is as follows:

[0007] A method for uniformly displaying multiple remote desktops based on a Web mode includes: building a remote desktop gateway based on Guacamole to uniformly encapsulate different remote desktop protocols; building a remote protocol proxy service to convert request parameters into a uniform encapsulation protocol required by the remote desktop gateway; configuring connection parameters and canvas display position parameters of each remote desktop instance to be accessed in a browser;

[0008] To access each remote desktop, perform the following steps:

[0009] The browser sends the connection parameters of each remote desktop instance configured to the remote protocol proxy service through different requests;

[0010] The remote protocol proxy service converts each received connection parameter into a unified encapsulation protocol required by the remote desktop gateway, and sends the converted unified encapsulation protocol to the remote desktop gateway respectively;

[0011] The remote desktop gateway determines the corresponding remote desktop and protocol type according to the parameters contained in each received unified encapsulation protocol, converts the unified encapsulation protocol into a protocol of the corresponding type, and establishes a connection with the corresponding remote desktop based on the converted protocol;

[0012] After the connection between the remote desktop gateway and the remote desktop is successfully established, the screen frame data of the remote desktop is obtained in real time and forwarded to the remote protocol proxy service;

[0013] The remote protocol proxy service forwards the received picture frame data to the browser;

[0014] After receiving the picture frame data, the browser searches for the corresponding canvas display position parameters according to its corresponding remote desktop, and then uses the Guacamole js component to render the received picture frame data at the position corresponding to the browser's canvas display position parameters.

[0015] Furthermore, the communication between the remote protocol proxy service and the remote desktop gateway adopts the Socket protocol.

[0016] Furthermore, the browser and the remote protocol proxy service communicate via the Websocket protocol.

[0017] Furthermore, the process in which the remote protocol proxy service converts the request parameters into the unified encapsulation protocol required by the remote desktop gateway includes: parameter parsing, parameter mapping, parameter calculation and replacement.

[0018] Furthermore, the connection parameters include: the name of the remote desktop, the protocol type, the IP address, the port type, the user name, the password, the screen pixel width and height, and whether it is read-only.

[0019] Furthermore, when remote desktop operations need to be implemented, the browser captures the operation events and forwards the event source and information to the remote desktop gateway through the remote protocol proxy service; the remote desktop gateway translates them into corresponding remote desktop protocol instructions to operate the remote desktop, and sends the response returned by the remote desktop back to the browser through the remote protocol proxy service.

[0020] Furthermore, each remote desktop instance is configured in one or more canvases of the browser; multiple remote desktop instances can be configured in one canvas, and the configuration method is as follows: first, the canvas is divided into cells using the row and column method, and then several cell areas are selected on the canvas to display a specific remote desktop instance. The connection parameters corresponding to the remote desktop instance should be set, and the corresponding position and coordinate parameters of the selected remote desktop instance in the canvas are used as its canvas display position parameters; then the next remote desktop instance is configured in the same way, and when all remote desktop instances in a single canvas are configured, the configured parameter information is saved; the canvas display position parameters of each remote desktop instance can be modified and adjusted at any time.

[0021] The present invention adopts the above technical solution, adopts the B / S mode to realize the connection of multiple remote desktops, and displays multiple remote desktop connections in the same screen, which not only realizes cross-platform access to remote desktops, reduces the usage threshold and maintenance costs, but also simplifies the cumbersome desktop switching method in the previous use process, improves work efficiency, and can also meet various demand scenarios of global monitoring and chain operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Shown is a schematic diagram of a method according to an embodiment of the present invention.

[0023] Figure 2 Shown is a schematic diagram of configuring a remote desktop instance in a canvas according to this embodiment. DETAILED DESCRIPTION

[0024] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0025] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0026] The embodiment of the present invention provides a method for uniformly displaying multiple remote desktops based on a Web mode, such as Figure 1 As shown, the method comprises the following steps:

[0027] S1: Build a remote desktop gateway based on Guacamole to unify the encapsulation of different remote desktop protocols.

[0028] This embodiment analyzes, disassembles, refines, and encapsulates commonly used remote desktop protocols on the market, such as RDP, VNC, and SSH, extracts common parameters, and encapsulates them into a unified upper-layer protocol. In this way, users do not need to care about the definition and parameters of each specific protocol. They can directly use the encapsulated protocol to simply connect to the core parameters to adapt to access the remote desktop instance that uses the protocol.

[0029] This embodiment uses Guacamole to implement such an encapsulation protocol. Guacamole is an open source, clientless remote desktop gateway that supports multiple remote desktop protocols, such as RDP, VNC, SSH, etc. This embodiment uses the encapsulation capability of Guacamole to uniformly encapsulate different remote desktop protocols so that applications can access these remote desktops through a unified interface. The component that plays this role in Guacamole is guacd. The core function of guacd is protocol conversion. It receives Guacamole protocol instructions from the Guacamole server, and then translates these instructions into specific remote desktop protocol (such as VNC, RDP, etc.) instructions and sends them to the remote desktop server. At the same time, guacd also translates the response of the remote desktop server back to the Guacamole protocol and sends it back to the Guacamole server and client. The protocol consists of instructions, each of which is a list separated by commas and ends with a semicolon. The first element of the instruction is the instruction opcode, and the remaining elements are the parameters of the instruction. These parameters and the instruction opcode together constitute a complete instruction for controlling the behavior and display of the remote desktop. The instruction format is: [LENGTH].[VALUE],[LENGTH],[VALUE],...;[EOL], where LENGTH represents the length of VALUE (i.e. the number of Unicode characters, encoded in UTF-8), and VALUE represents the specific instruction or parameter value. The entire instruction ends with a semicolon (;), indicating the end of the instruction.

[0030] Common instruction types:

[0031] Control commands: used to establish or disconnect a connection with a remote desktop. For example, the select command is used to select the remote desktop protocol to connect to, and the connect command is used to attempt to establish a connection.

[0032] Event instructions: used to capture and transmit user input events such as keyboard and mouse. These events are converted into corresponding Guacamole protocol instructions and sent to guacd for processing.

[0033] Drawing instructions: used to draw images, text, etc. on the remote desktop. These instructions are usually sent by the server to the client to update the display content of the remote desktop.

[0034] For example, the command to set the window size is 4.size,1.0,4.1024,3.768; this command sets the window size to 1024x768 pixels. Among them, "4.size" represents the command opcode and length, "1.0" represents the index of the default layer of the image, and "4.1024" and "3.768" represent the width and height of the window respectively.

[0035] This embodiment implements the remote desktop protocol based on Guacamole, so the Guacamole server (i.e., remote desktop gateway), that is, the guacd component, must be deployed in advance. Guacd is generally deployed in a Docker container and is provided to the remote protocol proxy service for connection after deployment.

[0036] S2: Build a remote protocol proxy service to convert request parameters into the unified encapsulation protocol required by the remote desktop gateway.

[0037] Each remote desktop instance that needs to be connected has its corresponding original connection parameters. The original parameters cannot be directly sent to the remote desktop protocol server for parsing. An intermediate conversion layer is required to convert the original parameters requested by the front end into a parameter system that can be recognized by the unified protocol. This conversion process includes parameter parsing, parameter mapping, parameter calculation and replacement, etc. In this embodiment, this intermediate conversion layer is called "remote protocol proxy service".

[0038] The communication method between the remote protocol proxy service and the front-end page can use Websocket and Http. It is usually recommended to use Websocket. Websocket is a full-duplex communication protocol based on the Http protocol. After establishing a connection, the protocol can perform two-way full-duplex data exchange. The client can send request parameters and operation instructions to the server in real time, and the server can also push the desktop frame data returned by the remote desktop server to the client at the same time. The two-way communication is carried out simultaneously without interfering with each other, which greatly improves the efficiency compared to the frequent execution of request-connection-disconnection by the Http protocol. However, it must be pointed out that not all browsers support WebSocket. If a browser that does not support WebSocket is encountered, the front-end Guacamole-js component automatically degenerates to the Http protocol to give priority to ensuring the availability of the remote desktop connection function.

[0039] The remote protocol proxy service communicates with the remote desktop gateway (i.e., the Guacamole server) using the Socket protocol, which only requires the IP and port of the remote desktop instance to achieve stable data communication.

[0040] The front end sends the request parameters to the remote protocol proxy service through Websocket or Http protocol. After the proxy service converts the parameters, it connects to the remote desktop gateway through the Socket protocol for desktop frame data communication.

[0041] The remote protocol proxy service is a backend service that provides WebSocket and Http server support capabilities. It is used to accept the parameters of the front-end request and perform operations such as parameter verification, mapping, and completion. The parameters are converted into a parameter list that guacd can recognize, and then a connection is established with guacd through the Socket protocol. Guacd then connects to the real remote desktop host. After the connection is successful, guacd will capture the return frame data of different protocols and transmit it to the remote protocol proxy service. The proxy service then pushes the return frame data directly to the client through WebSocket. The front-end page receives the return desktop frame data and renders the page.

[0042] Different remote protocols have different processing requirements for request parameters, including both common parameters and parameters specific to each protocol. Common parameters include (example) protocol (protocol name RDP, VNC, SSH), hostname (host name or IP), port (protocol port number), username (user name), password (password); protocol-specific parameters include (example) RDP: ignore-cert (ignore server certificate), VNC: read-only (read-only), SSH: color-scheme (screen color); client image rendering parameters include width, height, dpi, etc. The remote protocol proxy service carries these parameters to establish a Socket connection with guacd, thereby establishing a connection with the real remote desktop host to start data communication.

[0043] S3: Configure the connection parameters and canvas display location parameters of each remote desktop instance that needs to be accessed in the browser.

[0044] In this embodiment, the entire global page displayed by the browser is called a "canvas". The position and size of each remote desktop instance on the canvas (i.e., the canvas display position parameters) can be set according to the needs. The user can arbitrarily frame an area on the canvas to display a specific remote desktop instance, set the size and range, and then select the instance source displayed in the area. If there is still free space on the canvas, the second, third...Nth remote desktop instance can be configured and displayed in the same way. After the setting of a single canvas is completed, the setting information is saved, generally in a relational database, for use in extracting the setting parameters when the actual screen is rendered. Obviously, a canvas can configure and display multiple independent remote desktop instances at the same time, and there can be multiple different canvases in the system to meet the remote desktop display needs of various different scenarios.

[0045] The connection parameters of the remote desktop instance configured in this embodiment mainly include the name of the remote desktop (the name of the remote desktop instance displayed in the canvas), the protocol type (choose one of RDP, VNC, and SSH), the host address (usually an IP address), the port type (the port used by the protocol, different protocols use different ports, the default for RDP is 3389, the default for VNC is 5900, and the default for SSH is 22), the user name (the user name for logging into the remote desktop), the password (the password for logging into the remote desktop), the pixel width and height of the host screen (determines the aspect ratio displayed in the remote desktop, which is invalid in VNC), and whether it is read-only (the remote screen can only be viewed but not operated, and is only valid in VNC). After all the necessary parameters are configured, they are saved (usually saved in a relational database) for use when the canvas configures the instance source.

[0046] The actual settings and layout of the canvas are as follows Figure 2 As shown, the entire page is regarded as a whole. The row and column method can be used to divide the cells. The default row and column layout can be set to 10*10 or 20*20, or the total number of rows and columns can be defined according to the needs. A single remote desktop instance occupies at least one cell. After the initial setting, the size of the occupied cell can be adjusted. It can be dragged, moved, stretched, and reduced at will, which is very convenient to operate. A single remote desktop instance is not allowed to overwrite the cells occupied by other set instances. At the same time, quick operation options are provided: select the instance that needs to be operated, modify the properties of the current instance, and delete the current instance. In short, the design and arrangement of the canvas needs to be easy to operate, the threshold for getting started should be as low as possible, and it is more convenient to use. After the entire canvas is designed, it needs to be saved and submitted in json format. It is necessary to record the basic information of the current canvas and the configured remote desktop instance source information (i.e. connection parameters) and coordinate information (canvas display position parameters) to facilitate the restoration of the set layout and instance resources during rendering. The submitted json format example is as follows:

[0047] {

[0048] id:1;

[0049] name: "Canvas 1";

[0050] startup:true;

[0051] items:[{"i":"1","x":0,"y":0,"w":3,"h":3,"resourceId":111,"autoFit":false,"isActive":false} ...]

[0053] }.

[0054] Among them, startup means setting this canvas as the startup page, items is the information of all remote desktop instances configured in the canvas, and its elements x, y, w, h are the xy coordinates of the left vertex of the instance and the number of cells occupied by the width and height of the instance, respectively, resourceId is the configured instance id, autoFit indicates whether to automatically adapt to the window size, and isActive indicates whether the instance is selected in the current canvas.

[0055] This embodiment only proposes a method for canvas design and arrangement, and does not involve specific front-end technical components. Users can choose appropriate technical solutions to implement the functions of the canvas according to their own technical system and characteristics.

[0056] This embodiment deploys three services through the above method, namely a front-end service and two back-end services. The front-end service includes the configuration and maintenance of the remote desktop instance, canvas configuration and canvas rendering functions, which are packaged into Nginx for deployment; the back-end includes the remote protocol proxy service and the remote desktop gateway, both of which can be deployed using docker.

[0057] After the above structure is deployed, when you need to access each remote desktop, perform the following steps:

[0058] S101: The browser sends the configured connection parameters of each remote desktop instance to the remote protocol proxy service through different requests.

[0059] S102: The remote protocol proxy service converts each received connection parameter into a unified encapsulation protocol required by the remote desktop gateway, and sends the converted unified encapsulation protocol to the remote desktop gateway respectively.

[0060] S103: The remote desktop gateway determines the corresponding remote desktop and protocol type according to the parameters contained in each received unified encapsulation protocol, converts the unified encapsulation protocol into a protocol of the corresponding type, and establishes a connection with the corresponding remote desktop based on the converted protocol.

[0061] S104: After the connection between the remote desktop gateway and the remote desktop is successfully established, the screen frame data of the remote desktop is obtained in real time and forwarded to the remote protocol proxy service.

[0062] S105: The remote protocol proxy service forwards the received picture frame data to the browser.

[0063] S106: After receiving the picture frame data, the browser searches for the corresponding canvas display position parameter according to its corresponding remote desktop, and then uses the Guacamole js component to render the received picture frame data at the position corresponding to the canvas display position parameter of the browser.

[0064] In this embodiment, the front end references the js file provided by Guacamole to parse the remote desktop frame data returned by the Guacamole server. By parsing these frame data and then using Guacamole's js component, the actual remote desktop screen effect can be rendered. If the front end is connected to the proxy service through the Websocket protocol, when the remote desktop screen frame changes, the proxy server will automatically push the new screen frame data to the front-end page. The front end uses the same method to render and update, so as to achieve "real-time synchronization" between the remote desktop screen displayed on the canvas and the actual remote desktop host screen. Although this real-time synchronization is not strictly completely real-time synchronization, it is almost impossible to feel the delay under good network conditions. It should be noted here that there will be a certain delay in theory, and the delay is greatly affected by the quality and rate of the network bandwidth.

[0065] In addition to displaying the remote desktop example screen, it is also necessary to implement remote desktop operations. Generally speaking, mouse operations and various keyboard operations can be performed by capturing mouse and keyboard click and input events on the canvas, transmitting the event source and information to the remote protocol proxy service through the Websocket channel, and then transmitted to the Guacamole server by the proxy service, and converted into real remote desktop protocol events to drive the remote desktop to make corresponding actions and reactions. These actions and reactions are usually reflected in the changes of the screen. After the screen changes, the server pushes the updated desktop frame data through Websocket, and the front end performs rendering and update, so that the purpose of remote desktop operation and interaction is achieved.

[0066] If the system has some keyboard interception or monitoring plug-ins installed, the remote protocol proxy service may not be able to capture the keyboard combination key operation, resulting in the failure of the remote desktop keyboard combination input event. To avoid this situation, you can provide a button outside each remote desktop instance to simulate the pressing of the combination key (such as Ctrl, Shift, Alt, etc.) to realize the function of inputting the combination key from the outside; you can also directly provide quick input of commonly used combination keys, such as Ctrl+Alt+Delete. This function is usually implemented directly on the front-end page by operating the keyboard code.

[0067] The embodiment of the present invention proposes a method for unified display of multiple remote desktops based on a web mode, which adopts a B / S mode to realize the connection of multiple remote desktops, and displays the multiple remote desktops in the same screen, which not only realizes cross-platform access to remote desktops, reduces the usage threshold and maintenance costs, but also simplifies the cumbersome desktop switching method in the previous use process, improves work efficiency, and can also meet various demand scenarios of global monitoring and chain operation.

[0068] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A method for uniformly displaying multiple remote desktops based on a Web mode, characterized in that: include: Build a remote desktop gateway based on Guacamole to unify the encapsulation of different remote desktop protocols; Build a remote protocol proxy service to convert request parameters into the unified encapsulation protocol required by the remote desktop gateway; configure the connection parameters and canvas display position parameters of each remote desktop instance that needs to be accessed in the browser; To access each remote desktop, perform the following steps: The browser sends the connection parameters of each remote desktop instance configured to the remote protocol proxy service through different requests; The remote protocol proxy service converts each received connection parameter into a unified encapsulation protocol required by the remote desktop gateway, and sends the converted unified encapsulation protocol to the remote desktop gateway respectively; The remote desktop gateway determines the corresponding remote desktop and protocol type according to the parameters contained in each received unified encapsulation protocol, converts the unified encapsulation protocol into a protocol of the corresponding type, and establishes a connection with the corresponding remote desktop based on the converted protocol; After the connection between the remote desktop gateway and the remote desktop is successfully established, the screen frame data of the remote desktop is obtained in real time and forwarded to the remote protocol proxy service; The remote protocol proxy service forwards the received picture frame data to the browser; After receiving the picture frame data, the browser searches for the corresponding canvas display position parameters according to its corresponding remote desktop, and then uses the Guacamole js component to render the received picture frame data at the position corresponding to the browser's canvas display position parameters.

2. The method for uniformly displaying multiple remote desktops based on the Web mode according to claim 1, characterized in that: The communication between the remote protocol proxy service and the remote desktop gateway uses the Socket protocol.

3. The method for uniformly displaying multiple remote desktops based on the Web mode according to claim 1, characterized in that: The browser and the remote protocol proxy service communicate via the Websocket protocol.

4. The method for uniformly displaying multiple remote desktops based on a Web mode according to claim 1, characterized in that: The process by which the remote protocol proxy service converts request parameters into the unified encapsulation protocol required by the remote desktop gateway includes: parameter parsing, parameter mapping, parameter calculation, and replacement.

5. The method for uniformly displaying multiple remote desktops based on the Web mode according to claim 1, characterized in that: The connection parameters include: the name of the remote desktop, protocol type, IP address, port type, user name, password, screen pixel width and height, and whether it is read-only.

6. The method for uniformly displaying multiple remote desktops based on the Web mode according to claim 1, characterized in that: When remote desktop operations need to be implemented, the browser captures the operation events and forwards the event source and information to the remote desktop gateway through the remote protocol proxy service; the remote desktop gateway translates them into corresponding remote desktop protocol instructions to operate the remote desktop, and sends the response returned by the remote desktop back to the browser through the remote protocol proxy service.

7. The method for uniformly displaying multiple remote desktops based on a Web mode according to claim 1, characterized in that: Each remote desktop instance is configured in one or more canvases of the browser; multiple remote desktop instances can be configured in one canvas, and the configuration method is as follows: first, use the row and column method to divide the canvas into cells, and then select several cell areas on the canvas to display a specific remote desktop instance. The connection parameters corresponding to the remote desktop instance should be set, and the corresponding position and coordinate parameters of the selected remote desktop instance in the canvas are used as its canvas display position parameters; then use the same method to configure the next remote desktop instance. When all remote desktop instances in a single canvas are configured, save the configured parameter information; the canvas display position parameters of each remote desktop instance can be modified and adjusted at any time.

Images