Remote desktop control method and device

Abstract

The invention provides a remote desktop control method and device, and the method comprises the steps: responding to a group creation instruction received from a controlled end, creating a group, and generating a target authentication code corresponding to the group; receiving an authentication code sent by at least one control end, and adding a target control end into the group; controlling the target control end and the controlled end to synchronously display the initial desktop picture; receiving an operation instruction which is sent by the target control end and executed for the initial desktop picture, and sending the operation instruction to the controlled end; receiving area picture change information sent by the controlled end, wherein the area picture change information is used for indicating the area position and change content of picture change generated when the controlled end executes the operation instruction; and transmitting the picture change information to the target control end, so that the target control end updates the display content at the regional position according to the change content. According to the invention, the control efficiency of the remote desktop is improved.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a remote desktop control method and device. Background Art

[0002] Currently, remote desktop is a technology that allows users to access and control devices from a remote location. Through a remote desktop connection, users can operate remote devices on their computers and mobile apps. This technology is very useful for people who need to remotely control devices for maintenance, troubleshooting, and other operations.

[0003] Automatic navigation is a system that uses technologies such as the Global Positioning System (GNSS), Inertial Navigation System (INS), and sensors to enable agricultural machinery to drive autonomously along a pre-set path, achieving precise pesticide application, sowing, plowing, and other operations.

[0004] However, after the automatic navigation system is sold to the market, it is spread all over the world. After the automatic navigation system is installed and before operation, parameter setting and function debugging are required. Relying solely on on-site technical support by after-sales personnel will result in problems such as large workload, high cost, and slow service response, which makes it difficult to meet user needs.

[0005] In addition, the automatic navigation system uses an Internet of Things card, which has low traffic and bandwidth. The existing technical solutions transmit desktop images and mouse and keyboard operations, which occupy a lot of traffic and high bandwidth, and can easily cause the remote desktop to freeze, and the automatic navigation system to be unable to be remotely controlled normally, and the automatic navigation system to shut down due to insufficient traffic on the Internet of Things card. Summary of the invention

[0006] In view of this, the purpose of the present application is to provide a remote desktop control method and device to overcome at least one of the above-mentioned defects.

[0007] In a first aspect, an embodiment of the present application provides a remote desktop control method, which is applied to a server, the method comprising: in response to a group creation instruction received from a controlled terminal, creating a group and generating a target authentication code corresponding to the group, the group including the controlled terminal; receiving an authentication code sent by at least one control terminal, adding a target control terminal to the group, the target control terminal being a control terminal whose authentication code is consistent with the target authentication code; controlling the target control terminal to synchronously display an initial desktop screen with the controlled terminal, the initial desktop screen referring to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group; receiving an operation instruction sent by the target control terminal to execute on the initial desktop screen, and sending the operation instruction to the controlled terminal; receiving regional screen change information sent by the controlled terminal, the regional screen change information being used to indicate the regional position and change content of the screen change that occurs when the controlled terminal executes the operation instruction; transmitting the screen change information to the target control terminal so that the target control terminal updates the display content at the regional position according to the change content.

[0008] In an optional embodiment of the present application, the controlled end and the server end meet identity authentication to establish a connection, wherein whether the controlled end meets the identity authentication trigger condition is determined in the following manner: obtaining a first control parameter of the controlled end, the first control parameter including a first terminal number and a first identity identification code of the controlled end, the first identity identification code being generated by the controlled end according to the first terminal code; extracting the first terminal number of the controlled end, and generating a first target identity identification code according to a preset identity authentication algorithm; determining whether the first target identity identification code is consistent with the first identity identification code of the controlled end; if the first target identity identification code is inconsistent with the first identity identification code of the controlled end, it is considered that the controlled end does not meet the identity authentication trigger condition, and if the first target identity identification code is consistent with the first identity identification code of the controlled end, it is considered that the controlled end meets the identity authentication trigger condition.

[0009] In an optional embodiment of the present application, at least one control end and the service end meet identity authentication to establish a connection, wherein whether the control end meets the identity authentication trigger condition is determined in the following manner: obtaining a second control parameter of the control end, the second control parameter including a second terminal number and a second identity identification code of the control end, the second identity identification code being generated by the controlled end according to the second terminal code; extracting the second terminal number of the control end, and generating a second target identity identification code according to a preset identity authentication algorithm; determining whether the second target identity identification code is consistent with the second identity identification code of the control end; if the second target identity identification code is inconsistent with the second identity identification code of the control end, it is considered that the control end does not meet the identity authentication trigger condition, and if the second target identity identification code is consistent with the second identity identification code of the control end, it is considered that the control end meets the identity authentication trigger condition.

[0010] In an optional embodiment of the present application, the method also includes: after detecting that the target control terminal has joined the group, sending a desktop screen upload instruction to the controlled terminal; receiving a target desktop screen sent by the controlled terminal, wherein the target desktop screen is the entire desktop screen of the controlled terminal when the controlled terminal receives the desktop screen upload instruction; and sending the target desktop screen to the target control terminal newly joined the group for display at the target control terminal.

[0011] In an optional embodiment of the present application, the controlled end is configured to: receive the operation instruction, and control the desktop to perform the corresponding operation according to the operation instruction; record the area position and change content of the screen changes that occur when the desktop executes the operation instruction.

[0012] In an optional embodiment of the present application, the method also includes: when receiving the first end remote control command from the target control terminal, determining whether there are other target control terminals in the group; if there are other target control terminals in the group, disconnecting the target control terminal that sends the first end remote command from the group; if there are no other target control terminals in the group, disconnecting the target control terminal, the controlled terminal and the server terminal, and removing the group information; when receiving the second end remote command from the controlled terminal, directly disconnecting the controlled terminal, all target control terminals and the server terminal in the group, and removing the group information.

[0013] In a second aspect, an embodiment of the present application further provides a remote desktop control device, which is applied to a server, and the remote desktop control device includes: a group creation module, which is used to create a group in response to a group creation instruction received from a controlled terminal, and generate a target authentication code corresponding to the group, wherein the group includes the controlled terminal; a group joining module, which is used to receive an authentication code sent by at least one control terminal, and join a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code; an initial desktop screen synchronization module, which is used to control the target control terminal to synchronously display an initial desktop screen with the controlled terminal, wherein the initial desktop screen refers to the target control terminal being added to the group; The entire desktop screen displayed on the controlled terminal when joining the group; an operation instruction receiving module, used to receive the operation instruction sent by the target control terminal to be executed on the initial desktop screen, and send the operation instruction to the controlled terminal; a regional screen change information receiving module, used to receive the regional screen change information sent by the controlled terminal, the regional screen change information is used to indicate the regional position and change content of the screen change that occurs when the controlled terminal executes the operation instruction; a display content update module, used to transmit the screen change information to the target control terminal, so that the target control terminal updates the display content at the regional position according to the change content.

[0014] In a third aspect, an embodiment of the present application further provides a remote desktop control system, the remote desktop control system comprising: at least one target control terminal connected to a server terminal; a controlled terminal connected to a server terminal; the server terminal being configured to perform the following processing: in response to a group creation instruction received from a controlled terminal, creating a group and generating a target authentication code corresponding to the group, the group including the controlled terminal; receiving an authentication code sent by at least one control terminal, adding a target control terminal to the group, the target control terminal being a control terminal whose authentication code is consistent with the target authentication code; controlling the target control terminal to synchronously display an initial desktop with the controlled terminal; The control unit comprises a control unit that receives an initial desktop screen, wherein the initial desktop screen refers to the overall desktop screen displayed on the controlled terminal when the target control terminal is added to the group; receives an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sends the operation instruction to the controlled terminal; receives regional screen change information sent by the controlled terminal, wherein the regional screen change information is used to indicate the regional position and change content of the screen change that occurs when the controlled terminal executes the operation instruction; transmits the screen change information to the target control terminal, so that the target control terminal updates the display content at the regional position according to the change content.

[0015] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method described above are performed.

[0016] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are executed.

[0017] The control method and device of the remote desktop provided by the embodiment of the present application, in response to the group creation instruction received from the controlled end, creates a group and generates a target authentication code corresponding to the group; receives the authentication code sent by at least one control end, and adds the target control end to the group; controls the target control end to synchronously display the initial desktop screen with the controlled end; receives the operation instruction sent by the target control end for the initial desktop screen, and sends the operation instruction to the controlled end; receives the regional screen change information sent by the controlled end, and the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled end executing the operation instruction; transmits the screen change information to the target control end, so that the target control end updates the display content at the regional position according to the change content. Through this application, the control efficiency of the remote desktop is improved.

[0018] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flowchart of a remote desktop control method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of the structure of a remote desktop control device provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0024] First, the application scenarios to which this application is applicable are introduced. This application can be applied in the field of information security technology.

[0025] Research has found that after the automatic navigation system is sold to the market, it is spread all over the world. After the automatic navigation system is installed and before operation, parameter settings and function joint debugging are required. Relying solely on on-site technical support by after-sales personnel will result in heavy workload, high cost, slow service response, and other problems, making it difficult to meet user needs. In addition, the automatic navigation system uses an Internet of Things card, which has low traffic and low bandwidth. The existing technical solutions transmit desktop images and mouse and keyboard operations, which occupy a lot of traffic and high bandwidth, and can easily lead to remote desktop freezes, inability to remotely control the automatic navigation system normally, and automatic navigation system shutdown due to insufficient Internet of Things card traffic.

[0026] Based on this, the embodiment of the present application provides a control method and device for a remote desktop provided by the embodiment of the present application, in response to a group creation instruction received from a controlled terminal, a group is created, and a target authentication code corresponding to the group is generated; an authentication code sent by at least one control terminal is received, and a target control terminal is added to the group; the target control terminal is controlled to synchronously display the initial desktop screen with the controlled terminal; an operation instruction sent by the target control terminal for executing the initial desktop screen is received, and the operation instruction is sent to the controlled terminal; regional screen change information is received from the controlled terminal, and the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction; the screen change information is transmitted to the target control terminal, so that the target control terminal updates the display content at the regional position according to the change content. Through this application, the control efficiency of the remote desktop is improved.

[0027] This application mainly includes three parts: server, control and controlled. The executor of this application is the server.

[0028] The server is a remote desktop server that supports the WebSocket protocol and is developed using the Java development language and JDK1.8 development environment based on the Netty open source network framework.

[0029] The control end supports two types of terminals, mobile APP (mobile phone) and PC web page (computer). Here, the mobile APP uses Android APP developed in Kotlin language or IOS APP developed in Swift language, and uses Java-WebSocket class library to communicate with the server. The PC web page uses VUE development language, HTML5 standard, and uses WebSocket client to communicate with the server.

[0030] The controlled end runs an application in the automatic navigation system. The automatic navigation is based on the embedded Linux operating system and QT development language, and uses the WebSocket client to communicate with the server.

[0031] See also Figure 1 , Figure 1 This is a flow chart of the remote desktop control method provided in the embodiment of the present application. Figure 1 As shown in , the remote desktop control method provided by the embodiment of the present application includes:

[0032] S101. In response to a group creation instruction received from a controlled terminal, a group is created and a target authentication code corresponding to the group is generated.

[0033] Here, the group includes the controlled terminal, and by way of example, the target authentication code is a 6-digit number.

[0034] Before the server receives the group creation instruction, it needs to determine whether the controlled end and the server meet the identity authentication in order to connect. The following methods are used to determine whether the controlled end meets the identity authentication trigger condition:

[0035] Obtain a first control parameter of the controlled end.

[0036] The first control parameter includes a first terminal number and a first identity code of the controlled terminal, where the first identity code is generated by the controlled terminal according to the first terminal code;

[0037] For example, the first identity code can be obtained in the following way:

[0038] Symmetric encryption algorithm uses the same key to encrypt and decrypt data. Token can be generated by taking the first terminal number as input and encrypting it using the symmetric key.

[0039] Asymmetric encryption algorithms, such as RSA and ECDSA, are combined with digital signatures to generate tokens. In this method, the server holds a private key, which is used to sign a message containing information such as the first terminal number to generate a token. The server holds the corresponding public key, which is used to verify whether the signature of the token is valid.

[0040] Extracting the first terminal number of the controlled terminal, and generating a first target identity identification code according to a preset identity authentication algorithm;

[0041] Determining whether the first target identity code is consistent with the first identity code of the controlled terminal;

[0042] If the first target identity identification code is inconsistent with the first identity identification code of the controlled terminal, it is considered that the controlled terminal does not meet the identity authentication trigger condition; if the first target identity identification code is consistent with the first identity identification code of the controlled terminal, it is considered that the controlled terminal meets the identity authentication trigger condition.

[0043] Preferably, the first control parameter includes a first terminal number (terminal ID), a first control type, a first terminal type, a first screen resolution, and a first identity code (authentication token). The first terminal number and the first identity code are used to determine whether the controlled end meets the identity authentication requirements with the server in order to establish a connection. The first control type includes 0 and 1, 0 represents the controlled end, and 1 represents the controlling end, so that the server can distinguish whether the terminal uploading the first control parameter is the controlled end or the controlling end.

[0044] The first terminal types include 1, 2, 3 and 4, where 1 represents the automatic navigation system, 2 represents the Android terminal, 3 represents the Apple terminal, and 4 represents the PC terminal (computer terminal). It is used to record the type of the current terminal and count the frequency of each type being authenticated and joined the group.

[0045] The first screen resolution is because the controlling end is a mobile phone APP or a computer PC, and the controlled end is the automatic navigation system. Their screen sizes are different, so the display screen needs to be scaled. For example: the automatic navigation system's screen is small, and the PC's screen is large, so the screen data of the automatic navigation system needs to be enlarged to the PC's screen size; for example: the automatic navigation screen is small, and the mobile phone's screen is even smaller, so the screen data of the automatic navigation system needs to be reduced to the mobile phone's screen size to adapt to the mobile phone screen.

[0046] The server can enlarge or reduce the screen data according to the screen resolution of the control end and the controlled end according to a simple formula, so that the screen data of the controlled end can be adapted to the control end (mobile APP end, PC web page end).

[0047] For example, you can adjust the screen data of the controlled terminal in the following ways:

[0048] First, the server needs to obtain the screen resolution information of the control terminal and the controlled terminal. This is usually achieved through API calls, user input, or automatic detection.

[0049] The server can then calculate a scaling ratio based on the resolutions of the two devices. This ratio is some relationship between the resolution of the controlled end and the resolution of the controlling end, such as a width ratio or a height ratio, or some weighted average of the two.

[0050] Scaling ratio = (a certain dimension of the resolution of the controlled end ÷ the corresponding dimension of the resolution of the controlling end).

[0051] Finally, the screen data is enlarged or reduced using the calculated scaling factor, which can be achieved through a graphics processing library such as HTML5 Canvas, SVG, WebGL, or a server-side image processing library.

[0052] Data (such as images, texts, UI elements, etc.) are enlarged or reduced accordingly according to the scaling ratio to ensure that they can be displayed correctly and without distortion on the controlled end screen.

[0053] In addition to simple scaling, more complex adaptation strategies need to be considered, such as responsive design, elastic layout, etc., to ensure a good user experience on different devices.

[0054] Responsive design often involves using media queries to apply different style rules based on device screen width, height, and other characteristics.

[0055] At least one control terminal and the service terminal meet the identity authentication to connect, wherein whether the control terminal meets the identity authentication trigger condition is determined by the following method:

[0056] Get the second control parameter of the control terminal.

[0057] The second control parameter includes a second terminal number and a second identity code of the control terminal.

[0058] The second identity identification code is generated by the controlled terminal according to the second terminal code.

[0059] The method for obtaining the second identity identification code is the same as the method for obtaining the first identity identification code described above, and will not be described in detail here.

[0060] Extract the second terminal number of the control terminal, and generate a second target identity identification code according to a preset identity authentication algorithm;

[0061] Determining whether the second target identity code is consistent with the second identity code of the control terminal;

[0062] If the second target identity identification code is inconsistent with the second identity identification code of the control terminal, it is considered that the control terminal does not meet the identity authentication trigger condition; if the second target identity identification code is consistent with the second identity identification code of the control terminal, it is considered that the control terminal meets the identity authentication trigger condition.

[0063] Preferably, the second control parameters include a second terminal number (terminal ID), a second control type, a second terminal type, a second screen resolution, and a second identity code (authentication token). The second terminal number and the second identity code are used to determine whether the control terminal meets the identity authentication requirements with the server in order to establish a connection. The second control type includes 0 and 1, 0 represents the controlled terminal, and 1 represents the controlling terminal, so that the server can distinguish whether the terminal uploading the second control parameters is the controlled terminal or the controlling terminal.

[0064] The second terminal type has the same data type and function as the first terminal type, and the second screen resolution has the same data type and function as the first screen resolution, which will not be elaborated here.

[0065] S102: Receive an authentication code sent by at least one control terminal, and add a target control terminal to a group.

[0066] The target control terminal is a control terminal whose authentication code is consistent with the target authentication code.

[0067] When it is detected that the authentication code sent by the control terminal is consistent with the target authentication code, the control terminal is allowed to join the group as the target control terminal.

[0068] After the controlled end successfully logs in, it sends a group creation command. The server generates a 6-digit target authentication code, creates a group successfully, and sends the target authentication code to the controlled end. The controlled end displays the target authentication code on the interface. The remote control personnel enters the authentication code on the control end. The server verifies whether the authentication code is successful. If successful, the control end can be controlled to join the group. If the authentication code verification fails, the authentication failure is returned. The authentication code needs to be re-entered and the server verifies the authentication code again.

[0069] Only the control terminals that join the same group can remotely control the controlled terminals. Multiple control terminals are allowed to join the group and remotely control the same controlled terminal at the same time.

[0070] S103, controlling the target control terminal and the controlled terminal to synchronously display the initial desktop screen.

[0071] The initial desktop screen refers to the overall desktop screen displayed on the controlled terminal when the target control terminal is added to the group.

[0072] In an optional embodiment, the target control terminal in the group does not generate an operation instruction. After detecting that the target control terminal joins the group, it sends a desktop screen upload instruction to the controlled terminal and receives the target desktop screen sent by the controlled terminal.

[0073] The target desktop image is the overall desktop image of the controlled terminal when receiving the desktop acquisition signal;

[0074] The target desktop screen is sent to the target control terminal newly added to the group for display on the target control terminal.

[0075] In another optional embodiment, the target control terminal in the group generates an operation instruction. At this time, when it is detected that there is a target control terminal newly joined the group, the server sends a desktop screen upload instruction to the controlled terminal to obtain the current desktop of the controlled terminal to the overall screen, and transmits the desktop to the overall screen to the target control terminal newly joined the group for display.

[0076] S104: receiving an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sending the operation instruction to the controlled terminal.

[0077] Here, the operation instructions include but are not limited to the remote control personnel performing mouse and keyboard operations on the mobile APP and PC web pages. The control end will collect and upload the mouse coordinate points, clicks, drags, keyboard keys and other operations in real time. The server end forwards the mouse and keyboard operations to the controlled end in the same group. After receiving the mouse and keyboard operation instructions, the controlled end parses and executes the mouse and keyboard operations to achieve the purpose of remote control automatic navigation.

[0078] S105: Receive the area screen change information sent by the controlled terminal.

[0079] The regional screen change information is used to indicate the regional position and change content of the screen change that occurs when the controlled terminal executes the operation instruction.

[0080] Specifically, the controlled end is configured as follows:

[0081] Receive operation instructions and control the desktop to perform corresponding operations according to the operation instructions;

[0082] Records the area location and content of screen changes that occur when the desktop executes operation instructions.

[0083] The controlled end will first upload the entire desktop image, and then use the image comparison algorithm to upload only the area location and content of the changed image if there is any change. Otherwise, it will not be uploaded, which greatly saves IoT card traffic.

[0084] S106: Transmit the picture change information to the target control terminal, so that the target control terminal updates the display content at the regional position according to the changed content.

[0085] When the server receives the screen change information, it forwards the screen change information to all control terminals in the group. The control terminal updates the desktop screen. The remote control personnel (control terminal) can update the display content at the regional location according to the changed content and see the desktop of the automatic navigation system remotely in real time.

[0086] In an optional embodiment, the control of the remote desktop is ended in the following manner:

[0087] Upon receiving the first end remote control instruction from the target control terminal, determining whether there are other target control terminals in the group;

[0088] If there are other target control terminals in the group, disconnect the target control terminal that sent the first remote termination instruction from the group;

[0089] If there is no other target control terminal in the group, disconnect the target control terminal, the controlled terminal and the server terminal, and remove the group information;

[0090] When receiving the second remote termination instruction from the controlled terminal, the connection between the controlled terminal, all target control terminals and the server in the group is directly disconnected, and the group information is removed.

[0091] Specifically, there are two main types of initiators to end the remote process:

[0092] When the control end initiates the remote end, the server determines whether there are other control ends in the group. If there are no other control ends, the server notifies the controlled end to end the control, and then disconnects from the control end and the controlled end, and removes the group information. If there are other control ends, the server only disconnects the current control end without affecting the work of other control ends and the controlled end.

[0093] The controlled end initiates the remote termination, and the server notifies all control ends in the group to terminate the remote termination, then disconnects the controlled end and all control ends in the group and removes the group information.

[0094] A private communication protocol was developed based on the WebSocket protocol. A total of 13 instructions were developed. The control instructions for the remote desktop are shown in Table 1 below:

[0095]

[0096] Table 1

[0097] Among them, the heartbeat response refers to the response made by one party (usually the server) after receiving the heartbeat request (such as the PINGREQ message in the MQTT protocol) sent by the other party (controller / controlled end) in the network communication protocol, especially in those occasions where the connection needs to be maintained for a long time (such as the MQTT protocol). The main purpose of the heartbeat and heartbeat response mechanism is to detect and maintain the connection status between the communicating parties to ensure the continued validity of the connection.

[0098] The control method and device of the remote desktop provided by the embodiment of the present application, in response to the group creation instruction received from the controlled end, creates a group and generates a target authentication code corresponding to the group, wherein the group includes the controlled end; receives the authentication code sent by at least one control end, and adds the target control end to the group; controls the target control end to synchronously display the initial desktop screen with the controlled end; receives the operation instruction sent by the target control end for the initial desktop screen, and sends the operation instruction to the controlled end; receives the regional screen change information sent by the controlled end, and the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled end executing the operation instruction; transmits the screen change information to the target control end, so that the target control end updates the display content at the regional position according to the change content. Through this application, the control efficiency of the remote desktop is improved.

[0099] In the usage scenario of automatic navigation, the present application realizes the function of automatic navigation of remote desktop through mobile APP and PC web page. The control end realizes mobile APP and PC web page, supports multiple control ends to join the group, and automatic remote desktop navigation at the same time, which is convenient for multi-person collaboration. The controlled end uses image comparison algorithm to compare the regional screen change information of the automatic navigation interface (controlled end), and only uploads the regional screen change information. If there is no change, no data will be uploaded. In the scenario where the traffic of the automatic navigation IoT card is limited, it greatly saves the traffic and bandwidth of the IoT card, and provides stable and smooth automatic navigation remote desktop service.

[0100] This application has developed a set of lightweight protocols, which include: terminal login and terminal login response, group creation and group creation response, group joining and group joining response, desktop screen upload instructions and desktop screen data upload, mouse control and keyboard control, end control / being controlled, heartbeat and heartbeat response, etc. In addition, the controlled end sends a request, the server is responsible for generating a 6-digit authentication code, the controlling end enters the authentication code, and the authentication code is verified by the server before the automatic navigation can be remotely controlled, thereby improving security.

[0101] This solution is used in after-sales and joint debugging scenarios of automatic navigation products. The control end supports mobile APP (Android, IOS) and PC web pages (Windows, Linux, and Mac platforms), providing more options for remote personnel. Limited by the low traffic of the automatic navigation IoT, this solution has developed a lightweight private protocol based on WebSocket. The controlled end uses an image comparison algorithm to upload only the regional screen change information of the changed area, and does not upload the non-changed area. 0penCV is used to encode and compress the desktop image of the changed area for transmission to save traffic.

[0102] Based on the same inventive concept, the embodiment of the present application also provides a remote desktop control device corresponding to the remote desktop control method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the remote desktop control method mentioned above in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0103] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of the remote desktop control device provided in the embodiment of the present application. Figure 2 As shown in , the remote desktop control device 200 includes:

[0104] A group creation module 201, for creating a group in response to a group creation instruction received from a controlled terminal, and generating a target authentication code corresponding to the group, wherein the group includes the controlled terminal;

[0105] The group joining module 202 is used to receive an authentication code sent by at least one control terminal and add a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code;

[0106] An initial desktop screen synchronization module 203 is used to control the target control terminal and the controlled terminal to synchronously display an initial desktop screen, wherein the initial desktop screen refers to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group;

[0107] An operation instruction receiving module 204 is used to receive an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and send the operation instruction to the controlled terminal;

[0108] The regional screen change information receiving module 205 is used to receive the regional screen change information sent by the controlled terminal, wherein the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction;

[0109] The display content updating module 206 is used to transmit the screen change information to the target control terminal, so that the target control terminal updates the display content at the area position according to the changed content.

[0110] The embodiment of the present application further provides a remote desktop control system, the remote desktop control system comprising:

[0111] At least one target control terminal connected to the server terminal;

[0112] The controlled end is connected to the server end;

[0113] The server is configured to perform the following processing:

[0114] In response to a group creation instruction received from a controlled terminal, a group is created and a target authentication code corresponding to the group is generated, wherein the group includes the controlled terminal;

[0115] receiving an authentication code sent by at least one control terminal, and adding a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code;

[0116] Controlling the target control terminal and the controlled terminal to synchronously display an initial desktop screen, wherein the initial desktop screen refers to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group;

[0117] receiving an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sending the operation instruction to the controlled terminal;

[0118] receiving the regional screen change information sent by the controlled terminal, wherein the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction;

[0119] The screen change information is transmitted to the target control terminal, so that the target control terminal updates the display content at the area position according to the change content.

[0120] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 3 As shown in , the electronic device 300 includes a processor 310 , a memory 320 and a bus 330 .

[0121] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 via the bus 330. When the machine-readable instructions are executed by the processor 310, the above-mentioned Figure 1 The steps of the remote desktop control method in the method embodiment shown are specifically implemented in the method embodiment and will not be described in detail here.

[0122] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the remote desktop control method in the method embodiment shown are specifically implemented in the method embodiment and will not be described in detail here.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A remote desktop control method, characterized in that: Applied to the server, including: In response to a group creation instruction received from a controlled terminal, a group is created and a target authentication code corresponding to the group is generated, wherein the group includes the controlled terminal; receiving an authentication code sent by at least one control terminal, and adding a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code; Controlling the target control terminal and the controlled terminal to synchronously display an initial desktop screen, wherein the initial desktop screen refers to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group; receiving an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sending the operation instruction to the controlled terminal; receiving the regional screen change information sent by the controlled terminal, wherein the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction; The screen change information is transmitted to the target control terminal, so that the target control terminal updates the display content at the area position according to the change content.

2. The method according to claim 1, characterized in that The controlled end and the server end meet the identity authentication to establish a connection, The following methods are used to determine whether the controlled terminal meets the identity authentication triggering conditions: Acquire a first control parameter of the controlled terminal, where the first control parameter includes a first terminal number and a first identity code of the controlled terminal, where the first identity code is generated by the controlled terminal according to the first terminal code; Extracting the first terminal number of the controlled terminal, and generating a first target identity identification code according to a preset identity authentication algorithm; Determining whether the first target identity code is consistent with the first identity code of the controlled terminal; If the first target identity identification code is inconsistent with the first identity identification code of the controlled end, it is considered that the controlled end does not meet the identity authentication trigger condition; if the first target identity identification code is consistent with the first identity identification code of the controlled end, it is considered that the controlled end meets the identity authentication trigger condition.

3. The method according to claim 1, characterized in that The at least one control terminal and the service terminal meet the identity authentication requirements to establish a connection. The following methods are used to determine whether the control terminal meets the identity authentication triggering conditions: Acquire a second control parameter of the control end, the second control parameter including a second terminal number and a second identity code of the control end, the second identity code being generated by the controlled end according to the second terminal code; Extracting the second terminal number of the control terminal, and generating a second target identity identification code according to a preset identity authentication algorithm; Determining whether the second target identity code is consistent with the second identity code of the control terminal; If the second target identity identification code is inconsistent with the second identity identification code of the control end, it is considered that the control end does not meet the identity authentication trigger condition; if the second target identity identification code is consistent with the second identity identification code of the control end, it is considered that the control end meets the identity authentication trigger condition.

4. The method according to claim 1, characterized in that: The method further comprises: After detecting that the target control terminal has joined the group, sending a desktop screen upload instruction to the controlled terminal; Receiving a target desktop screen sent by the controlled terminal, wherein the target desktop screen is the entire desktop screen of the controlled terminal when receiving the desktop screen upload instruction; The target desktop screen is sent to a target control terminal newly added to the group so as to be displayed on the target control terminal.

5. The method according to claim 1, characterized in that The controlled end is configured as: Receive the operation instruction, and control the desktop to perform a corresponding operation according to the operation instruction; The area position and the content of the screen changes caused by the desktop executing the operation instruction are recorded.

6. The method according to claim 1, characterized in that The method further comprises: Upon receiving a first end remote control instruction from a target control terminal, determining whether there are other target control terminals in the group; If there are other target control terminals in the group, disconnecting the target control terminal that sends the first remote end instruction from the group; If there is no other target control terminal in the group, disconnect the target control terminal, the controlled terminal and the server terminal, and remove the group information; When receiving the second remote termination instruction from the controlled terminal, the connection between the controlled terminal, all target control terminals and the server in the group is directly disconnected, and the group information is removed.

7. A remote desktop control device, characterized in that: Applied to the server, including: A group creation module, used to create a group in response to a group creation instruction received from a controlled terminal, and generate a target authentication code corresponding to the group, wherein the group includes the controlled terminal; A group joining module, used to receive an authentication code sent by at least one control terminal, and add a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code; An initial desktop screen synchronization module, used to control the target control terminal and the controlled terminal to synchronously display an initial desktop screen, wherein the initial desktop screen refers to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group; An operation instruction receiving module, used for receiving an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sending the operation instruction to the controlled terminal; A regional screen change information receiving module, used for receiving regional screen change information sent by the controlled terminal, wherein the regional screen change information is used for indicating the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction; The display content updating module is used to transmit the screen change information to the target control terminal, so that the target control terminal updates the display content at the area position according to the change content.

8. A remote desktop control system, characterized in that: include: At least one target control terminal connected to the server terminal; The controlled end is connected to the server end; The server is configured to perform the following processing: In response to a group creation instruction received from a controlled terminal, a group is created and a target authentication code corresponding to the group is generated, wherein the group includes the controlled terminal; receiving an authentication code sent by at least one control terminal, and adding a target control terminal to the group, wherein the target control terminal is a control terminal whose authentication code is consistent with the target authentication code; Controlling the target control terminal and the controlled terminal to synchronously display an initial desktop screen, wherein the initial desktop screen refers to the entire desktop screen displayed on the controlled terminal when the target control terminal is added to the group; receiving an operation instruction sent by the target control terminal to be executed on the initial desktop screen, and sending the operation instruction to the controlled terminal; receiving the regional screen change information sent by the controlled terminal, wherein the regional screen change information is used to indicate the regional position and change content of the screen change caused by the controlled terminal executing the operation instruction; The screen change information is transmitted to the target control terminal, so that the target control terminal updates the display content at the area position according to the change content.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images