Remote Control Method for Industrial Internet of Things Devices and Computer-Readable Storage Medium

Abstract

The present application relates to the technical field of industrial Internet of Things data processing, and provides a remote control method for industrial Internet of Things devices and a computer-readable storage medium. The method includes that the first communication module, in response to the P2P tunnel creation notification, determines the second access address of the second device and the tunnel negotiation method based on the P2P tunnel creation notification; sends a first probe packet to the second access address according to the tunnel negotiation method; determines that the P2P tunnel is successfully established after receiving the second probe packet sent by the second device; after determining that the P2P tunnel is successfully established, sends a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used to trigger the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, thereby improving the data transmission efficiency between remote control devices and improving the user experience.

Description

Technical Field

[0001] This application relates to the technical field of industrial Internet of Things data processing, and in particular to a remote control method for industrial Internet of Things devices and a computer-readable storage medium. Background Art

[0002] The industrial Internet of Things is based on Internet of Things technology. By connecting and interconnecting industrial devices, sensors, controllers, systems, and personnel, it realizes the digitization, intelligence, and automation of industrial production and operation processes. With the rapid development of industrial Internet of Things technology, more and more enterprises hope that not only can they monitor and control production on-site through industrial control devices such as HMI (Human Machine Interface) devices, but also hope to remotely connect to HMI devices through mobile Internet devices such as mobile phones and operate and control HMI devices just like operating locally.

[0003] Currently, in the solutions proposed in the industry for the above requirements, there is a method that realizes the remote control of HMI devices by mobile devices based on VNC (Virtual Network Console), and the cloud server realizes data forwarding between the two during the remote control process. It can be seen that the remote control delay between the HMI device and the mobile device is equal to the delay between the HMI device and the cloud server plus the delay between the cloud server and the mobile device.

[0004] However, when the remote control operations are relatively frequent, obvious lags and delays will occur in the VNC window, seriously affecting the user experience and may cause misoperations. Summary of the Invention

[0005] In order to improve the data transmission efficiency between the master control end and the controlled end during the remote control process and enhance the user experience, the embodiments of this application provide a remote control method for industrial Internet of Things devices and a computer storage medium.

[0006] On the one hand, the remote control method for industrial Internet of Things devices provided by the embodiments of the present application is applied to a first device. The first device includes a first VNC module and a first communication module. The first VNC module sends a remote control request for a second device to the first communication module. The first communication module forwards the remote control request to a cloud server. The cloud server determines the communication method between the first device and the second device, and when the determined communication method is to communicate based on a P2P network, it sends a P2P tunnel creation notice to the first device and the second device respectively. Among them, the second device is located in the industrial Internet of Things and includes a second VNC module and a second communication module. The method includes the steps: The first communication module, in response to the P2P tunnel creation notice, determines the second access address of the second device and the tunnel negotiation method based on the P2P tunnel creation notice; sends a first probe packet to the second access address according to the tunnel negotiation method; after receiving the second probe packet sent by the second device, determines that the P2P tunnel is successfully established; after determining that the P2P tunnel is successfully established, sends a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used to trigger the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, where the remote control connection data is the response data generated by the second VNC module after receiving the remote control request forwarded by the cloud server; the first communication module forwards the remote control connection data to the first VNC module based on a local TCP connection; the first VNC module establishes a remote control connection with the second VNC module based on the remote control connection data.

[0007] Based on the above technical solution, a P2P connection between the first device and the second device can be realized, enabling the transmission of remote control data between the first device and the second device based on a P2P tunnel, thereby improving the data transmission efficiency, optimizing the situation of latency and jitter, enhancing the user experience, and at the same time reducing the load and bandwidth cost of the cloud server; at the same time, after the P2P tunnel is established, based on the first data verification packet, the second device can be triggered to directly send the remote control connection data to the first device through the P2P tunnel, so as to achieve a normal response to the remote control request.

[0008] In one implementation, the sending of the first probe packet to the second access address according to the tunnel negotiation method includes: directly sending the first probe packet to the second access address; or, after receiving the P2P tunnel creation notice and after a preset time interval, sending the first probe packet to the second access address.

[0009] Based on the above technical solution, by defining the sending and receiving order of P2P detection packets, it is ensured that the detection packets can be successfully recognized and received by the second device, thereby improving the penetration rate of P2P.

[0010] In one implementation, the method of sending the first detection packet to the second access address after receiving the P2P tunnel creation notification and waiting for a preset duration includes: if a second detection packet from the second device is received before sending the first detection packet, then send the first detection packet to the second address; if the second detection packet is not received before sending the first detection packet, then resend the remote control request to the cloud server.

[0011] Based on the above technical solution, the first device determines whether the P2P tunnel can be successfully created by the reception situation of the first detection packet. When it is found that the creation fails, the request is resent in a timely manner, thus avoiding ineffective waiting and improving communication efficiency.

[0012] In one implementation, before the step of responding to the P2P tunnel creation notification, the method further includes: the first communication module responds to the address reporting instruction sent by the cloud server, and sends response packets to multiple specified ports of the cloud server respectively, so that the cloud server determines the communication type of the first device based on the source address information of the multiple response packets, and determines the tunnel negotiation method based on the communication type of the first device.

[0013] In one implementation, the method further includes: the first communication module receives the control view data returned by the second device based on the P2P tunnel, and forwards the control view data to the first VNC module; the first VNC module renders the control view data to the remote control window in real time.

[0014] On the other hand, the remote control method for industrial Internet of Things devices provided by the embodiments of the present application is applied to a cloud server. The cloud server is communicatively connected to a first device and a second device respectively. The second device is located in the industrial Internet of Things. The first device sends a remote control request to the cloud server to request remote control of the second device. A signaling interaction service and a communication control service are deployed on the cloud server. The method includes: The signaling interaction service, in response to the remote control request, sends an address reporting instruction to the first device and the second device to instruct the first device and the second device to send response messages to respective specified ports, and notifies the communication control service to monitor the specified ports; The communication control service monitors each of the specified ports to receive a first response message sent by the first device to each of the specified ports and a second response message sent by the second device to each of the specified ports; The signaling interaction service respectively determines the communication types of the first device and the second device based on multiple first response messages and multiple second response messages; If the communication types of the first device and the second device are both of the first type, it is determined that the communication mode between the first device and the second device is to communicate based on a P2P network; Determine the tunnel negotiation method according to the communication quality of the first device and the second device; Send P2P tunnel creation notifications to the first device and the second device respectively and forward the remote control request to the second device; The P2P tunnel creation notification carries a target access address and the tunnel negotiation method, so that the first device and the second device establish a P2P tunnel based on the target access address and the tunnel negotiation method.

[0015] Based on the above technical solution, the cloud server determines the communication types of the first device and the second device according to the response messages received by multiple ports, and then generates corresponding P2P tunnel creation notifications, which are sent to the first device and the second device respectively. In this way, the first device and the second device can directly complete the negotiation and establishment of the P2P tunnel according to the negotiation method in the creation notification, thereby realizing the point-to-point transmission of remote control data between the two, improving the transmission efficiency, enhancing the user experience, and at the same time ensuring the penetration rate of the P2P tunnel.

[0016] In one embodiment, the determining the communication type of the first device based on the multiple first response messages respectively includes: determining whether the source addresses of the first response messages are the same. If they are the same, it is determined that the communication type of the first device is of the first type.

[0017] Based on the above technical solution, directly determining the communication type of the device according to the source address of the response message is more suitable for the application scenario of industrial Internet of Things compared with the existing direct request for NAT type, which can improve the judgment accuracy of the device communication type, thereby avoiding the problem that the P2P tunnel cannot be created due to misjudgment of the type.

[0018] In one implementation, the determining the tunnel negotiation method according to the communication quality of the first device and the second device includes: determining the sending order of the detection packets based on the communication quality between the first device and the second device and the cloud server respectively; wherein, the end with poorer communication quality sends the detection packet to the peer first after receiving the P2P tunnel creation notification, and the end with better communication quality sends the detection packet to the peer after a preset time interval after receiving the P2P tunnel creation notification; wherein, the preset time interval is determined based on the maximum value of the network transmission delays corresponding to the first response message and the second response message.

[0019] In one implementation, the method further includes: if the communication type of the first device or the second device is the second type; then directly forwarding the remote control request to the second device, and when receiving the control view data returned by the second device, forwarding the control view data to the first device.

[0020] In addition, an embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation to this application.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1A Illustrate the schematic structural diagram of the first device provided by the embodiment of this application.

[0024] Figure 1B Illustrate the schematic structural diagram of the second device provided by the embodiment of this application.

[0025] Figure 1C Illustrate the schematic structural diagram of the cloud server provided by the embodiment of this application.

[0026] Figure 1D Shows a schematic structural diagram of a remote control service system provided by an embodiment of the present application.

[0027] Figure 2 Shows a flowchart of a method for remotely controlling industrial Internet of Things devices provided by an embodiment of the present application.

[0028] Figure 3 Shows a flowchart of a method for remotely controlling industrial Internet of Things devices provided by an embodiment of the present application. Detailed implementation manners

[0029] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0030] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "plural" refers to two or more, and the "first", "second" and various numerical numbers are only for the convenience of description and do not limit the scope of the embodiments of the present application.

[0031] The features, structures or characteristics in the present application can be combined in one or more embodiments in any suitable manner. In various embodiments of the present application, the order numbers of the various processes do not mean the order of execution, and the execution order of the various processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0032] Some optional features in the embodiments of the present application can, in some scenarios, be implemented independently without relying on other features, solve the corresponding technical problems, achieve the corresponding effects, and can also be combined with other features according to requirements in some scenarios.

[0033] In the present application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of the present application, if there is no special specification and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referred to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships. The implementation manners of the present application do not constitute a limitation to the protection scope of the present application.

[0034] The embodiments of the present application will be described in detail below in conjunction with the drawings.

[0035] The remote control method for industrial Internet of Things devices provided by the embodiments of the present application is used to implement the remote control of a second device located within the industrial Internet of Things by a first device. In practical applications, the first device can be a user's intelligent terminal device, including but not limited to mobile phones, computers, smart TVs, etc.; the second device can be a control device such as an HMI device.

[0036] Please refer to Figure 1A , the first device 11 provided by the embodiments of the present application includes a first VNC module 111 and a first communication module 112. Among them, the first VNC module 111 is implemented based on the VNC Viewer program installed on the first device 11 and is used to request remote control of the VNC Server. The first communication module 112 is a self-developed module and is used to forward the received and transmitted data of the first VNC module 111.

[0037] Please refer to Figure 1B , the second device 12 provided by the embodiments of the present application includes a second VNC module 121 and a second communication module 122. Among them, the second VNC module 121 is implemented based on the VNC Server program installed on the second device 12 and is used to respond to the remote control request of the VNC Viewer. The second communication module 122 is a self-developed module and is used to forward the received and transmitted data of the second VNC module 121.

[0038] Please refer to Figure 1C , in the remote control method provided by the embodiments of the present application, a cloud server 13 is also involved. A signaling interaction service 131 and a communication control service 132 are deployed on the cloud server 13.

[0039] The first device 11, the second device 12, and the cloud server 13 can form a remote control service system. Please refer to Figure 1D , the signaling interaction service 131 can establish communication connections with the first communication module 112 and the second communication module 122 respectively. The communication control service 132 is used to obtain the public network access addresses of the first communication module 112 and the second communication module 122.

[0040] To implement the remote control of the second device by the first device, the devices need to be initialized first. In one example, the initialization of the first device includes starting the first communication module 112, initiating a TCP connection to the signaling interaction service 131 through the first communication module 112, and registering as a control end role (visitor); starting the first VNC module 111 and establishing a local TCP connection with the first communication module 112. The initialization of the second device includes starting the second communication module 122, initiating a TCP connection to the signaling interaction service 131 through the second communication module 122, and registering as a device end role; starting the second VNC module and establishing a local TCP connection with the second communication module 122.

[0041] After initialization is completed, the first VNC module 111 can initiate a remote control request for the second device to the first communication module 112 based on a local TCP connection. When the first communication module 112 receives the remote control request, it can first determine whether there is already a P2P tunnel with the second communication device. If so, it directly forwards the remote control request based on this P2P tunnel. If not, it forwards the remote control request to the signaling interaction service 131 of the cloud server 13, and at the same time sends a remote control permission code to the signaling interaction service 131, enabling the signaling interaction service 131 to access the second VNC module based on the permission code. That is, after forwarding the remote control request to the second VNC module, the second VNC module verifies the legitimacy of the remote control request based on the synchronously sent permission code, and starts to respond to the remote control request when the verification passes.

[0042] When the signaling interaction service 131 receives the remote control request, it can determine whether the second device is online according to the status of the TCP connection with the second device. If the TCP connection is valid, it determines that the second device is online; if the connection has been disconnected, it determines that the second device is not online and rejects the remote control request.

[0043] When it is determined that the second device is online, please refer to Figure 2 , in the remote control method provided by the embodiments of the present application, the method for the cloud server to process the remote control request includes the following steps.

[0044] S201, in response to the remote control request, send an address reporting instruction to the first device and the second device.

[0045] Specifically, after the signaling interaction service 131 receives the remote control request from the first device based on the TCP connection, if it determines that the second device is online, it first sends an address reporting instruction to the first device and the second device based on the established TCP connection to obtain the public network access addresses of the two devices. Among them, the address reporting instruction is used to instruct the first device and the second device to send response messages to multiple specified ports, where the specified port is the local port on the cloud server, and the response message is generated based on the UDP protocol.

[0046] At the same time, the signaling interaction service 131 notifies the communication control service 132 to listen on each specified port to receive the response messages from the first device and the second device. Among them, the response message is transmitted based on the UDP protocol.

[0047] S202, listen on each specified port to receive the first response message sent by the first device to each specified port and the second response message sent by the second device to each specified port.

[0048] Please combineFigure 1D The communication control service 132 starts a corresponding number of service processes according to the number of specified ports, respectively monitors the corresponding specified ports, and extracts the source addresses of the messages when receiving the first response message and the second response message from the first device and the second device, and sends them to the signaling interaction service 131.

[0049] It should be noted that if the number of the first response message or the second response message is inconsistent with the number of the corresponding specified ports, that is, there is a situation where a response message is lost or not sent, the communication control service 132 can directly send a communication exception notification to the signaling interaction service 131 to end the P2P tunnel creation process and switch to a communication method in which the signaling interaction service 131 forwards the data during the remote control process.

[0050] S203, respectively determine the communication types of the first device and the second device based on multiple first response messages and multiple second response messages.

[0051] In implementation, after receiving the multiple source addresses extracted by the communication control service 132 from multiple first response messages, the signaling interaction service 131 determines whether the source addresses are the same. If they are the same, it is determined that the first device is of the first type. Among them, the first type is the type that can create a P2P tunnel. The same source address means that the IP address and the port number are both the same, that is, the quadruple is the same.

[0052] The method for determining the communication type of the second device is the same as that of the first device, and will not be elaborated here.

[0053] It should be noted that in the embodiment of the present application, the signaling interaction service 131 determines whether a device is suitable for creating a P2P tunnel according to whether the source addresses of the response messages received by multiple specified ports are the same. Compared with the prior art, the method of determining whether to create a P2P tunnel by obtaining the NAT type is more applicable to the industrial Internet of Things. The reason is that the existing NAT technology divides NAT into 5 types, which is a comprehensive classification obtained from a theoretical perspective. In the embodiment of the present application, considering the implementation level of P2P technology in the industrial Internet of Things in the actual application process, for P2P connectivity, NAT is simplified into two types: the first type that can be jointly debugged and the second type that is difficult to connect. In this way, by simplifying the types, on the one hand, it can greatly reduce the steps of the handshake process between the two parties, which is especially important for the weak network of the industrial Internet of Things; on the other hand, it can greatly reduce the code volume, which is also particularly important for the industrial Internet of Things devices, especially for the scenarios where the embedded devices are particularly sensitive to resource consumption.

[0054] In addition, in the industrial Internet of Things environment, the network to which a device is connected may change at any time. For example, a relay device failure or a device power-off and restart may both trigger a re-automatic network formation. As a result, the NAT proxy to which the device is connected may also change. Therefore, directly determining the device type by judging whether the source addresses are the same can effectively handle the scenario where the NAT type changes due to a change in the network environment.

[0055] S204. If the communication types of the first device and the second device are both of the first type, determine that the communication method between the first device and the second device is to communicate based on a P2P network.

[0056] When both the first device and the second device are of the first type, it indicates that the P2P penetration rate between them is the highest. Therefore, a P2P tunnel connection can be selected to establish, so as to optimize the communication efficiency, while also ensuring the success probability of the P2P connection and reducing unnecessary resource consumption.

[0057] S205. Determine the tunnel negotiation method according to the communication quality between the first device and the second device.

[0058] In implementation, the method for determining the tunnel negotiation method includes: determining the sending order of probe packets based on the communication quality between the first device and the second device and the cloud server respectively; among them, the end with poorer communication quality sends a probe packet to the peer first after receiving the P2P tunnel creation notification, and the end with better communication quality sends a probe packet to the peer after a preset time interval after receiving the P2P tunnel creation notification; where the preset time interval is determined by the communication control service based on the network transmission delay corresponding to the first response message and the second response message.

[0059] Among them, the communication quality between the first device and the second device and the cloud server can be determined by the signaling interaction service 131 respectively based on the TCP connection status between the signaling interaction service 131 and the first device and the second device during the current session and the historical session, the transmission delay of the address reporting instruction, and the reception time of each first response message and each second response message, to determine the communication quality between the first device and the cloud server and between the second device and the cloud server respectively.

[0060] Taking the first device as an example, the signaling interaction service 131 obtains the stability S1 and transmission delay T1 of the TCP connection between the first device and the signaling interaction service 131 during the current session, the transmission delay T2 of the address reporting instruction, and the difference D1 between the average arrival time of each first response message and the average arrival time of each second response message, and combines the corresponding weights a and b to calculate the communication quality value Q1 corresponding to the first device. For example, Q1 can be calculated based on the formula: Q1=a(T1+T2) / S1+bD1, where the smaller the Q1 value, the higher the communication quality. The stability S1 can be determined based on the packet loss rate, disconnection probability, etc. of the TCP connection between the first device and the signaling interaction service 131 during the current session and historical sessions. The greater the packet loss rate or disconnection probability, the worse the stability and the smaller the S1 value.

[0061] The calculation method of the communication quality value Q2 corresponding to the second device is the same as that of the first device, and will not be repeated here.

[0062] The signaling interaction service 131 can determine the end with better quality and the end with worse quality by comparing the sizes of Q1 and Q2.

[0063] In one example, if it is determined that the communication quality of the second device is poor, the corresponding tunnel negotiation method includes: after receiving the P2P tunnel creation notification, the second device directly sends a UDP detection packet to the public network access address of the first device; after receiving the P2P tunnel creation notification, the first device delays for a preset period of time, such as 500 milliseconds, and then sends a UDP detection packet to the public network access address of the second device; when the second device receives the UDP detection packet from the first device, the P2P tunnel is established, and a UDP heartbeat packet is started to keep the connection alive.

[0064] In one example, the preset duration can be determined based on the maximum value of the network transmission delays corresponding to the first response message and the second response message. Specifically, when receiving each first response message and each second response message, the communication control service 132 records the corresponding reception time, determines the latest reception time, and synchronizes it to the signaling interaction service 131. The signaling interaction service 131 calculates the difference between the sending time of the address upload instruction and the latest reception time as the preset duration. The preset duration determined in this way can, on the one hand, largely ensure the smooth arrival of the first-sent probe packet, and on the other hand, minimize the impact of excessive waiting time on overall communication efficiency.

[0065] It should be noted that the sending time of the subsequent detection packet is limited based on a preset duration to ensure that the prior detection packet has been successfully sent before the subsequent detection packet is sent. That is, before the subsequent detection packet is sent, the prior detection packet has been received by the peer end. In this way, invalid detections can be avoided, thereby affecting the communication efficiency. Therefore, in the above example, if the first device does not receive the detection packet sent by the second device within the preset waiting duration, it can directly exit the tunnel establishment process, end the current session, and resend a remote control request to the signaling interaction service 131.

[0066] It should be noted that in the embodiments of this application, by setting the sending order of the UDP detection packets of both parties, the penetration rate of the P2P tunnel can be improved. The reason is that for the party that sends the detection packet first, it takes a certain amount of time to open the firewall of its NAT. Therefore, for the other party, only when the firewall of the other party is opened can the packets it sends enter. In this way, by limiting the sending order and the interval delay of the detection packets, it can be ensured that the detection packets can be normally received. Further, determining the end with better communication quality as the subsequent sender can, to a certain extent, ensure the smooth completion of the hole punching process.

[0067] S206. Send P2P tunnel creation notifications to the first device and the second device respectively.

[0068] Among them, the P2P tunnel creation notification carries the target access address and the tunnel negotiation method, enabling the first device and the second device to establish a P2P tunnel based on the target access address and the tunnel negotiation method. At the same time, the signaling interaction service 131 forwards the remote control request to the second device.

[0069] Specifically, the signaling interaction service 131 generates P2P tunnel creation notifications for the first device and the second device respectively. Among them, the P2P tunnel creation notifications sent to the first device and the second device both carry the target access address, which is the public network access address of the other party. The signaling interaction service 131 sends the P2P tunnel creation notifications to the first device and the second device respectively based on the previously created TCP connection.

[0070] In one example, after the signaling interaction service 131 sends the P2P tunnel creation notification to the second device, it also forwards the remote control request sent by the first device to the second device, enabling the second device to respond to the remote control request. After the second communication module 122 of the second device receives the remote control request, it sends the remote control request to the second VNC module 121 based on the local TCP connection, so that the second VNC module 121 can respond to the remote control request.

[0071] After receiving the P2P tunnel creation notification, the first device and the second device start to create a P2P tunnel and conduct remote control communication. For the specific process, please refer to Figure 3 .

[0072] As Figure 3 shown, the method for the first device and the second device to create a P2P tunnel specifically includes the following steps.

[0073] S301, determine the second access address of the second device and the tunnel negotiation method based on the P2P tunnel creation notification.

[0074] Specifically, after receiving the P2P tunnel creation notification based on the TCP connection between the first communication module 112 and the signaling interaction service 131, the first communication module 112 analyzes the notification message to obtain the target access address carried in the notification message, determines it as the second access address of the second device, that is, the public network access address of the second device, and at the same time, determines the tunnel negotiation method.

[0075] S302, send a first probe packet to the second access address according to the tunnel negotiation method.

[0076] As described above, the tunnel negotiation method is determined based on the communication quality between the first device and the second device. Correspondingly, in this step, the first device sending a first probe packet to the second access address of the second device according to the tunnel negotiation method may include directly sending a first probe packet to the second access address after receiving the P2P tunnel creation notification; or, waiting for a preset duration after receiving the P2P tunnel creation notification and then sending the first probe packet.

[0077] In an example, when the first device is the later sender, if it does not receive a second probe packet before sending the first probe packet, it determines that a P2P tunnel cannot be established and resends a remote control request to the cloud server to avoid invalid probing and waiting, thereby improving communication efficiency.

[0078] S303, after receiving the second probe packet sent by the second device, determine that the P2P tunnel is successfully established.

[0079] S304, after determining that the P2P tunnel is successfully established, send a first data verification packet to the second access address based on the P2P tunnel.

[0080] S305, after receiving the second data verification packet in response to the first data verification packet sent by the second device, establish a remote control connection with the second device based on the P2P tunnel.

[0081] Among them, the first data verification packet carries a remote control permission code for the second VNC module to perform permission verification on the first VNC module, and after the verification passes, returns remote control connection data.

[0082] On the one hand, the first data verification packet is used for both parties to establish a data transmission tunnel. Specifically, after the first device determines that the P2P tunnel is successfully established, it sends the first data verification packet carrying the remote control permission code to the second device based on the P2P tunnel. After receiving the first data verification packet, the second device verifies the first data verification packet through the second VNC module to determine the legality of the data packet. For example, it verifies the permission code to determine whether it is the master control party that issues the remote control request. If it is legal, it sends the second data verification packet to respond to the first device. After receiving the second data verification packet, the first device determines that the data channel is connected and starts port listening to prepare to receive the remote control connection data from the second device based on the P2P tunnel.

[0083] On the other hand, the first data verification packet is also used to trigger the second communication module to forward the remote control connection data sent by the second VNC module in response to the remote control request through the P2P tunnel to the first communication module, so as to start establishing a remote control connection, thereby completing the response to the remote control request. The remote control connection data is forwarded by the first communication module based on the local TCP connection to the first VNC module, and the first VNC module can establish a remote control connection with the second VNC module based on the remote control connection data.

[0084] Specifically, when the second VNC module receives the remote control request forwarded by the signaling interaction service, it first verifies the legality of the request based on the permission code sent by the signaling interaction service, and after the verification passes, it starts to respond to the remote control request and returns the remote control connection data. The remote control connection data is received and intercepted by the second communication control module based on the local TCP connection. When the first data verification packet passes the verification, the second communication control module sends the remote control connection data to the first device through the P2P tunnel.

[0085] Among them, the second communication control module determines whether to intercept the remote control connection data according to the notification type received after the address upload instruction. Among them, when the received is a P2P tunnel creation notification, it is determined to intercept the remote control connection data. When the received is a forwarding notification, the remote control connection data is directly forwarded to the signaling interaction service. The forwarding notification is sent by the signaling interaction service to the second device when it determines that the first device and the second device do not communicate based on the P2P network.

[0086] As described above, after the signaling interaction service 131 sends the P2P tunnel creation notification, it forwards the remote control request sent by the first device to the second transmission module 122 on the second device. After receiving the remote control request, the second transmission module 122 forwards it to the second VNC module 121 based on the local TCP connection to request the second VNC module 121 to process the remote control request. After receiving the remote control request, the second VNC module 121 starts to establish a remote control connection with the first VNC module 111, including sending remote control connection data such as the protocol version number to the first VNC module 111 for handshake authentication.

[0087] The second VNC module 121 sends the remote control connection data to the second communication module 122 based on the local TCP connection. The second communication module 122 intercepts the remote control connection data first, and after the first data verification packet passes the verification, it forwards the remote control connection data to the first device through the P2P tunnel. The first communication module 112 of the first device receives and forwards the remote control connection data to the first VNC module 111. Thus, remote control interaction can be performed between the first VNC module 111 and the second VNC module 121 based on the P2P tunnel.

[0088] In one implementation, after the first VNC module 111 and the second VNC module 121 complete the establishment of the remote control connection, that is, after the handshake and authentication are completed, the second VNC module 121 obtains the control interface of the second device and generates corresponding control view data, which is sent to the first communication module 112 through the P2P tunnel. After receiving the control view data, the first communication module 112 forwards it to the first VNC module 111 based on the local TCP connection. The first VNC module 111 renders the control view in real time to the remote control window for display to the user. In this way, the user can operate on the remote control window to achieve remote control of the second device.

[0089] To ensure data transmission security, the first data verification packet, the second data verification packet, and the remote control interaction data are all constructed based on the QUIC (Quick UDP Internet Connection) protocol. To enable the first data verification packet and the second data verification packet to be correctly identified, they can be set as non-standard data packets to distinguish them from other data packets. Among them, a non-standard data packet is a data packet that can be distinguished from widely used standard data packets, such as the 0xff data packet.

[0090] Based on the above technical solution, after the first device and the second device complete the establishment of the P2P tunnel, the connection of the data transmission channel is achieved through the interaction of the first data verification packet and the second data verification packet, so that the response data of the second VNC module for the remote control request can be transmitted to the first VNC module, thus enabling the smooth progress of the establishment of the remote control connection. Otherwise, after the first VNC module issues the remote control connection, it will always be in the handshake information waiting state and will not actively send data to the second VNC module before receiving the version negotiation message sent by the second VNC module. Therefore, the response data of the second VNC module for the remote control request cannot be transmitted based on the P2P tunnel. Based on this, the first communication module actively sends the first data verification packet to the second communication module to notify the second communication module that it can transmit remote control-related data based on the P2P tunnel, so that the response data of the second VNC module for the remote control request, that is, the version negotiation notification, can be sent to the first communication module based on the P2P tunnel and finally received by the first VNC module.

[0091] In the above embodiment, after the second device receives the P2P tunnel creation notification, the process of detecting the P2P tunnel with the first device based on the target access address and the P2P tunnel negotiation method is the same as that of the first device and will not be elaborated. It is worth noting that when the second communication module of the second device receives the response data of the second VNC module for the remote control request, it first caches the response data and then forwards the response data after successfully sending the second data verification packet.

[0092] In some other embodiments of the present application, there may be a situation where the P2P tunnel cannot be successfully created between the first device and the second device. For such situations, when the second device caches the response data, it simultaneously sets a timer task. When the first data verification packet is not received after exceeding the preset duration, the response data is automatically sent to the signaling interaction service 131, and the signaling interaction service 131 forwards the response data, thereby achieving a timely response to the remote control request.

[0093] In the above step S203, if it is determined that the communication type of the first device or the second device is the second type, in one implementation, in order to ensure the smooth implementation of remote control, the signaling interaction service 131 can directly forward the remote control request and abandon the creation of the P2P tunnel, that is, perform the forwarding process in the existing manner.

[0094] In another implementation, in order to improve the remote data transmission rate between the first device and the second device, the signaling interaction service 131 can first obtain the edge proxy near the first device or the second device. Among them, the edge proxy includes the proxy service deployed in the industrial Internet of Things and the proxy service deployed on the user network side, which is configured with a fixed public network access address and has the ability to forward messages.

[0095] When the signaling interaction service 131 determines that both the first device and the second device belong to the second type, it indicates that a P2P tunnel cannot be established between the first device and the second device. Optionally, the signaling interaction service 131 can respectively obtain the location information of the first device and the second device, and then determine all edge agents whose distances from the first device and the second device are within the threshold range, and select the edge agent with the minimum sum of distances from the first device and the second device as the target edge agent.

[0096] The signaling interaction service 131 respectively generates corresponding P2P tunnel creation notifications for the first device, the second device, and the target edge agent to notify the establishment of a P2P connection between the first device and the target edge agent and the establishment of a P2P connection between the second device and the target edge agent.

[0097] Among them, in the P2P tunnel creation notification generated by the signaling interaction service 131 for the target edge agent, the target access address includes the public network access addresses of the first device and the second device, and the P2P tunnel negotiation method is to directly send the third probe packet to the public network access addresses of the first device and the second device; in the P2P tunnel creation notifications generated by the signaling interaction service 131 for the first device and the second device, the target access address is the public network access address of the target edge agent, and the P2P tunnel negotiation method includes sending the fourth probe packet to the public network access address of the target edge agent at an interval of a preset duration after receiving the notification. It should be noted that the target access addresses received by the first device and the second device are both the public network access address of the target edge agent, but they are different from each other. In one example, the IP addresses are the same but the port numbers are different.

[0098] After the first device, the second device, and the target edge agent receive the P2P tunnel creation notification, they complete the negotiation and establishment of the corresponding P2P tunnel, and the target edge agent connects the two ports for communicating with the first device and the second device respectively to achieve the transparent transmission of data received and sent by the two ports.

[0099] In one example, a port forwarding relationship can be established based on the device identifiers of the first device and the second device, so that the target edge agent can determine the forwarding port according to the device identifier of the received packet. The device identifier can be set when the first device and the second device register with the signaling interaction service and is synchronized by the signaling interaction service to the target edge agent.

[0100] In this way, the data sent by the first device to the target edge proxy through the P2P tunnel can be transmitted to the second device through the P2P tunnel between the target edge proxy and the second device. Thus, the data transmission between the first device and the second device is achieved. Compared with the direct forwarding via the cloud server, the communication real-time performance is better, while reducing the data processing pressure on the cloud server and saving the bandwidth operation cost of the cloud, thereby ensuring the load capacity, concurrency performance and stability of the cloud.

[0101] An embodiment of the present application relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the embodiments of the above method are implemented.

[0102] Those skilled in the art can understand that all or part of the steps in implementing the above method embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions for causing a device (such as a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks and other various media that can store program codes.

[0103] The above are all the preferred embodiments of the present application. The protection scope of the present application is not limited by this. Therefore, all equivalent changes made according to the structure, shape and principle of the present application shall be covered within the protection scope of the present application.

Claims

1. A remote control method for industrial Internet of Things equipment, characterized in that: Applied to a first device, the first device includes a first VNC module and a first communication module, the first VNC module sends a remote control request for a second device to the first communication module, the first communication module forwards the remote control request to a cloud server, the cloud server determines the communication mode between the first device and the second device, and when the communication mode is determined to be based on a P2P network, sends a P2P tunnel creation notification to the first device and the second device respectively, wherein the second device is located in an industrial Internet of Things and includes a second VNC module and a second communication module, the method comprising the steps of: The first communication module determines, in response to the P2P tunnel creation notification, a second access address of the second device and a tunnel negotiation mode based on the P2P tunnel creation notification; Sending a first detection packet to the second access address according to the tunnel negotiation method; After receiving the second detection packet sent by the second device, determining that the P2P tunnel is successfully established; After determining that the P2P tunnel is successfully established, sending a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used to trigger the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, wherein the remote control connection data is response data generated by the second VNC module after receiving the remote control request forwarded by the cloud server; The first communication module forwards the remote control connection data to the first VNC module based on the local TCP connection; The first VNC module establishes a remote control connection with the second VNC module based on the remote control connection data.

2. The method according to claim 1, characterized in that The sending of the first detection packet to the second access address based on the tunnel negotiation method includes: directly sending the first detection packet to the second access address; or, After receiving the P2P tunnel creation notification and after a preset time interval, the first detection packet is sent to the second access address.

3. The method according to claim 2, characterized in that The method of sending the first detection packet to the second access address after receiving the P2P tunnel creation notification and after a preset time interval includes: If a second detection packet is received from the second device before sending the first detection packet, sending the first detection packet to the second access address; If the second detection packet is not received before the first detection packet is sent, the remote control request is resent to the cloud server.

4. The method according to claim 1, wherein Before responding to the P2P tunnel creation notification step, the method further includes: The first communication module responds to the address reporting instruction sent by the cloud server and sends response messages to multiple designated ports of the cloud server respectively, so that the cloud server determines the communication type of the first device based on the source address information of the multiple response messages, and determines the tunnel negotiation method based on the communication type of the first device.

5. The method according to claim 1, characterized in that The method further comprises: The first communication module receives the control view data returned by the second device based on the P2P tunnel, and forwards the control view data to the first VNC module; The first VNC module renders the control view data to a remote control window in real time.

6. A remote control method for industrial Internet of Things equipment, characterized in that: Applied to a cloud server, the cloud server is communicatively connected to a first device and a second device respectively, the second device is located in an industrial Internet of Things, the first device sends a remote control request to the cloud server to request remote control of the second device, and a signaling interaction service and a communication control service are deployed on the cloud server. The method includes: The signaling interaction service sends an address reporting instruction to the first device and the second device in response to the remote control request, so as to instruct the first device and the second device to send a response message to each designated port, and notify the communication control service to monitor the designated port; The communication control service monitors each of the designated ports to receive a first response message sent by the first device to each of the designated ports, and a second response message sent by the second device to each of the designated ports; The signaling interaction service determines the communication type of the first device and the second device based on the plurality of first response messages and the plurality of second response messages respectively; If the communication types of the first device and the second device are both the first type, determining that the communication mode between the first device and the second device is communication based on a P2P network; determining a tunnel negotiation mode according to communication quality between the first device and the second device; A P2P tunnel creation notification is sent to the first device and the second device respectively, and the remote control request is forwarded to the second device; the P2P tunnel creation notification carries the target access address and the tunnel negotiation method, so that the first device and the second device establish a P2P tunnel based on the target access address and the tunnel negotiation method.

7. The method according to claim 6, characterized in that The method for determining the communication type of the first device based on the multiple first response messages respectively includes: Determine whether the source addresses of the first response messages are the same; if they are the same, determine that the communication type of the first device is the first type.

8. The method according to claim 6, characterized in that Determining the tunnel negotiation mode according to the communication quality between the first device and the second device includes: Based on the communication quality between the first device and the second device and the cloud server respectively, the order of sending the detection packets is determined; wherein, after receiving the P2P tunnel creation notification, the end with poor communication quality first sends a detection packet to the other end, and after receiving the P2P tunnel creation notification, the end with good communication quality sends a detection packet to the other end at a preset time interval; wherein, the preset time is determined based on the maximum value of the network transmission delay corresponding to the first response message and the second response message.

9. The method according to claim 6, characterized in that The method further comprises: If the communication type of the first device or the second device is the second type; The remote control request is directly forwarded to the second device, and upon receiving the control view data returned by the second device, the control view data is forwarded to the first device.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images