Remote control method of industrial Internet of Things equipment and computer readable storage medium

Abstract

The invention relates to the technical field of industrial Internet of Things data processing, and provides a remote control method of industrial Internet of Things equipment and a computer readable storage medium. The method comprises the following steps: the first communication module responds to the P2P tunnel creation notification, and determines a second access address and a tunnel negotiation mode of the second equipment based on the P2P tunnel creation notification; sending a first detection packet to the second access address according to the tunnel negotiation mode; after a second detection packet sent by the second equipment is received, determining that the P2P tunnel is successfully established; after determining that the P2P tunnel is successfully established, sending a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used for triggering the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, so that the data transmission efficiency between remote control equipment is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of industrial Internet of Things data processing, and in particular to a remote control method for industrial Internet of Things equipment and a computer-readable storage medium. Background Art

[0002] The Industrial Internet of Things is based on the Internet of Things technology, and realizes the digitization, intelligence and automation of industrial production and operation processes by connecting and interconnecting industrial equipment, sensors, controllers, systems and personnel. With the rapid development of the Industrial Internet of Things technology, more and more companies hope to not only monitor and control production on site through industrial control equipment such as HMI (Human Machine Interface) equipment, but also hope to remotely connect to HMI equipment through mobile Internet devices such as mobile phones, and operate and control HMI equipment just like local operations.

[0003] At present, the industry has proposed solutions to meet the above requirements, including remote control of HMI devices by mobile devices based on VNC (Virtual Network Console), and data forwarding between the two during the remote control process by the cloud server. It can be seen that the remote control delay between the HMI device and the mobile device will be equal to the delay between the HMI device and the cloud server, plus the delay between the cloud server and the mobile device.

[0004] However, when remote control operations are frequent, the VNC window will experience noticeable freezes and delays, which seriously affects the user experience and may cause misoperations. Summary of the invention

[0005] In order to improve the data transmission efficiency between the master terminal and the controlled terminal during the remote control process and enhance the user experience, the embodiments of the present application provide a remote control method and a computer storage medium for an industrial Internet of Things device.

[0006] On the one hand, the remote control method of an industrial Internet of Things device provided in an embodiment of the present application is applied to a first device, wherein the first device includes a first VNC module and a first communication module, the first VNC module sends a remote control request for a second device to the first communication module, the first communication module forwards the remote control request to a cloud server, the cloud server determines the communication mode between the first device and the second device, and when the communication mode determined is based on a P2P network, sends a P2P tunnel creation notification to the first device and the second device respectively, wherein the second device is located in the industrial Internet of Things and includes a second VNC module and a second communication module, and the method includes the steps of: the first communication module responds to the P2P tunnel creation notification, determines a second access address of the second device based on the P2P tunnel creation notification and Tunnel negotiation method; sending a first detection packet to the second access address according to the tunnel negotiation method; after receiving the second detection packet sent by the second device, determining that the P2P tunnel is successfully established; after determining that the P2P tunnel is successfully established, sending a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used to trigger the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, wherein the remote control connection data is the response data generated by the second VNC module after receiving the remote control request forwarded by the cloud server; the first communication module forwards the remote control connection data to the first VNC module based on the local TCP connection; the first VNC module establishes a remote control connection with the second VNC module based on the remote control connection data.

[0007] Based on the above technical solution, a P2P connection between the first device and the second device can be achieved, so that remote control data can be transmitted between the first device and the second device based on the P2P tunnel, thereby improving data transmission efficiency, optimizing delays and freezes, improving user experience, and reducing the load and bandwidth costs of the cloud service end; at the same time, after the P2P tunnel is established, the second device can be triggered based on the first data verification packet to send the remote control connection data directly to the first device through the P2P tunnel, thereby achieving a normal response to the remote control request.

[0008] In one implementation, sending the first detection packet to the second access address based on the tunnel negotiation method includes: sending the first detection packet directly to the second access address; or sending the first detection packet to the second access address after receiving the P2P tunnel creation notification and after a preset time interval.

[0009] Based on the above technical solution, the order of sending and receiving the P2P detection packet is limited to ensure that the detection packet can be successfully identified and received by the second device, thereby improving the penetration rate of P2P.

[0010] In one implementation, the method of sending the first detection packet to the second access address after receiving the P2P tunnel creation notification and at a preset time interval includes: if a second detection packet is received from the second device before sending the first detection packet, sending the first detection packet to the second address; if the second detection packet is not received before sending the first detection packet, resending the remote control request to the cloud server.

[0011] Based on the above technical solution, the first device determines whether the P2P tunnel can be successfully created by receiving the first detection packet. If it is found that the P2P tunnel cannot be successfully created, the request can be resent in time, thereby avoiding invalid waiting and improving communication efficiency.

[0012] In one implementation, before responding to the P2P tunnel creation notification step, the method also includes: the first communication module responds to the address reporting instruction sent by the cloud server, and sends response messages to multiple designated ports of the cloud server respectively, so that the cloud server determines the communication type of the first device based on the source address information of the multiple response messages, and determines the tunnel negotiation mode based on the communication type of the first device.

[0013] In one implementation, the method further includes: the first communication module receives control view data returned by the second device based on the P2P tunnel, and forwards the control view data to the first VNC module; the first VNC module renders the control view data to the remote control window in real time.

[0014] On the other hand, the remote control method of an industrial Internet of Things device provided in an embodiment of the present application is applied to a cloud server, wherein the cloud server is communicatively connected to a first device and a second device respectively, the second device is located in the industrial Internet of Things, the first device sends a remote control request to the cloud server to request remote control of the second device, and a signaling interaction service and a communication control service are deployed on the cloud server. The method includes: the signaling interaction service sends an address reporting instruction to the first device and the second device in response to the remote control request to instruct the first device and the second device to send a response message to each designated port, and notifies the communication control service to monitor the designated port; the communication control service monitors each of the designated ports to receive a first response message sent by the first device to each of the designated ports, so as to and a second response message sent by the second device to each of the designated ports; the signaling interaction service determines the communication type of the first device and the second device based on multiple first response messages and multiple second response messages respectively; if the communication types of the first device and the second device are both the first type, determine that the communication mode between the first device and the second device is based on a P2P network for communication; determine a tunnel negotiation mode according to the communication quality of the first device and the second device; send a P2P tunnel creation notification to the first device and the second device respectively and forward the remote control request to the second device; the P2P tunnel creation notification carries a target access address and the tunnel negotiation mode, so that the first device and the second device establish a P2P tunnel based on the target access address and the tunnel negotiation mode.

[0015] Based on the above technical solution, the cloud server determines the communication type between the first device and the second device according to the response messages received by multiple ports, and then generates a corresponding P2P tunnel creation notification, which is sent to the first device and the second device respectively, so that the first device and the second device can directly complete the negotiation and establishment of the P2P tunnel according to the negotiation method in the creation notification, thereby realizing point-to-point transmission of remote control data between the two, improving transmission efficiency, improving user experience, and ensuring the penetration rate of the P2P tunnel.

[0016] In one implementation, determining the communication type of the first device based on the multiple first response messages respectively includes: determining whether the source addresses of the first response messages are the same, and if they are the same, determining that the communication type of the first device is the first type.

[0017] Based on the above technical solution, the communication type of the device is determined directly according to the source address of the response message. Compared with the existing direct request NAT type, it is more suitable for the application scenarios of the industrial Internet of Things, and can improve the accuracy of judging the communication type of the device, thereby avoiding the problem of P2P tunnel failure due to misjudgment of the type.

[0018] In one implementation, the tunnel negotiation method determined according to the communication quality between the first device and the second device includes: determining the order of sending detection packets based on the communication quality between the first device and the second device and the cloud server respectively; wherein, after receiving the P2P tunnel creation notification, the end with poor communication quality first sends a detection packet to the other end, and after receiving the P2P tunnel creation notification, the end with good communication quality sends a detection packet to the other end at a preset time interval; wherein, the preset time is determined based on the maximum value of the network transmission delay corresponding to the first response message and the second response message.

[0019] In one implementation, the method further includes: if the communication type of the first device or the second device is the second type; directly forwarding the remote control request to the second device, and upon receiving the control view data returned by the second device, forwarding the control view data to the first device.

[0020] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings constituting a part of the present application are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1A A schematic diagram of the first device structure provided in an embodiment of the present application is shown.

[0024] Figure 1B A schematic diagram of the second device structure provided in an embodiment of the present application is shown.

[0025] Figure 1C A schematic diagram of the cloud server structure provided in an embodiment of the present application is shown.

[0026] Figure 1D A schematic diagram of the structure of a remote control service system provided in an embodiment of the present application is shown.

[0027] Figure 2 A flow chart of a remote control method for industrial Internet of Things devices provided in an embodiment of the present application is shown.

[0028] Figure 3 A flow chart of a remote control method for industrial Internet of Things devices provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, and "first", "second" and various digital numbers are only distinguished for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0031] The features, structures or characteristics in this application may be combined in one or more embodiments in any suitable manner. In various embodiments of this application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0032] Some optional features in the embodiments of the present application may be implemented independently in some scenarios without relying on other features to solve corresponding technical problems and achieve corresponding effects. They may also be combined with other features in some scenarios as needed.

[0033] In this application, unless otherwise specified, the same or similar parts between the various embodiments can refer to each other. In the various embodiments of this application, if there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0034] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0035] The remote control method of the industrial Internet of Things device provided in the embodiment of the present application is used to realize the remote control of the second device located in the industrial Internet of Things by the first device. In actual applications, the first device can be a user's smart terminal device, including but not limited to mobile phones, computers, smart TVs, etc.; the second device can be a control device such as an HMI device.

[0036] Please refer to Figure 1A The first device 11 provided in the embodiment of the present application includes a first VNC module 111 and a first communication module 112, wherein the first VNC module 111 is implemented based on a VNC Viewer program installed on the first device 11, and is used to request remote control of a VNC Server. The first communication module 112 is a self-developed module, and is used to forward the data sent and received by the first VNC module 111.

[0037] Please refer to Figure 1B The second device 12 provided in the embodiment of the present application includes a second VNC module 121 and a second communication module 122, wherein the second VNC module 121 is implemented based on a VNC Server program installed on the second device 12, and is used to respond to a remote control request of a VNC Viewer. The second communication module 122 is a self-developed module, and is used to forward the received and sent data of the second VNC module 121.

[0038] Please refer to Figure 1C The remote control method provided in the embodiment of the present application also involves a cloud server 13, on which a signaling interaction service 131 and a communication control service 132 are deployed.

[0039] The first device 11, the second device 12 and the cloud server 13 can constitute a remote control service system, please refer to Figure 1D The signaling interaction service 131 can establish communication connections with the first communication module 112 and the second communication module 122 respectively, and the communication control service 132 is used to obtain the public network access addresses of the first communication module 112 and the second communication module 122.

[0040] In order to realize the remote control of the second device by the first device, the device needs to be initialized first. In one example, the initialization of the first device includes starting the first communication module 112, initiating a TCP connection to the signaling interaction service 131 through the first communication module 112, and registering as a control end role (visitor); starting the first VNC module 111, and establishing a local TCP connection with the first communication module 112. The initialization of the second device includes starting the second communication module 122, initiating a TCP connection to the signaling interaction service 131 through the second communication module 122, and registering as a device end role; starting the second VNC module, and establishing a local TCP connection with the second communication module 122.

[0041] After the initialization is completed, the first VNC module 111 can initiate a remote control request for the second device to the first communication module 112 based on the local TCP connection. When receiving the remote control request, the first communication module 112 can first determine whether a P2P tunnel already exists with the second communication device. If so, the remote control request is directly forwarded based on the P2P tunnel. If not, the remote control request is forwarded to the signaling interaction service 131 of the cloud server 13, and a remote control permission code is sent to the signaling interaction service 131 at the same time, so that the signaling interaction service 131 can access the second VNC module based on the permission code, that is, after forwarding the remote control request to the second VNC module, the second VNC module verifies the legitimacy of the remote control request based on the permission code sent synchronously, and starts to respond to the remote control request if the verification passes.

[0042] When receiving a remote control request, the signaling interaction service 131 can determine whether the second device is online based on the status of the TCP connection between the second device and the second device. If the TCP connection is valid, the second device is determined to be online; if the connection is disconnected, the second device is determined to be offline and the remote control request is rejected.

[0043] When the second device is online, refer to Figure 2 In the remote control method provided in the embodiment of the present application, the method for processing the remote control request by the cloud server includes the following steps.

[0044] S201, in response to a remote control request, sending an address reporting instruction to a first device and a second device.

[0045] Specifically, after the signaling interaction service 131 receives a remote control request from the first device based on a TCP connection, if it is determined that the second device is online, it first sends an address reporting instruction to the first device and the second device based on the established TCP connection to obtain the public network access addresses of the two devices, wherein the address reporting instruction is used to instruct the first device and the second device to send response messages to multiple designated ports, wherein the designated ports are local ports on the cloud server, and the response messages are generated based on the UDP protocol.

[0046] At the same time, the signaling interaction service 131 notifies the communication control service 132 to monitor each designated port to receive response messages from the first device and the second device. The response messages are transmitted based on the UDP protocol.

[0047] S202: Monitor each designated port to receive a first response message sent by the first device to each designated port and a second response message sent by the second device to each designated port.

[0048] Please combine Figure 1D The communication control service 132 starts a corresponding number of service processes according to the number of designated ports to monitor the corresponding designated ports respectively, and when receiving the first response message and the second response message from the first device and the second device, extracts the source address of the message and sends it to the signaling interaction service 131.

[0049] It is worth noting that if the number of the first response message or the second response message is inconsistent with the corresponding designated port number, that is, the response message is lost or not sent, the communication control service 132 can directly send a communication abnormality notification to the signaling interaction service 131 to end the P2P tunnel creation process and switch to a communication mode in which the signaling delivery service 131 forwards the data in the remote control process.

[0050] S203: Determine the communication type of the first device and the second device based on the multiple first response messages and the multiple second response messages respectively.

[0051] In implementation, after receiving multiple source addresses extracted from multiple first response messages by the communication control service 132, the signaling interaction service 131 determines whether the source addresses are the same, and if they are the same, determines that the first device is of the first type. The first type is a type that can create a P2P tunnel. The same source address means that the IP address and port number are the same, that is, the four-tuple is the same.

[0052] The method for determining the communication type of the second device is the same as that of the first device and will not be repeated here.

[0053] It is worth noting that in the embodiment of the present application, the signaling interaction service 131 determines whether the device is suitable for creating a P2P tunnel based on whether the source addresses of the response messages received by multiple designated ports are the same. Compared with the prior art method of determining whether to create a P2P tunnel by obtaining the NAT type, it is more suitable for the industrial Internet of Things. The reason is that the existing NAT technology divides NAT into 5 types, which is a large and comprehensive classification obtained from a theoretical analysis. In the embodiment of the present application, combined with the actual application process, the implementation level of P2P technology in the industrial Internet of Things is considered. With regard to P2P connectivity, NAT is simplified into two types: a first type that can be connected and a second type that is difficult to connect. In this way, by simplifying the types, on the one hand, the handshake process steps of both parties can be greatly reduced, which is especially important for weak networks such as the industrial Internet of Things; on the other hand, the amount of code can be greatly reduced, which is also particularly important for industrial Internet of Things devices, especially embedded devices that are particularly sensitive to resource consumption.

[0054] In addition, in an industrial IoT environment, the network to which the device is connected may change at any time. For example, if a transit device fails or the device is powered off and restarted, it may automatically re-network. In this case, the NAT proxy to which the device is connected may also change. Therefore, directly determining the device type by judging whether the source address is consistent can effectively deal with scenarios where the NAT type changes due to changes in the network environment.

[0055] S204: If the communication types of the first device and the second device are both the first type, determine that the communication method between the first device and the second device is to communicate based on a P2P network.

[0056] When the first device and the second device are both of the first type, it indicates that the P2P penetration rate between the two is the highest, so a P2P tunnel connection can be selected to optimize communication efficiency, while also ensuring the success probability of the P2P connection and reducing unnecessary resource consumption.

[0057] S205: Determine a tunnel negotiation method according to the communication quality between the first device and the second device.

[0058] In implementation, the method for determining the tunnel negotiation mode includes: determining the order of sending detection packets based on the communication quality between the first device and the second device and the cloud server respectively; wherein, after receiving the P2P tunnel creation notification, the end with poor communication quality first sends a detection packet to the other end, and after receiving the P2P tunnel creation notification, the end with good communication quality sends a detection packet to the other end at a preset time interval; wherein, the preset time is determined by the communication control service based on the network transmission delay corresponding to the first response message and the second response message.

[0059] Among them, the communication quality between the first device and the second device and the cloud server can be determined by the signaling interaction service 131 based on the TCP connection status between the first device and the second device during the current session and the historical session, the transmission delay of the address reporting instruction, the reception time of each first response message and each second response message, to respectively determine the communication quality between the first device and the cloud server, and between the second device and the cloud server.

[0060] Taking the first device as an example, the signaling interaction service 131 obtains the stability S1 and transmission delay T1 of the TCP connection between the first device and the signaling interaction service 131 during the current session, the transmission delay T2 of the address reporting instruction, and the difference D1 between the average arrival time of each first response message and the average arrival time of each second response message, and combines the corresponding weights a and b to calculate the communication quality value Q1 corresponding to the first device. For example, Q1 can be calculated based on the formula: Q1=a(T1+T2) / S1+bD1, where the smaller the Q1 value, the higher the communication quality. The stability S1 can be determined based on the packet loss rate, disconnection probability, etc. of the TCP connection between the first device and the signaling interaction service 131 during the current session and historical sessions. The larger the packet loss rate or disconnection probability, the worse the stability and the smaller the S1 value.

[0061] The calculation method of the communication quality value Q2 corresponding to the second device is the same as that of the first device, which will not be repeated here.

[0062] The signaling interaction service 131 can determine the end with better quality and the end with worse quality by comparing the sizes of Q1 and Q2.

[0063] In one example, if it is determined that the communication quality of the second device is poor, the corresponding tunnel negotiation method includes: after receiving the P2P tunnel creation notification, the second device directly sends a udp detection packet to the public network access address of the first device; after receiving the P2P tunnel creation notification, the first device delays for a preset period of time, such as 500 milliseconds, and then sends a udp detection packet to the public network access address of the second device; when the second device receives the udp detection packet from the first device, the P2P tunnel is established, and a udp heartbeat packet is started to keep the connection alive.

[0064] In one example, the preset duration can be determined based on the maximum value of the network transmission delays corresponding to the first response message and the second response message. Specifically, when the communication control service 132 receives each first response message and each second response message, it records the corresponding receiving time, determines the latest receiving time and synchronizes it to the signaling interaction service 131, and the signaling interaction service 131 calculates the difference between the sending time of the address upload instruction and the latest receiving time as the preset duration. The preset duration determined in this way can, on the one hand, guarantee the smooth arrival of the first-sent detection packet to a large extent, and on the other hand, it can minimize the impact of excessive waiting on the overall communication efficiency.

[0065] It is worth noting that the sending time of the subsequent detection packet is limited based on the preset duration in order to ensure that the prior detection packet has been successfully sent before the subsequent detection packet is sent, that is, the prior detection packet has been received by the other end before the subsequent detection packet is sent. In this way, invalid detection can be avoided, thereby affecting communication efficiency. Therefore, in the above example, if the first device does not receive the detection packet sent by the second device within the preset waiting time, it can directly exit the tunnel establishment process, end the session, and re-send the remote control request to the signaling interaction service 131.

[0066] It is worth noting that in the embodiment of the present application, by setting the sending order of the UDP detection packets of both parties, the penetration rate of the P2P tunnel can be improved. The reason is that: for the party that sends the detection packet first, it takes a certain amount of time to open the firewall of the NAT where it is located. Therefore, for the other party, only when the firewall of the other party is opened can the packet it sends enter. In this way, by limiting the sending order of the detection packet and the interval delay, it can be ensured that the detection packet can be received normally. Furthermore, determining the end with good communication quality as the later sender can determine the smooth completion of the hole-making process to a certain extent.

[0067] S206: Send a P2P tunnel creation notification to the first device and the second device respectively.

[0068] The P2P tunnel creation notification carries the target access address and the tunnel negotiation mode, so that the first device and the second device establish a P2P tunnel based on the target access address and the tunnel negotiation mode. At the same time, the signaling interaction service 131 forwards the remote control request to the second device.

[0069] Specifically, the signaling interaction service 131 generates a P2P tunnel creation notification for the first device and the second device, respectively, wherein the P2P tunnel creation notifications sent to the first device and the second device both carry a target access address, which is the public network access address of the other party. The signaling interaction service 131 sends the P2P tunnel creation notification to the first device and the second device, respectively, based on the previously created TCP connection.

[0070] In one example, after sending a P2P tunnel creation notification to the second device, the signaling interaction service 131 also forwards the remote control request sent by the first device to the second device, so that the second device responds to the remote control request. After receiving the remote control request, the second communication module 122 of the second device sends the remote control request to the second VNC module 121 based on the local TCP connection, so that the second VNC module 121 responds to the remote control request.

[0071] After receiving the P2P tunnel creation notification, the first device and the second device start to create a P2P tunnel and perform remote control communication. For the specific process, please refer to Figure 3 .

[0072] like Figure 3 As shown, the method for the first device and the second device to create a P2P tunnel specifically includes the following steps.

[0073] S301, determining a second access address of a second device and a tunnel negotiation mode based on a P2P tunnel creation notification.

[0074] Specifically, after receiving the P2P tunnel creation notification based on the TCP connection between the first communication module 112 and the signaling interaction service 131, the first communication module 112 parses the notification message to obtain the target access address carried in the notification message, and determines it as the second access address of the second device, that is, the public network access address of the second device, and at the same time, determines the tunnel negotiation method.

[0075] S302: Send a first detection packet to the second access address according to the tunnel negotiation method.

[0076] As described above, the tunnel negotiation method is determined based on the communication quality between the first device and the second device. Accordingly, in this step, the first device sending the first detection packet to the second access address of the second device according to the tunnel negotiation method may include directly sending the first detection packet to the second access address after receiving the P2P tunnel creation notification; or, after receiving the P2P tunnel creation notification, waiting for a preset time and sending the first detection packet to the second access address.

[0077] In one example, when the first device is the later sender, if the second detection packet is not received before sending the first detection packet, it is determined that the P2P tunnel cannot be established, and the remote control request is resent to the cloud server to avoid invalid detection and waiting, thereby improving communication efficiency.

[0078] S303: After receiving the second detection packet sent by the second device, determine that the P2P tunnel is successfully established.

[0079] S304: After determining that the P2P tunnel is successfully established, send a first data verification packet to the second access address based on the P2P tunnel.

[0080] S305: After receiving a second data verification packet from the second device in response to the first data verification packet, establish a remote control connection with the second device based on the P2P tunnel.

[0081] The first data verification package carries a remote control authority code for the second VNC module to perform authority verification on the first VNC module, and returns remote control connection data after the verification is passed.

[0082] On the one hand, the first data verification packet is used for both parties to establish a data transmission tunnel. Specifically, after the first device determines that the P2P tunnel is successfully established, it sends the first data verification packet carrying the remote control permission code to the second device based on the P2P tunnel. After receiving the first data verification packet, the second device verifies the first data verification packet through the second VNC module to determine the legitimacy of the data packet, such as verifying the permission code to determine whether it is the master who issued the remote control request. If it is legitimate, it sends the second data verification packet to respond to the first device. After receiving the second data verification packet, the first device determines that the data channel is connected and starts port monitoring to prepare to receive remote control connection data from the second device based on the P2P tunnel.

[0083] On the other hand, the first data verification packet is also used to trigger the second communication module to forward the remote control connection data sent by the second VNC module in response to the remote control request to the first communication module through the P2P tunnel to start establishing a remote control connection, thereby completing the response to the remote control request. The remote control connection data is forwarded to the first VNC module by the first communication module based on the local TCP connection, and the first VNC module can establish a remote control connection with the second VNC module based on the remote control connection data.

[0084] Specifically, when the second VNC module receives a remote control request forwarded by the signaling interaction service, it first verifies the legitimacy of the request based on the permission code sent by the signaling interaction service, and after the verification is passed, it starts to respond to the remote control request and returns the remote control connection data. The remote control connection data is received and intercepted by the second communication control module based on the local TCP connection. When the first data verification package is verified, the second communication control module sends the remote control connection data to the first device through the P2P tunnel.

[0085] The second communication control module determines whether to intercept the remote control connection data according to the notification type received after the address upload instruction, and when the notification received is a P2P tunnel creation notification, it is determined to intercept the remote control connection data, and when the notification received is a forwarding notification, the remote control connection data is directly forwarded to the signaling interaction service. The forwarding notification is sent to the second device by the signaling interaction service when it is determined that the first device and the second device communicate not based on the P2P network.

[0086] As described above, after sending the P2P tunnel creation notification, the signaling interaction service 131 forwards the remote control request sent by the first device to the second transmission module 122 on the second device. After receiving the remote control request, the second transmission module 122 forwards it to the second VNC module 121 based on the local TCP connection to request the second VNC module 121 to process the remote control request. After receiving the remote control request, the second VNC module 121 starts to establish a remote control connection with the first VNC module 111, including sending remote control connection data such as the protocol version number to the first VNC module 111 for handshake authentication.

[0087] The second VNC module 121 sends the remote control connection data to the second communication module 122 based on the local TCP connection. The second communication module 122 first intercepts the remote control connection data and forwards the remote control connection data to the first device through the P2P tunnel after the first data verification packet is verified. The first communication module 112 of the first device receives and forwards the remote control connection data to the first VNC module 111. At this point, the first VNC module 111 and the second VNC module 121 can perform remote control interaction based on the P2P tunnel.

[0088] In one implementation, when the first VNC module 111 and the second VNC module 121 complete the establishment of the remote control connection, that is, the handshake and authentication are completed, the second VNC module 121 obtains the control interface of the second device, and generates corresponding control view data and sends it to the first communication module 112 through the P2P tunnel. After receiving the control view data, the first communication module 112 forwards it to the first VNC module 111 based on the local TCP connection. The first VNC module 111 renders the control view in real time to the remote control window to display it to the user. In this way, the user can operate the remote control window to achieve remote control of the second device.

[0089] In order to ensure data transmission security, the first data verification package, the second data verification package and the remote control interaction data are all constructed based on the QUIC (Quick UDP Internet Connection) protocol, wherein in order to enable the first data verification package and the second data verification package to be correctly identified, they can be set as non-standard data packets to distinguish them from other data packets. Among them, the non-standard data packet is a data packet that can be distinguished from the widely used standard data packet, such as the 0xff data packet.

[0090] Based on the above technical solution, after the first device and the second device complete the establishment of the P2P tunnel, they realize the connection of the data transmission channel through the interaction of the first data verification package and the second data verification package, so that the response data of the second VNC module to the remote control request can be transparently transmitted to the first VNC module, thereby smoothly promoting the establishment of the remote control connection. Otherwise, since the first VNC module will be in a handshake information waiting state after issuing the remote control connection, it will not actively send data to the second VNC module before receiving the version negotiation message sent by the second VNC module. Therefore, the response data of the second VNC module to the remote control request cannot be transmitted based on the P2P tunnel. Based on this, the first communication module actively sends the first data verification package to the second communication module to notify the second communication module that the remote control related data can be transmitted based on the P2P tunnel, so that the response data of the second VNC module to the remote control request, that is, the version negotiation notification, can be sent to the first communication module based on the P2P tunnel and finally received by the first VNC module.

[0091] In the above embodiment, after receiving the P2P tunnel creation notification, the process of detecting the P2P tunnel between the second device and the first device based on the target access address and the P2P tunnel negotiation method is the same as that of the first device and will not be repeated. It is worth noting that when the second communication module of the second device receives the response data of the second VNC module to the remote control request, it first caches the response data and forwards the response data after successfully sending the second data verification packet.

[0092] In some other embodiments of the present application, there may be a situation where a P2P tunnel cannot be successfully created between the first device and the second device. For these situations, when the second device caches the response data, it also sets a timer task. When the first data verification packet is not received within a preset time period, the response data is automatically sent to the signaling interaction service 131, and the signaling interaction service 131 forwards the response data, thereby achieving a timely response to the remote control request.

[0093] In the above step S203, if it is determined that the communication type of the first device or the second device is the second type, in order to ensure the smooth implementation of remote control, in one implementation, the signaling interaction service 131 can directly forward the remote control request and abandon the creation of the P2P tunnel, that is, forwarding processing is performed in the existing manner.

[0094] In another implementation, in order to improve the remote data transmission rate between the first device and the second device, the signaling interaction service 131 may first obtain an edge agent near the first device or the second device, wherein the edge agent includes a proxy service deployed in the industrial Internet of Things and a proxy service deployed on the user network side, which is configured with a fixed public network access address and has message forwarding capabilities.

[0095] When the signaling interaction service 131 determines that the first device and the second device both belong to the second type, it indicates that a P2P tunnel cannot be established between the first device and the second device. Optionally, the signaling interaction service 131 may obtain the location information of the first device and the second device respectively, and then determine all edge proxies whose distances to the first device and the second device are within a threshold range, and select the edge proxy with the smallest sum of distances to the first device and the second device as the target edge proxy.

[0096] The signaling interaction service 131 generates corresponding P2P tunnel creation notifications for the first device, the second device and the target edge proxy respectively, so as to notify the first device to establish a P2P connection with the target edge proxy and the second device to establish a P2P connection with the target edge proxy.

[0097] Among them, in the P2P tunnel creation notification generated by the signaling interaction service 131 for the target edge agent, the target access address includes the public network access address of the first device and the second device, and the P2P tunnel negotiation method is to directly send a third detection packet to the public network access address of the first device and the second device; in the P2P tunnel creation notification generated for the first device and the second device, the target access address is the public network access address of the target edge agent, and the P2P tunnel negotiation method includes sending a fourth detection packet to the public network access address of the target edge agent at intervals of a preset time after receiving the notification. It is worth noting that the target access addresses received by the first device and the second device are both the public network access addresses of the target edge agent, but they are different. In one example, the IP addresses are the same but the port numbers are different.

[0098] After receiving the P2P tunnel creation notification, the first device, the second device and the target edge agent complete the negotiation and establishment of the corresponding P2P tunnel, and the target edge agent connects the two ports communicating with the first device and the second device respectively to achieve transparent transmission of data received and sent by the two ports.

[0099] In one example, a port forwarding relationship can be established based on the device identifiers of the first device and the second device, so that the target edge agent can determine the forwarding port according to the device identifier of the received message. The device identifier can be set when the first device and the second device register with the signaling interaction service, and synchronized to the target edge agent by the signaling interaction service.

[0100] In this way, the data sent by the first device to the target edge agent based on the P2P tunnel can be transmitted to the second device through the P2P tunnel between the target edge agent and the second device, thereby realizing data transmission between the first device and the second device. Compared with forwarding directly through the cloud server, the communication real-time performance is better, and at the same time, the data processing pressure of the cloud server is reduced and the bandwidth operating costs of the cloud are saved, thereby ensuring the load capacity, concurrency performance and stability of the cloud.

[0101] An embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above-mentioned method when executed by a processor.

[0102] Those skilled in the art can understand that all or part of the steps in the above-mentioned implementation method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A remote control method for industrial Internet of Things equipment, characterized in that: Applied to a first device, the first device includes a first VNC module and a first communication module, the first VNC module sends a remote control request for a second device to the first communication module, the first communication module forwards the remote control request to a cloud server, the cloud server determines the communication mode between the first device and the second device, and when the communication mode determined is based on a P2P network, sends a P2P tunnel creation notification to the first device and the second device respectively, wherein the second device is located in an industrial Internet of Things and includes a second VNC module and a second communication module, and the method includes the steps of: The first communication module determines, in response to the P2P tunnel creation notification, a second access address of the second device and a tunnel negotiation mode based on the P2P tunnel creation notification; Sending a first detection packet to the second access address according to the tunnel negotiation method; After receiving the second detection packet sent by the second device, determining that the P2P tunnel is successfully established; After determining that the P2P tunnel is successfully established, sending a first data verification packet to the second access address based on the P2P tunnel; the first data verification packet is used to trigger the second communication module to forward the intercepted remote control connection data to the first communication module through the P2P tunnel, wherein the remote control connection data is the response data generated by the second VNC module after receiving the remote control request forwarded by the cloud server; The first communication module forwards the remote control connection data to the first VNC module based on the local TCP connection; The first VNC module establishes a remote control connection with the second VNC module based on the remote control connection data.

2. The method according to claim 1, characterized in that The sending of the first detection packet to the second access address based on the tunnel negotiation method includes: directly sending the first detection packet to the second access address; or, After receiving the P2P tunnel creation notification and after a preset time interval, the first detection packet is sent to the second access address.

3. The method according to claim 2, characterized in that The method of sending the first detection packet to the second access address after receiving the P2P tunnel creation notification and after a preset time interval includes: If a second detection packet is received from the second device before the first detection packet is sent, sending the first detection packet to the second address; If the second detection packet is not received before the first detection packet is sent, the remote control request is resent to the cloud server.

4. The method according to claim 1, characterized in that: Before responding to the P2P tunnel creation notification step, the method further includes: The first communication module responds to the address reporting instruction sent by the cloud server and sends response messages to multiple designated ports of the cloud server respectively, so that the cloud server determines the communication type of the first device based on the source address information of the multiple response messages, and determines the tunnel negotiation mode based on the communication type of the first device.

5. The method according to claim 1, characterized in that The method further comprises: The first communication module receives the control view data returned by the second device based on the P2P tunnel, and forwards the control view data to the first VNC module; The first VNC module renders the control view data to a remote control window in real time.

6. A remote control method for industrial Internet of Things equipment, characterized in that: Applied to a cloud server, the cloud server is respectively connected to a first device and a second device in communication, the second device is located in an industrial Internet of Things, the first device sends a remote control request to the cloud server to request remote control of the second device, and a signaling interaction service and a communication control service are deployed on the cloud server, the method comprising: The signaling interaction service sends an address reporting instruction to the first device and the second device in response to the remote control request, so as to instruct the first device and the second device to send a response message to each designated port, and notify the communication control service to monitor the designated port; The communication control service monitors each of the designated ports to receive a first response message sent by the first device to each of the designated ports, and a second response message sent by the second device to each of the designated ports; The signaling interaction service determines the communication type of the first device and the second device based on the plurality of the first response messages and the plurality of the second response messages respectively; If the communication types of the first device and the second device are both the first type, determining that the communication mode between the first device and the second device is communication based on a P2P network; Determining a tunnel negotiation mode according to communication quality between the first device and the second device; A P2P tunnel creation notification is sent to the first device and the second device respectively, and the remote control request is forwarded to the second device; the P2P tunnel creation notification carries a target access address and the tunnel negotiation method, so that the first device and the second device establish a P2P tunnel based on the target access address and the tunnel negotiation method.

7. The method according to claim 6, characterized in that The method for determining the communication type of the first device based on the multiple first response messages respectively includes: Determine whether the source addresses of the first response messages are the same; if they are the same, determine that the communication type of the first device is the first type.

8. The method according to claim 6, characterized in that Determining the tunnel negotiation mode according to the communication quality between the first device and the second device includes: Based on the communication quality between the first device and the second device and the cloud server respectively, the order of sending the detection packets is determined; wherein, after receiving the P2P tunnel creation notification, the end with poor communication quality first sends a detection packet to the other end, and after receiving the P2P tunnel creation notification, the end with good communication quality sends a detection packet to the other end at a preset time interval; wherein the preset time is determined based on the maximum value of the network transmission delays corresponding to the first response message and the second response message.

9. The method according to claim 6, characterized in that The method further comprises: If the communication type of the first device or the second device is the second type; The remote control request is directly forwarded to the second device, and when the control view data returned by the second device is received, the control view data is forwarded to the first device.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images