Secure remote control system, device and method

Abstract

The present application discloses a secure remote control system, device and method. A system, device and method for implementing secure control of AV equipment connected to an audio-visual (AV) gateway are disclosed. The solution includes utilizing a user device to remotely control the AV gateway by connecting both the user device and the AV gateway to an Internet of Things (IoT) network, and facilitating communication between the user device and the AV gateway via messages using an MQTT broker or an IoT cloud service.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 150,802, filed on February 18, 2021, which is incorporated herein by reference in its entirety. Technical Field

[0003] The following relates to systems, devices, and methods for implementing secure control of audio-visual (AV) equipment connected to an AV gateway by remotely controlling the AV gateway using a user device. Background Art

[0004] Corporate building environments are equipped with audiovisual systems to enhance operational functionality. These AV systems may be installed in a number of different rooms and may be controlled by one or more central AV gateway devices. However, cost considerations result in AV gateway devices not being installed in every room where AV equipment is present, which makes it difficult to control the AV equipment in these rooms in real time. In addition, it may be desirable to restrict access to AV gateway devices to avoid contamination by the public or to enhance network security. Summary of the invention

[0005] According to one non-limiting exemplary embodiment described herein, a computing device is disclosed. The computing device includes an image capture device configured to capture an image, a display screen, a processor, and a storage device configured to store machine-readable instructions, which, when executed by the processor, cause the processor to: register the computing device with an MQTT broker; control the image capture device to capture an image of a machine-readable code displayed on a display screen of an AV gateway; open a browser application to connect to a website identified in the machine-readable code; display an AV equipment control graphical user interface (GUI) on the display screen corresponding to the website; receive a control command via the displayed AV equipment control GUI; and transmit the control command to the MQTT broker, wherein the MQTT broker is configured to publish the control command in a message received by the AV gateway for the AV gateway to control a remote AV device.

[0006] According to another non-limiting exemplary embodiment described herein, a computing device is disclosed. The computing device includes a display screen, a processor, and a storage device, the storage device being configured to store machine-readable instructions, which, when executed by the processor, cause the processor to: register the computing device as a subscriber of a topic published by an MQTT agent, display a machine-readable code on the display screen, receive a message corresponding to the subscribed topic from the MQTT, wherein the message includes a control command for controlling an AV device, and control a remote AV device according to the control command.

[0007] According to another non-limiting exemplary embodiment described herein, a computing device is disclosed. The computing device includes an image capture device, a display screen, a processor, and a storage device, the storage device being configured to store machine-readable instructions, which, when executed by the processor, cause the processor to: establish a communication channel between the computing device and a message routing component; control the image capture device to scan an image of a machine-readable code displayed on a display screen of a gateway device; extract website address information from the scanned image; open a browser application; control the browser application to connect to a website based on the website address information; display a graphical user interface (GUI) on a display screen corresponding to the website; receive a control command via the displayed GUI; and transmit the control command to the message routing component, wherein the message routing component is configured to convey the control command within a message for receipt by the gateway device.

[0008] According to another non-limiting exemplary embodiment described herein, a computing device is disclosed. The computing device includes a display screen, a processor, and a storage device, the storage device being configured to store machine-readable instructions, which, when executed by the processor, cause the processor to: establish a communication channel between the computing device and a message routing component; display a machine-readable code on the display screen; receive a message from the message routing component, the message including a control command for controlling a remote device; and control the remote device according to the control command included in the message.

[0009] Below and attached Figure 1 The following describes detailed implementations of these and other non-limiting exemplary embodiments of secure remote control systems, devices, and methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 An exemplary system block diagram of a secure remote control system according to an embodiment of the present disclosure is shown.

[0011] Figure 2 An exemplary flow chart describing a process for implementing secure remote control according to an embodiment of the present disclosure is shown.

[0012] Figure 3 A diagram showing an exemplary graphical user interface (GUI) for controlling AV equipment displayed on a display screen according to an exemplary embodiment of the present disclosure is shown.

[0013] Figure 4 A block diagram of an exemplary computing device system representing computing devices included in a secure remote control system is shown according to an embodiment of the present disclosure.

[0014] Figure 5 An exemplary system block diagram of a secure remote control system according to an alternative embodiment of the present disclosure is shown.

[0015] Figure 6 An exemplary flow chart describing a method for implementing a secure remote control process according to an alternative embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0016] As required, detailed non-limiting embodiments are disclosed herein. However, it should be understood that the disclosed embodiments are merely exemplary and may take various alternative forms. The drawings are not necessarily to scale, and some features may be exaggerated or minimized to show the details of specific components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as restrictive, but only as a representative basis for teaching those skilled in the art. For ease of illustration and ease of understanding, similar reference numerals may be used herein for similar components and features throughout the accompanying drawings. In addition, although the embodiments described herein refer to remote control of audio-visual (AV) equipment connected together via an AV network, the secure remote access solution described herein may also be applied to remote control of devices connected together via a private network that more generally uses a switch / controller device instead of an AV gateway.

[0017] Many enterprise building environments now employ AV equipment in many different locations. To help control all remote AV equipment, one or more AV gateway devices may be installed at different locations within the building. An AV gateway is a control device that a user can use to control one or more AV equipment that communicates with the AV gateway. To help provide control capabilities, the AV gateway may include a touch screen display for controlling AV equipment within the same structure or as a separate device in close proximity. However, for various reasons (e.g., hygiene, efficiency, equipment life reasons), it may not be desirable to allow all users to physically interact with the touch screen device. Therefore, in order to address this situation, a solution is currently disclosed that allows users to use their own mobile devices to transmit control commands to the AV gateway in a secure manner by utilizing the Internet of Things (IoT) network protocol and / or IoT cloud services.

[0018] Figure 1 An exemplary block diagram of a secure remote control system 100 that utilizes IoT protocols to enable a user to securely control an AV gateway 30 from the user's mobile device 10 is shown.

[0019] The AV gateway 30 includes a display screen 20, wherein the display screen 20 may be a touch screen. The display screen 20 is provided to display a control graphical user interface (GUI) 310 for controlling various control options related to the AV equipment connected to the AV gateway, such as Figure 3 An exemplary control GUI 310 is shown in FIG. Figure 3In the example display screen 300, the display screen 20 or the display screen 11 portion of the mobile device 10 may be representative of the display screen 20 or the mobile device 10. According to some embodiments, the display screen 20 may be an integral part of the same AV gateway 30, or according to other embodiments, the display screen 20 may be a separate device that communicates directly with the AV gateway 30. Various remote AV equipment 71-73 are also coupled to the AV gateway 30. The remote AV equipment may be a speaker, a display device, an AV mixer, an expander, or other equipment that may be part of the building's AV system 110. Figure 1 As shown, the building's AV system 110 is connected via an AV local area network (LAN) 2, wherein the AV LAN 2 is behind a firewall 40 to protect the building's AV system 110 from external intrusion.

[0020] The security remote control system 100 is configured to allow a user to enter a building and interact with the building's AV system 110 by utilizing the user's own mobile device 10. The mobile device 10 includes a display screen 11 and an image capture device 12 capable of taking still images and / or video images. The mobile device 10 includes a network interface to connect to the Internet 1 via a Wi-Fi network connection 3 or a cellular data network connection 4. The mobile device 10 is capable of communicating with an MQTT broker 60a through its connection to the Internet 1 and registering itself. The mobile device 10 is also configured to access a static website 50 that is also served by the AV gateway 30. The static website 50 is configured to be read-only. The static website 50 hosts a GUI that reflects the control GUI 310 displayed on the display screen 20 corresponding to the AV gateway 30. The control options may include, for example, selecting a specific AV device to be controlled based on the device name and / or building location, controlling the AV features of the selected AV device (e.g., volume control, display control, etc.), turning on / off the selected AV device, or arranging a conference room. The mobile device 10 is able to display the control GUI 310 on the display screen 11 of the mobile device 10 by opening the static website 50 in a browser running on the mobile device 10, as will be described in more detail herein. The static website 50 can be hosted on a content delivery network (CDN). The CDN provides the static website 50 through its normal operation, thereby providing additional security by blocking the binding (in-bound) of the HTTP POST protocol that is usually used to control the equipment. The static website 50 is hosted on a network server disconnected from the AV gateway 30. The CDN also uses a cached website that loads very quickly to more easily deploy applications locally to the area where the mobile device is located. It can also update the website and let the CDN provider synchronize / push updates initiated from customers.

[0021] Since the mobile device 10 is restricted from using the Internet to directly connect to the private AV LAN 2 to which the AV gateway 30 is connected, the AV gateway 30 uses the IoT cloud service to create a secure public endpoint for the mobile device 10 to open the static website 50 served by the AV gateway 30. The address of the static website 50 is encoded into a machine-readable code 21 (e.g., a barcode or QR code) displayed on the display screen 20. The mobile device 10 controls its image capture device 12 to capture an image of the machine-readable code 21, thereby providing a solution that spans the air gap between the mobile device 10 and the display screen 20. The air gap is a security measure for preventing off-site attacks from attempting to access the AV gateway 30 by intruding into the AV LAN 2. This is because the air gap requires the user to actually be present in the room where the display screen 20 is located to gain remote access to the AV gateway 30. When capturing the image of the machine-readable code 21, the mobile device 10 will decode the machine-readable code 21 to identify the address of the static website 50 encoded within the machine-readable code 21. The mobile device then executes a browser on the mobile device 10 to navigate to the address. In order to enable the mobile device 10 to decode and identify the machine-readable code 21, the mobile device 10 may have previously downloaded an application for decoding and identifying the machine-readable code 21. Once the browser navigates to the address of the static website 50, the same control GUI 310 belonging to the display screen 20 for controlling the AV gateway 30 will be displayed on the display screen 11 of the mobile device. The user can then interact with the control GUI 310 displayed on the mobile device 10 to control the AV gateway 30, which is the same as the case where the user interacts with the control GUI 310 displayed on the display screen 20 corresponding to the AV gateway 30. The process of connecting the mobile device 10 to the AV gateway 30 by utilizing the IoT protocol is provided in more detail below.

[0022] Figure 2 An exemplary flowchart 200 is shown describing a process for implementing secure remote control of an AV gateway 30. The description of flowchart 200 refers to components included in secure remote control system 100, although the process may be applicable to other similar systems. According to some embodiments, as described herein, features attributed to mobile device 10 can be enabled by downloading and executing a secure application for obtaining access to AV gateway 30 on mobile device 10.

[0023] At 201, both the mobile device 10 and the AV gateway 30 establish independent connections to the MQTT broker 60a. For the mobile device 10, the mobile device 10 registers itself as a client of the MQTT broker 60a so that it can transmit messages to the MQTT broker 60a. For the AV gateway 30, the AV gateway 30 registers itself as a subscriber of the MQTT broker 60a so that it can receive messages from the MQTT broker 60a. More specifically, the AV gateway 30 can register as a subscriber to a specific topic (e.g., an AV gateway control topic) published by the MQTT broker 60a.

[0024] At 202, the display screen 20 corresponding to the AV gateway 30 displays a machine-readable code 21. The machine-readable code 21 can be, for example, a QR code. According to some embodiments, an additional authentication step can be added before allowing the user to capture an image of the machine-readable code 21. For example, the user may need to execute a security application, where the security application requires the user to enter a password or provide some other authentication protocol (e.g., a token or authentication) before allowing the mobile device 10 to proceed to the subsequent steps in the process described in the flowchart 200. In addition or alternatively, the QR code can also be replaced with a PIN or other alphanumeric password displayed on the display screen 20. The machine-readable code 21 or its equivalent can be periodically updated to be different to enhance security.

[0025] At 203, the mobile device 10 uses its image capture device 12 to capture an image of the machine-readable code 21. For example, the user may open a camera application on the mobile device 10 so that the image capture device 12 can be used to capture and / or scan an image of the machine-readable code 21. The mobile device 10 may then decode and read the machine-readable code 21 to obtain the URL address of the static website 50.

[0026] The image of the machine-readable code 21 is scanned to obtain information, and the security application can further prevent / limit the image of the machine-readable code 21 from being stored on the mobile device 10 or the image capture device 12, which enhances security by requiring physical access to the display screen 20. According to some embodiments, additional authentication steps can be added before decoding the machine-readable code 21. For example, the user may be required to enter a password or provide some other authentication protocol (e.g., a token or authentication) before the machine-readable code 21 is decoded and / or the mobile device 10 is allowed to proceed to subsequent steps in the process described by the flowchart 200.

[0027] At 204, the mobile device 10 opens a browser that navigates to the URL address of the static website 50 identified from the machine-readable code 21. Alternatively, the security application can coordinate navigation to the URL address of the static website 50 based on entry of the correct PIN (or other human-readable code) displayed on the display screen 20.

[0028] At 205 , the browser displays the control GUI 310 hosted on the static website 50 , wherein the control GUI 310 is the same as the control GUI 310 provided on the display screen 20 for controlling the AV gateway 30 .

[0029] At 206 , the mobile device 10 receives a user command input through the control GUI 310 . The command input is used to control the AV gateway 30 .

[0030] At 207 , the mobile device 10 transmits a message to the MQTT broker 60 a , wherein the message includes the user command input.

[0031] At 208, the MQTT agent 60a receives the message and publishes (i.e., broadcasts) the user command input in its own topic message, where the subscribers of the MQTT agent 60a are set to receive topic messages. Therefore, the subscribers with the AV gateway control topic of the MQTT agent 60a will receive the new topic message including the user command input.

[0032] At 209 , as a subscriber to the AV gateway control topic with the MQTT broker 60 a , the AV gateway 30 receives the topic message, wherein the topic message includes the user command input.

[0033] At 210 , the AV gateway 30 reads the user command input from the subject message and executes the user command input to control the corresponding remote AV equipment 71 - 73 in communication with the AV gateway 30 .

[0034] The flowchart 200 is provided for exemplary purposes, as the secure remote control system 100 may implement other processes including fewer or additional steps to accomplish secure remote control of the AV gateway 30 using the IoT protocol.

[0035] Figure 5 An exemplary block diagram of a secure remote control system 500 is shown that utilizes an IoT cloud service 60b to enable a user to securely control an AV gateway 30 from the user's mobile device 10. The secure remote control system 500 is similar to the secure remote control system 100 in that it shares many of the same components; however, in the secure remote control system 500, the MQTT broker 60a is replaced by an IoT cloud service 60b. Thus, the message routing component has changed, while other system components remain the same.

[0036] The IoT cloud service 60b is a network service provider that provides network infrastructure to host websites and cloud services. According to some embodiments, the IoT cloud service 60b can be the same network service provider that hosts the static website 50, or a separate network service provider. The IoT cloud service 60b hosts a private website that will be used as a cloud proxy that is configured to shuttle control commands between the static website 50 and the AV gateway 30, such as Figure 6 The illustrated flowchart 600 is described in greater detail.

[0037] In order to create a connection between the IoT cloud service 60b and the AV gateway 30, a WebSocket is established between the AV gateway 30 and the IoT cloud service 60b. For ease of implementation, another WebSocket can be established between the mobile device 10 and the IoT cloud service 60b. Then, the secure remote control system 500 is configured to connect the WebSocket from the mobile device 10 to the WebSocket from the AV gateway 30, and manage these connections using a security token (e.g., a private key). For example, in the secure remote control system 500, the machine-readable code 21 may be generated to include a digital signature in addition to the URL of the static website 50, wherein the digital signature is provided to limit access to the AV gateway 30. Therefore, the AV gateway 30 or the IoT cloud service 60b, or both the AV gateway 30 and the IoT cloud service, generates a private key to encode the digital signature and the URL address included in the machine-readable code 21. According to some embodiments, the digital signature can be a hash function applied to data (e.g., including at least the URL address) used by a web browser running on the mobile device 10. The IoT cloud service 60b receives data through the WebSocket established between the mobile device 10 and the IoT cloud service 60b. The data may also include routing information for reaching the correct AV gateway 30. Comparing the digital signature in the IoT cloud service 60b cloud service (e.g., a comparison performed by a router device included in the IoT cloud service 60b) ensures that the routing data for locating the correct AV gateway 30 has not been tampered with. The WebSocket connection from the browser to the IoT cloud service is stateful, so the hash can be saved with the connection information to prevent tampering. In addition, a digital signature comparison (i.e., hash function verification) may be further performed on the AV gateway 30 to provide enhanced security.

[0038] This security mechanism is provided to prevent man-in-the-middle or spoofing attacks. Once the WebSocket connection is established and the security token is successfully exchanged, the cloud proxy can be used to shuttle control commands between the mobile device 10 (via the static website 50) and the AV gateway 30, as described in accordance with Figure 6The illustrated flowchart 600 is described in greater detail.

[0039] Figure 6 An exemplary flowchart 600 is shown describing a process for implementing secure remote control of an AV gateway 30. The description of flowchart 600 refers to components included in secure remote control system 500, although the process may be applicable to other similar systems. According to some embodiments, features attributed to mobile device 10 can be enabled by downloading and executing a secure application for obtaining access to AV gateway 30 on mobile device 10, as described herein.

[0040] At 601, both the mobile device 10 and the AV gateway 30 establish independent connections to the IoT cloud service 60b. For the mobile device 10, a first WebSocket connection is established to enable the mobile device 10 to transmit messages to the IoT cloud service 60b, wherein the IoT cloud service 60b includes a cloud agent (e.g., a cloud network service agent). For the AV gateway 30, a second WebSocket connection is established to enable the AV gateway 30 to receive messages from the IoT cloud service 60b. More specifically, the AV gateway 30 can receive control command messages from the cloud agent included in the IoT cloud service 60b via the WebSocket connection established between the AV gateway and the IoT cloud service 60b.

[0041] At 602, the display screen 20 corresponding to the AV gateway 30 displays a machine-readable code 21. The machine-readable code 21 may be, for example, a QR code generated by the AV gateway 30. The QR code includes a digital signature encoded by a private key generated by the AV gateway 30 or received from the IoT cloud service 60b.

[0042] According to some embodiments, additional authentication steps may be added before allowing the user to capture an image of the machine-readable code 21. For example, the user may need to execute a security application that requires the user to enter a password or provide some other authentication protocol (e.g., a token or certification) before allowing the mobile device 10 to proceed to subsequent steps in the process described in the flowchart 200. Additionally or alternatively, the QR code may also be replaced with a PIN or other alphanumeric password displayed on the display screen 20. The machine-readable code 21 or its equivalent may be periodically updated to be different to enhance security.

[0043] At 603, the mobile device 10 uses its image capture device 12 to capture an image of the machine-readable code 21. For example, the user may open a camera application on the mobile device 10 so that the image capture device 12 can be used to capture and / or scan an image of the machine-readable code 21. The mobile device 10 may then decode and read the machine-readable code 21 to obtain the URL address of the static website 50.

[0044] The image of the machine-readable code 21 is scanned to obtain information, and the security application can further prevent / limit the image of the machine-readable code 21 from being stored on the mobile device 10 or image capture device 12, which enhances security by requiring physical access to the display screen 20. According to some embodiments, additional authentication steps can be added before decoding the machine-readable code 21. For example, the user may be required to enter a password or provide some other authentication protocol (e.g., a token or authentication) before the machine-readable code 21 is decoded and / or the mobile device 10 is allowed to proceed to subsequent steps in the process described by the flowchart 200.

[0045] At 604, the mobile device 10 opens a browser that navigates to the URL address of the static website 50 identified from the machine-readable code 21. Alternatively, the security application can coordinate navigation to the URL address of the static website 50 based on entry of the correct PIN (or other human-readable code) displayed on the display screen 20.

[0046] At 605 , the browser running on the mobile device 10 displays the control GUI 310 hosted on the static website 50 , wherein the control GUI 310 is the same as the control GUI 310 provided on the display screen 20 for controlling the AV gateway 30 .

[0047] At 606 , the mobile device 10 receives a user command input through the control GUI 310 . The command input is used to control the AV gateway 30 .

[0048] At 607 , the mobile device 10 transmits a message to the IoT cloud service 60 b (eg, a cloud proxy), wherein the message includes the user command input.

[0049] At 608 , the IoT cloud service 60 b receives the message and pushes the message to the AV gateway 30 via the WebSocket established between the IoT cloud service 60 b and the AV gateway 30 .

[0050] At 609, the AV gateway 30 receives a message from the IoT cloud service 60b, wherein the message includes a user command input. According to some embodiments, the message may further include a digital signature encoded with a private key stored in the AV gateway 30 and included in the machine-readable code 21. In these embodiments, the AV gateway 30 includes a corresponding security token (digital signature) that is used to calculate a unique hash for comparison, thereby authenticating the message.

[0051] At 610 , the AV gateway 30 reads the user command input from the message and executes the user command input to control the corresponding remote AV equipment 71 - 73 in communication with the AV gateway 30 .

[0052] The flowchart 600 is provided for exemplary purposes, as the secure remote control system 500 may implement other processes including fewer or additional steps to accomplish secure remote control of the AV gateway 30 using the IoT cloud service 60 b .

[0053] Figure 4 400. For example, the computing device system 400 may represent an exemplary computer architecture for a computing device system 400. Figure 1 The components included in one or more of the mobile device 10, AV gateway 30, or MQTT broker 60 shown in the secure remote control system 100. Although not specifically shown, the computing device system 400 may additionally include software, hardware, and / or circuitry for implementing the attributed features described herein.

[0054] The computing device system 400 includes a processor 410, a main memory 420, a static memory 430, an output device 450 (e.g., a display or speaker), an input device 460, and a storage device 470 that communicate via a bus 401. The bus 401 may represent one or more buses, such as USB, PCI, ISA (Industry Standard Architecture), X-Bus, EISA (Extended Industry Standard Architecture), or any other suitable bus and / or bridge (also called a bus controller).

[0055] The processor 410 represents a central processing unit of any type of architecture, such as CISC (Complex Instruction Set Computing), RISC (Reduced Instruction Set Computing), VLIW (Very Long Instruction Word), or a hybrid architecture, although any suitable processor may be used. The processor 410 executes instructions 421, 431, 472 stored on one or more of the main memory 420, static memory 430, or storage device 470, respectively. The processor 410 may also include a portion of the computing device system 400 that controls the operation of the entire computing device system 400. The processor 410 may also represent a controller that organizes data and program storage in memory and transfers data and other information between various parts of the computing device system 400.

[0056] Processor 410 is configured to receive input data and / or user commands through input device 460 or from network 402 through network interface 440. Input device 460 may be a keyboard, mouse or other pointing device, trackball, scroll, button, touch pad, touch screen, keypad, microphone, voice recognition device, video recognition device, accelerometer, gyroscope, global positioning system (GPS) transceiver, or any other suitable mechanism for a user to input data to computing device system 400 and control the operation of computing device system 400. Figure 4 The input devices 460 shown may represent any number and type of input devices.

[0057] The processor 410 may also communicate with other computer systems via the network 402 to receive control commands or instructions 421, 431, 472, wherein the processor 410 may control the storage of such control commands or instructions 421, 431, 472 in any one or more of the main memory 420 (e.g., random access memory (RAM)), the static memory 430 (e.g., read-only memory (ROM)), or the storage device 470. The processor 410 may then read and execute the instructions 421, 431, 472 from any one or more of the main memory 420, the static memory 430, or the storage device 470. The instructions 421, 431, 472 may also be stored in any one or more of the main memory 420, the static memory 430, or the storage device 470 through other sources. The instructions 421, 431, 472 may correspond to, for example, instructions for controlling the operation of the processor 410 in the main memory 420, the static memory 430, or the storage device 470. Figure 1 Instructions for AV equipment 71-73 included in the secure remote control system 100 are shown.

[0058] Although the computing device system 400 is Figure 4 Although shown as a single processor 410 and a single bus 401, the disclosed embodiments are also applicable to a computing device system that may have multiple processors and a computing device system that may have multiple buses, some or all of which perform different functions in different ways.

[0059] Storage device 470 represents one or more mechanisms for storing data. For example, storage device 470 may include computer-readable media 471, such as read-only memory (ROM), RAM, non-volatile storage media, optical storage media, flash memory devices, and / or other machine-readable media. In other embodiments, any suitable type of storage device may be used. Although only one storage device 470 is shown, multiple storage devices and multiple types of storage devices may exist. In addition, although computing device system 400 is drawn to include storage device 470, storage device 470 may be distributed across other computer systems (such as servers communicating with computing device system 400) that communicate with computing device system 400. For example, when computing device system 400 represents mobile device 10, storage device 470 may be distributed across multiple devices to include a cloud storage platform.

[0060] As described herein, the storage device 470 may include a controller (not shown) and a computer readable medium 471 storing instructions 472 that can be executed by the processor 410 to perform control of the remote AV equipment 71-73. In another embodiment, some or all of the functions are performed via hardware rather than a processor-based system. In some embodiments, the controller included is a web application browser, but in other embodiments, the controller may be a database system, a file system, an email system, a media manager, an image manager, or may include any other functionality capable of accessing data items.

[0061] Output device 450 is configured to present information to the user.For example, output device 450 can be a display, such as a liquid crystal display (LCD), a flat panel display based on gas or plasma, or a traditional cathode ray tube (CRT) display or other known types of displays, which can also include touch screen functions, or can not include touch screen functions.Therefore, as described herein, output device 450 can be used to display a graphical user interface (GUI), such as a GUI for enabling a user to control AV equipment.In other embodiments, output device 450 can be a speaker configured to output audio information to the user.In still other embodiments, any combination of output devices can be represented by output device 450.

[0062] The computing device system 400 also includes a network interface 440 that allows communication with other computers via a network 402, where the network 402 can be any suitable network and can support any suitable protocol suitable for communication to / from the computing device system 400. In an embodiment, the network 402 can support wireless network communications. In another embodiment, the network 402 can support hard-wired communications, such as telephone lines or cables. In another embodiment, the network 402 can support Ethernet IEEE (Institute of Electrical and Electronics Engineers) 802.3x specifications. In another embodiment, the network 402 can be the Internet (e.g., Figure 1 In another embodiment, the network 402 may be a LAN (e.g., Figure 1AV LAN 2) or a wide area network (WAN) as shown. In another embodiment, network 402 may be a hotspot service provider network. In another embodiment, network 402 may be an intranet. In another embodiment, network 402 may be a GPRS (General Packet Radio Service) network. In another embodiment, network 402 may be any suitable cellular data network or cellular-based wireless network technology. In another embodiment, network 402 may be an IEEE 802.11 wireless network. In another embodiment, network 402 may represent an Internet of Things (IoT) network. In still another embodiment, network 402 may be any suitable network or combination of networks. Although Figure 4 One network 402 is shown in FIG. 4 , but network 402 may represent any number of networks (of the same or different types) that may be utilized.

[0063] The network interface 440 provides the computing device system 400 with a connection to the network 402 through any compatible communication protocol. The network interface 440 sends and / or receives data from the network 402 via a wireless or wired transceiver 441. The transceiver 441 may be cellular frequency, radio frequency (RF), infrared (IR), Bluetooth, or any of a number of known wireless or wired transmission systems capable of communicating with the network 402 or other computer devices having some or all of the features of the computing device system 400. Figure 4 The network interface 440 shown may represent a single network interface card configured to communicate with one or more different data sources. In addition, the network interface 440 may represent AV-related communication ports, such as High Definition Multimedia Interface (HDMI), DisplayPort or Mini DisplayPort (MDP), and data communication ports, such as Ethernet, Universal Serial Bus (USB), Power over Ethernet (POE) or Single Pair Ethernet (SPE).

[0064] The computing device system 400 can be implemented using any suitable hardware and / or software (such as a personal computer or other electronic computing device). In addition, the computing device system 400 can also be a smart phone, a portable computer, a laptop computer, a tablet computer or a notebook computer, a PDA, a home appliance, an IP phone, a server computer device, an AV gateway, an MQTT agent, a cloud service infrastructure device, or a mainframe computer.

[0065] As is apparent from the foregoing, various non-limiting embodiments of a secure remote control system have been described. Although various embodiments have been illustrated and described herein, they are exemplary and not intended to be limiting. Instead, the words used in the specification are words of description rather than limitation, and it should be understood that various changes may be made to the embodiments without departing from the spirit and scope of the following claims.

Claims

1. A computing device, comprising: Image capture device; Display screen; processor; as well as a storage device configured to store machine-readable instructions that, when executed by the processor, cause the processor to: establishing a communication channel between the computing device and the message routing component by registering the computing device as a client of the message routing component; controlling the image capture device to capture an image of a machine-readable code displayed on a display screen of a gateway device, wherein the gateway device is configured to subscribe to the message routing component to receive messages published by the message routing component; displaying a graphical user interface GUI on a display screen corresponding to a device GUI available on the gateway device; receiving a control command via an input corresponding to the displayed GUI; as well as The control command is transmitted to the message routing component, wherein the message routing component is configured to communicate the control command by publishing the control command within a message for receipt by the gateway device as a subscribed user of the message routing component.

2. The computing device of claim 1, wherein: The storage device is further configured to store machine-readable instructions that, when executed by the processor, cause the processor to: extracting website address information from the scanned image; Open the browser app; controlling the browser application to connect to a website based on the website address information; and The GUI is displayed on the display screen based on the GUI information received from the website.

3. The computing device of claim 2, wherein: The website is hosted on a content delivery network CDN.

4. The computing device of claim 1, wherein: The message routing component is a message queue telemetry transmission MQTT agent; wherein the storage device is configured to store machine-readable instructions which, when executed by the processor, cause the processor to: establishing the communication channel between the computing device and the MQTT proxy by registering the computing device as a client of the MQTT proxy; and The gateway device is configured to subscribe to the MQTT agent, so that the gateway device receives the message from the MQTT agent when MQTT publishes the message.

5. The computing device of claim 1, wherein: The message routing component is a cloud service website proxy; The storage device is further configured to store machine-readable instructions which, when executed by the processor, cause the processor to: Establishing the communication channel between the computing device and the cloud service website proxy by establishing a Web Socket between the computing device and the cloud service website proxy; and The cloud service website agent is configured to push the message to the gateway device.

6. The computing device of claim 5, wherein: The machine-readable code includes a digital signature encoded using a private key generated by one of the gateway device or the cloud service website agent.

7. The computing device of claim 5, wherein: The machine-readable code is periodically updated to a different form.

8. The computing device of claim 1, wherein: The computing device is one of a smartphone, a laptop, or a tablet computing device.

9. The computing device of claim 1, wherein: The gateway device is an AV gateway for controlling a remote audio-visual AV device, the AV gateway and the remote AV device being included as part of a private AV network behind a firewall.

10. A gateway computing device, comprising: Display screen; processor; as well as a storage device configured to store machine-readable instructions that, when executed by the processor, cause the processor to: establishing a communication channel between the gateway computing device and the message routing component by registering the gateway computing device as a client of the message routing component; subscribing to the message routing component to retrieve messages published by the message routing component; displaying on the display screen a machine-readable code that can be scanned by a computing device including an image capture device, wherein the machine-readable code provides website address information, the website hosting a graphical user interface (GUI) for controlling the gateway computing device; receiving the message from the message routing component via a subscription, the message comprising a control command for controlling a remote device; and The remote equipment is controlled according to the control command included in the message.

11. The gateway computing device of claim 10, wherein: The message routing component is a message queue telemetry transmission MQTT agent; wherein the storage device is configured to store machine-readable instructions which, when executed by the processor, cause the processor to: establishing the communication channel between the gateway computing device and the MQTT proxy by registering the computing device as a subscriber to a topic published by the MQTT proxy; and and The message received from the MQTT agent corresponds to the subscribed topic.

12. The gateway computing device of claim 10, wherein: The message routing component is a cloud service website proxy; wherein the storage device is configured to store machine-readable instructions which, when executed by the processor, cause the processor to: The communication channel is established between the gateway computing device and the cloud service website agent by establishing a WebSocket between the gateway computing device and the cloud service website agent.

13. The gateway computing device of claim 12, wherein: The gateway computing device is configured to control the remote equipment, and the gateway computing device and the remote equipment are configured to be included as part of a private network; and Wherein the machine-readable code comprises a digital signature encoded using a private key generated by one of the gateway computing device or the cloud service website agent.

14. The gateway computing device of claim 12, wherein: The gateway computing device is configured to control the remote equipment, and wherein the machine-readable code includes a digital signature encoded using a private key generated by one of the gateway computing device or the cloud service website agent.

15. The gateway computing device of claim 10, wherein: The gateway computing device is configured to control the remote equipment.

16. The gateway computing device of claim 10, wherein: The gateway computing device and the remote equipment are included as part of a private network behind a firewall.

17. The gateway computing device of claim 10, wherein: The gateway computing device is an audio-visual (AV) gateway, and wherein the AV gateway and the remote equipment are included as part of a private AV network behind a firewall.

18. The gateway computing device of claim 10, wherein: The display screen is a touch screen.

19. The computing device of claim 10, wherein: The machine-readable code is periodically updated to a different form.

Images