Using augmented reality to help with device installation

Abstract

The present invention is entitled "Using Augmented Reality to Assist Device Installation". A first image of an electronic device is captured and processed to identify the electronic device. After the electronic device is identified, information containing specific wiring instructions for connecting the electronic device is downloaded to a mobile device, and the specific wiring instructions are superimposed on the image of the electronic device to guide a user of the mobile device to wire the electronic device. A second image of the at least partially wired electronic device may be captured and processed to detect whether there are any wiring errors. When a remote server finds a wiring error, the mobile device receives a message from the remote server indicating the found wiring error.

Description

Technical Field

[0001] The present disclosure generally relates to installing and wiring devices within a building management system. More specifically, the present disclosure relates to methods and systems for using augmented reality to assist an installer in installing and wiring devices within a building management system. Background Art

[0002] Building management systems (BMS) may include a large number of individually wired components that together provide functionality, such as heating, ventilation and air conditioning (HVAC) systems, lighting systems, security systems, etc. It should be appreciated that with a large number of individually wired components there are many opportunities for wiring errors or even installing the wrong component in a particular location that connects to a particular set of wires. Therefore, there is a need for an improved way to wire components within a building management system. Summary of the invention

[0003] The present disclosure as a whole relates to methods and systems that provide an improved way to wire components within a building management system. In one example, a mobile device can be used to assist in installing and connecting an electronic device that can be mounted to a structure and includes one or more wiring terminals that can be connected to one or more different wires. An exemplary method includes capturing a first image of the electronic device using a mobile device and transmitting the captured first image to a remote server so that the remote server can process the captured first image to determine the identity of the electronic device. After the identity of the electronic device is determined, information containing specific wiring instructions for connecting the electronic device is downloaded to the mobile device. The specific wiring instructions can be superimposed on a currently displayed real-time image of the electronic device as displayed on a display of the mobile device to guide a user of the mobile device to wire the electronic device. After the electronic device has been at least partially wired, a second image of the at least partially wired electronic device can be captured using the mobile device, and the second captured image of the wired electronic device can be transmitted to the remote server so that the remote server can process the second captured image to detect whether there are any wiring errors. When the remote server finds a wiring error, the mobile device receives a message from the remote server indicating the found wiring error.

[0004] In another example, a non-transitory computer-readable medium may have executable instructions stored thereon that are used to instruct a user how to wire an electronic device. The executable instructions, when executed by one or more processors of a server, may cause the server to receive a first image of an electronic device captured by a mobile device and process the received first image in order to detect the identity of the electronic device. Information containing specific wiring instructions for wiring the electronic device is downloaded to the mobile device. The mobile device may superimpose the specific wiring instructions on a real-time image of the electronic device captured by the mobile device. The server may receive a second image of the electronic device after wiring, and the server may process the received second image to detect whether there are any wiring errors. When a wiring error is found, the server sends a message to the mobile device notifying the user that a wiring error has been found.

[0005] In another example, a mobile device may be used to help an installer install and connect electronic devices in a building management system (BMS). The electronic device may include one or more wiring terminals that can be connected to one or more different wires. Wiring instructions are superimposed on a currently displayed real-time image of the electronic device on a display of the mobile device to instruct a user of the mobile device how to wire the electronic device according to BMS specifications. After the electronic device has been at least partially wired by a user, an image of the at least partially wired electronic device may be captured using the mobile device. The captured image of the wired electronic device may be processed to detect whether there are any wiring errors. When a wiring error is found, the mobile device indicates the found wiring error and instructions to the user on how to correct the wiring error on the display of the mobile device.

[0006] The above summary is provided to facilitate understanding of some innovative features unique to the present disclosure, but is not intended to be a complete description. A comprehensive understanding of the present disclosure can be obtained by taking the entire specification, claims, drawings and abstract as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure may be more fully understood by considering the following description of various examples in conjunction with the accompanying drawings, in which:

[0008] Figure 1 Can be used to guide users

[0009] A schematic block diagram of an exemplary system for mounting and connecting an electronic device including one or more terminal blocks connectable to one or more different conductors;

[0010] Figure 2 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0011] Figure 3 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0012] Figure 4 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0013] Figure 5 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0014] Figure 6 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0015] Figure 7 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0016] Figure 8 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0017] Fig. 9 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0018] Fig.10 It shows that you can use Figure 1 A flowchart of an exemplary method performed by an exemplary system of the present invention;

[0019] Fig.11 is an exemplary screen capture from a display of a mobile device for assisting in installing and connecting an electronic device;

[0020] Fig.12 is an exemplary screen capture from a display of a mobile device for facilitating installation and connection of an electronic device; and

[0021] Fig.13 is an exemplary screen capture from a display of a mobile device used to assist in installing and connecting electronic devices.

[0022] Although the present disclosure is subject to various modifications and alternative forms, the details thereof have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the present disclosure to the specific examples described. On the contrary, it is intended to cover all modifications, equivalents and alternatives that fall within the essence and scope of the present disclosure. DETAILED DESCRIPTION

[0023] The following description should be read with reference to the accompanying drawings, in which similar elements in different drawings are numbered in the same manner. The accompanying drawings are not necessarily drawn to scale, and they depict examples that are not intended to limit the scope of the present disclosure. Although examples of various elements are shown, those skilled in the art will recognize that many of the examples provided have suitable alternatives that can be utilized.

[0024] It is assumed herein that all numbers are modified by the term "about" unless the content clearly dictates otherwise. The recitation of numerical ranges by endpoints includes all numbers subsumed within that range (eg, 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0025] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally employed in its sense including "and / or" unless the content clearly dictates otherwise.

[0026] It should be noted that references in the specification to "an embodiment," "some embodiments," "other embodiments," etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is contemplated that the feature, structure, or characteristic described in conjunction with the embodiment may apply to other embodiments whether or not explicitly described, unless otherwise explicitly stated to the contrary.

[0027] Figure 1 1 is a schematic block diagram of an exemplary system 10 that can be used to assist a user in installing and wiring any of a plurality of different electronic devices within a building management system (BMS). These electronic devices may include, for example, controllers, actuators, sensors, and the like. It should be understood that a BMS may include tens, hundreds, or even thousands of electronic devices, such as those described above, and each electronic device must be installed in an appropriate location and must be properly wired. In some cases, there are calibration or setup parameters that need to be correctly set on at least some of the electronic devices in order for the BMS to function properly. It should be understood that the information required to describe and explain how to install and wire each of these electronic devices constitutes a large amount of information. Therefore, the system 10 includes a remote server 12. The system 10 also includes a mobile device 14 and an engineering tool 16.

[0028] Although the remote server 12 is shown as a single server and is shown as a cloud-based server, this is not necessary in all cases. For example, the remote server 12 can be a single computer server or a group of computer servers. The remote server 12 can be located away from the building that houses the BMS, or the remote server 12 can be located away from the engineering tool 16, but can still be located in the aforementioned building that houses the BMS. In some cases, the remote server 12 and the engineering tool 16 can be co-located. As part of designing the BMS, information describing how each of the electronic devices in the BMS is connected and wired can be entered into the engineering tool 16. The engineering tool 16 can upload this information to the remote server 12 so that the mobile device 14 can receive the information required for the user to install at least some of the electronic devices of the BMS in the building and wire them.

[0029] The mobile device 14 may be a portable device, such as a laptop, a tablet computer, and / or a virtual reality headset. In some cases, the mobile device 14 may be a smart phone, sometimes with virtual reality capabilities. In any case, the mobile device 14 has a communication function including a networking block 18 (such as Bluetooth, WiFi, or cellular), so that the mobile device 14 can communicate with the remote server 12 via the networking block 18. The mobile device 14 also has a display 22 and a camera 20, each of which will be used to guide the user to install any of a plurality of different electronic devices of the BMS and to wire them. For example, the camera 20 may be used by the user to take a picture of a specific electronic device and upload the picture to the remote server 12, so that the remote server 12 can correctly identify the specific electronic device (e.g., from many other electronic devices of the BMS). In some cases, the specific electronic device may include a sticker or other label that provides a QR code or other code that the remote server 12 can easily identify. The display 22 may be used to display an image of the specific electronic device (or a portion thereof) and calibration, connection, and / or wiring information provided by the remote server 12, and may be superimposed on the image of the specific electronic device (or a portion thereof). The superimposed information may include text identifying a particular terminal, a description or picture of the appropriately colored wire that should be connected to the particular terminal, the setting of a particular switch or parameter, etc.

[0030] The engineering tool 16 can be used by a technician to input and / or otherwise provide all the information necessary to describe the BMS. This can include all the wiring extending between tens, hundreds, or even thousands of different electronic devices within the BMS. The information can also include information describing how to connect, wire, and configure each electronic device. Additional details about the exemplary engineering tool 16 can be found in, for example, the following co-pending patent applications: a patent application entitled Managing Certificates in a Building Management System filed on the same date as the present invention with attorney docket number 1456.1380101; a patent application entitled HVAC System Configuration with Automatic Parameter Generation filed on the same date as the present invention with attorney docket number 1456.1387101; and a patent application entitled HVAC System Configuration with Automatic Schematics and Graphics Generation filed on the same date as the present invention with attorney docket number 1456.1389101. Each of these patent applications is incorporated herein by reference in its entirety.

[0031] Figure 22 is a flowchart illustrating an illustrative method 24 for using a mobile device (such as mobile device 14) to help install and connect an electronic device of a BMS, which can be mounted to a structure and includes one or more wiring terminals that can be connected to one or more different wires. In some cases, the electronic device can be a specific building controller of the BMS. The illustrative method 24 includes capturing a first image of the electronic device using a camera 2020 of the mobile device 14 and transmitting the captured first image to a remote server (such as remote server 12), so that the remote server 12 can process the captured first image to determine the identity of the electronic device, as shown in box 26. In some cases, the first image can be a still image or a video stream. Identifying the electronic device can be as simple as reading a barcode, QR code, or other identifiable code, such as an alphanumeric identifier provided on a sticker or label on the electronic device. After the remote server 12 determines the identity of the electronic device, the remote server 12 downloads information containing specific wiring instructions for connecting the specific electronic device to the mobile device 14, as shown in box 28. In some cases, the remote server 12 may be configured to use machine learning to identify common wiring errors, and the specific wiring instructions provided by the remote server 12 to the mobile device 14 may include proactive guidance to avoid the identified common wiring errors.

[0032] The mobile device 14 may superimpose specific wiring instructions onto a currently displayed real-time image (e.g., a real-time still image or a real-time video stream) of the electronic device as displayed on the display 22 of the mobile device 14 to guide the user of the mobile device to wire the electronic device, as shown at block 30. The currently displayed real-time image of the electronic device may be a captured first image of the electronic device. In some cases, the currently displayed real-time image of the electronic device may alternatively be a real-time image of the electronic device currently captured by the camera 20 of the mobile device 14, sometimes displayed in a virtual reality mode on the mobile device 14.

[0033] After the electronic device has been at least partially wired (which may include only a single wire connected), a second image of the at least partially wired electronic device may then be captured using the camera 20 of the mobile device 14, and the second image may be transmitted to the remote server 12, as shown at box 32. In some cases, the second image may be captured after the electronic device has been fully wired. The remote server 12 processes the second captured image to detect whether there are any wiring errors, as shown at box 34. When the remote server 12 finds a wiring error, the mobile device 14 receives a message from the remote server 12 indicating the wiring error found, as shown at box 36. In some cases, the mobile device 14 may also receive information from the remote server 12 indicating how to correct the wiring error. If no wiring error is found, the mobile device 14 may receive a message from the remote server 12 indicating that the electronic device is correctly wired without a wiring error. In this way, the installer understands that the specific device is correctly wired according to the BMS specification, and the installer can move to the next electronic device.

[0034] In some cases, as optionally shown at block 38, an image of another electronic device to be connected to the electronic device may be captured via one or more of the different wires. The image may be transmitted to the remote server 12 so that the remote server 12 can process the image to identify the other electronic device, as shown at block 40. Thus, the remote server 12 may verify that the other electronic device is the correct device type, as shown at block 42, and should be connected to the electronic device. In some cases, the remote server 12 may download installation instructions for the other electronic device to the mobile device 14 so that the mobile device 14 may overlay the installation instructions on the image of the other electronic device.

[0035] Figure 34 is a flow chart showing an exemplary method 44 for using a mobile device (such as mobile device 14) to assist in installing and connecting an electronic device, wherein the electronic device can be mounted to a structure and includes one or more terminals that can be connected to one or more different wires. In some cases, method 44 optionally includes retrieving information identifying the location where the electronic device is to be installed from a remote server 12, as shown at block 46. Directions that assist a user of the mobile device 14 in physically locating the installation location of the electronic device can be displayed on a display 22 of the mobile device 14. Once the user is in the appropriate location, the user can use a camera 20 of the mobile device 14 to capture a first image of the electronic device and transmit the captured first image to a remote server (such as remote server 12) so that the remote server 12 can process the captured first image to determine the identity of the electronic device, as shown at block 26. It is contemplated that the first image can be a still image or a video stream. Identifying the electronic device can be as simple as reading a barcode, QR code, or other recognizable code, such as an alphanumeric identifier provided on a sticker or label on the electronic device. After the remote server 12 determines the identity of the electronic device, the remote server 12 downloads information to the mobile device 14 containing specific wiring instructions for connecting the specific electronic device according to the BMS specifications, as shown at block 28 .

[0036] The mobile device 14 superimposes the specific wiring instructions onto the currently displayed real-time image of the electronic device as displayed on the display 22 of the mobile device 14 to guide the user of the mobile device to wire the electronic device, as shown at block 30. The currently displayed real-time image of the electronic device may be a captured first image of the electronic device. In some cases, the currently displayed real-time image of the electronic device may alternatively be a real-time image of the electronic device captured by the camera 20 of the mobile device 14 (e.g., a real-time still image or a real-time video stream).

[0037] After the electronic device has been at least partially wired (which may include only a single wire connected), a second image of the at least partially wired electronic device may then be captured using the camera 20 of the mobile device 14, and the second image may be transmitted to the remote server 12, as shown at box 32. In some cases, the second image may be captured after the electronic device has been fully wired. It is contemplated that the second image may be a still image or a video stream. The remote server 12 processes the second captured image to detect whether there are any wiring errors, as shown at box 34. When the remote server 12 finds a wiring error, the mobile device 14 receives a message from the remote server 12 indicating the wiring error found, as shown at box 36. In some cases, the mobile device 14 may also receive information from the remote server 12 indicating how to correct the wiring error. If no wiring error is found, the mobile device 14 may receive a message from the remote server 12 indicating that the electronic device is correctly wired without a wiring error. In this way, the installer understands that the particular device is correctly wired according to the BMS specification, and the installer can move to the next device.

[0038] In some cases, method 44 optionally includes mobile device 14 retrieving configuration information for configuring the electronic device from remote server 12, as shown at block 50. The configuration information may be displayed on display 22 of mobile device 14 so that a user of mobile device 14 may use the configuration information to configure the electronic device according to the BMS specifications.

[0039] Figure 4 1 is a flow chart illustrating an illustrative method 54 for using a mobile device, such as mobile device 14, to assist in replacing an old electronic device with a new electronic device. Method 54 includes capturing a first image of the old electronic device using mobile device 14 and transmitting the captured first image to remote server 12, as shown at block 56. Remote server 12 processes the captured first image to determine the identity of the old electronic device, as shown at block 58. Remote server 12 may then provide instructions for removing the old electronic device, instructions for identifying a replacement electronic device, and / or wiring instructions for wiring the replacement electronic device using existing wiring of the old electronic device to mobile device 14, as shown at block 60. In response, mobile device 14 may superimpose removal instructions on the captured image of the old electronic device for removing the old electronic device, as shown at block 62.

[0040] Next, the mobile device 14 may overlay wiring instructions for the replacement electronic device on the captured image of the replacement electronic device as displayed on the display 22 of the mobile device 14, as shown at block 64. After the replacement electronic device has been at least partially wired, a second image of the at least partially wired replacement electronic device may be captured using the mobile device 14, and the second image may be transmitted to the remote server 12, as shown at block 66. The remote server 12 processes the second captured image to ascertain whether there are any wiring errors, as shown at block 68. When the remote server 12 finds a wiring error, the mobile device 14 receives a message from the remote server 12 indicating the found wiring error, as shown at block 70.

[0041] Figure 5 is a flow chart showing an illustrative method 72 that may be performed by one or more processors of a server (such as remote server 12), which execute executable instructions that may be stored on a non-transitory computer-readable medium. When the executable instructions are executed by one or more processors of the server, the server may receive a first image of an electronic device captured by a user using a mobile device such as mobile device 14, as shown at block 74. Process the received first image to ascertain the identity of the electronic device, as shown at block 76. In some cases, processing the received first image includes identifying a unique identifier contained within the received first image. The unique identifier may, for example, include a bar code or QR code displayed on the electronic device, or an alphanumeric identifier on the electronic device.

[0042] Information containing specific wiring instructions for wiring the electronic device is downloaded to the mobile device 14, as shown at block 78. The mobile device 14 may overlay the specific wiring instructions onto a real-time image of the electronic device captured by the mobile device 14, as shown at block 80. A second image of the electronic device may be received after wiring, as shown at block 82, and the second image may be processed to ascertain whether there are any wiring errors, as shown at block 84. When a wiring error is found, a message is sent to the mobile device 14 notifying the user that a wiring error has been found, as shown at block 86. In some cases, when no wiring error is found, a message may be sent to the mobile device 14 indicating that the electronic device is correctly wired without a wiring error.

[0043] Optionally, and as shown at block 88, method 72 may also include capturing an image of another electronic device to be connected to the electronic device via one or more of the different wires. The image may be processed to ascertain the identity of the other electronic device, as shown at block 90, and thus verify that the other electronic device is the correct device type, as shown at block 92.

[0044] Figure 6is a flow chart showing an illustrative method 94 that may be performed by one or more processors of a server (such as remote server 12), which execute executable instructions that may be stored on a non-transitory computer-readable medium. When the executable instructions are executed by one or more processors of the server, the server receives a first image of an electronic device captured by a user using a mobile device such as mobile device 14, as shown at block 74. The received first image is processed to ascertain the identity of the electronic device, as shown at block 76. In some cases, processing the received first image includes identifying a unique identifier contained within the received first image. The unique identifier may, for example, include a bar code or QR code displayed on the electronic device, or an alphanumeric identifier on the electronic device.

[0045] Information containing specific wiring instructions for wiring the electronic device is downloaded to the mobile device 14, as shown at box 78. The mobile device 14 superimposes the specific wiring instructions on a real-time image of the electronic device captured by the mobile device 14, as shown at box 80. A second image of the electronic device can be received after wiring, as shown at box 82, and the second image can be processed to detect whether there are any wiring errors, as shown at box 84. When a wiring error is found, a message notifying the user that a wiring error has been found is sent to the mobile device 14, as shown at box 86. In some cases, when no wiring error is found, a message indicating that the electronic device is correctly wired without a wiring error can be sent to the mobile device 14. In some cases, as optionally shown at box 96, the wiring errors found can be tracked over time to identify common wiring errors of the user, and wherein the specific wiring instructions include active guidance to avoid the identified common wiring errors.

[0046] Figure 7 1 is a flow chart showing an exemplary method 98 for using a mobile device to assist in installing and connecting an electronic device in a building management system (BMS), the electronic device including one or more wiring terminals connectable to one or more different wires. Wiring instructions are superimposed on a currently displayed real-time image of the electronic device on a display of the mobile device to instruct a user of the mobile device how to wire the electronic device, as shown at block 100. After the electronic device has been at least partially wired by the user, an image of the at least partially wired electronic device is captured using the mobile device, as shown at block 102. The captured image of the wired electronic device is processed to ascertain whether there are any wiring errors, as shown at block 104. When a wiring error is found, the mobile device 14 indicates the found wiring error on the display 22 of the mobile device 14 and instructions to instruct the user how to correct the wiring error, as shown at block 106.

[0047] Figure 8 It shows that Figure 1Flowchart of an exemplary method 108 performed by an exemplary system 10 of . The system 10 takes a floor plan including equipment locations as input, as shown at box 110. It should be understood that the engineering tool 16 can be used to input the floor plan. The system 10 then converts the floor plan into an augmented reality (AR) map, as shown at box 112. For example, this can occur at the remote server 12. The system 10 can then use the AR map to guide the user from the user's current location to the location of a specific device (such as the first device to be installed), as shown at box 114. Guiding the user may include transmitting a map or other directions from the remote server 12 to the mobile device 14. The mobile device 14 may display a map or other directions that help the user find the location of a specific device. In some cases, for example, the mobile device 14 may provide auditory instructions (i.e., "turn left in the next corridor", etc.).

[0048] As shown at block 116, the system 10 receives installation data for the particular use, such as from a remote server 12. The installation data may include step-by-step instructions for which wires should be connected to which terminals of the electrical device. For example, the installation data may also include configuration setting information. The system 10 may gradually overlay the installation data on an image of the particular device, as shown at block 118 (e.g., in augmented reality mode). In response, the user begins installing and / or wiring and / or replacing and / or repairing the particular device, as shown by the displayed instructions.

[0049] Fig. 9 It shows that Figure 1 10. In some cases, the exemplary method 122 may be viewed as an extended example of block 120, such as Figure 8 As shown. The system 10 superimposes the wiring details on the image of the actual device, as shown at box 124. This may, for example, include superimposing an image or text on the displayed image on the mobile device 14, sometimes in augmented reality mode. The user completes the connection as instructed, and a real-time image of the connection can be sent to the system 10, as shown at box 126. This may require sending the real-time image of the connection to the remote server 12. The system 10 (and therefore the remote server 12) can use image processing and / or machine learning to extract wiring details, as shown at box 128. The system 10 can detect, for example, the color of a specific wire and then determine whether a wire of the correct color has been attached to a specific terminal. At decision box 130, it is determined whether the connection is correct. If correct, control is transferred to box 132, where the user is guided to make the next connection. Alternatively, control is transferred to box 134, where the user is warned that an error has occurred and the user is instructed how to correct the problem.

[0050] Fig.10 It shows that Figure 1136 is a flowchart of an exemplary method 136 performed by an exemplary system 10 of FIG. The system 10 overlays configuration data such as, but not limited to, input / output assignment details on an image of an actual device, as shown at box 138. For example (as shown), the assignment details may be "Connect UI1NTC20K (sensor)". This may, for example, include overlaying an image or text on a displayed image on a mobile device 14, sometimes in augmented reality mode. The user completes the connection as instructed, and a real-time image of the connection may be sent to the system 10, as shown at box 140. This may require sending a real-time image of the connection to a remote server 12. The system 10 (and therefore the remote server 12) may use image processing and / or machine learning to extract text from the image, as shown at box 142. An example may be "The connected hardware sensor is NTC20K or other". At decision box 144, the system 10 determines whether the connection is correct. If correct, control passes to box 146, where the user is guided to the next connection. Alternatively, control passes to box 148, where the user is warned that an error has occurred and is instructed how to correct the problem.

[0051] Fig.11 is a screenshot showing an illustrative screen 150 that may be displayed by the mobile device 14. The screenshot 150 includes a first electronic device 152 and a second electronic device 154 that are shown mounted on a DIN rail 156. The screenshot 150 may represent an image captured by the mobile device 14. The second electronic device 154 includes a pair of terminals 156a and 156b. Additional information 158 may be viewed as superimposed on the screen 150, providing enhanced information about the sensor model to connect to each pin / port. The additional information 158 may be viewed as indicating that the specific sensor to be connected to the terminals 156a and 156b is a supply air temperature sensor, i.e., a NTC 20K type with an operating temperature range of -50°C to 150°C.

[0052] Fig.12 1 is a screenshot showing an illustrative screen 160 that may be displayed by the mobile device 14. The screenshot 160 includes a first electronic device 162 and a second electronic device 164 that are shown mounted on the DIN rail 156. The screenshot 160 may represent an image captured by the mobile device 14. The second electronic device 164 includes an adjustable setting 166 that indicates a board number. Additional information 168 may be viewed as being superimposed on the screen 160, thereby providing enhanced information about how to set the adjustable setting 166. As shown, the additional information 168 may be viewed as indicating that the adjustable setting 166 (board number) should be set to a value of 2 in order to comply with the BMS specification.

[0053] Fig.13170 is a screen shot showing an illustrative screen 170 that can be displayed by the mobile device 14. The screen 170 includes a first electronic device 172 and a second electronic device 174 that are shown to be mounted on the DIN rail 156. The screen shot 170 may represent an image captured by the mobile device 14. The first electronic device 172 that may, for example, represent a controller includes a first pair of terminals 176a and 176b. Additional information 178 provides an image of which wire should be connected to teach terminals 176a and 176b. As shown, terminal 176a is a negative terminal, and terminal 176b is a positive terminal. In some cases, the additional information 178 may show the actual color of the wire (for example, yellow represents negative and white represents positive), but in some cases, the additional information 178 may be provided in grayscale.

[0054] The screen 170 also includes additional information 180 superimposed on the image, which informs the user that the terminal 176a is a Modbus RTU-terminal and the terminal 176b is a Modbus RTU+ terminal. The first electronic device 172 also includes a second pair of terminals 182a and 182b that can be considered as power terminals. The additional information 184 includes a graphical indication of which power cable is connected to which terminal of the terminals 182a, 182b. In some cases, the additional information 182 can show the actual color (for example, a black line for a positive power supply and a red line for a negative power supply), but in some cases, the additional information 182 can be provided in grayscale.

[0055] Thus, having described several exemplary embodiments of the present disclosure, it will be readily appreciated by those skilled in the art that other embodiments may be made and used within the scope of the appended claims. However, it should be understood that the present disclosure is in many respects merely illustrative. Details, especially those related to shape, size, arrangement of parts, and exclusion and order of steps, may be changed without exceeding the scope of the present disclosure. Of course, the scope of the present disclosure is defined in the language in which the appended claims are expressed.

Claims

1. A method for using a mobile device to assist in installing and connecting an electronic device, the electronic device being mountable to a structure and comprising one or more terminals that are connectable to one or more different wires, the method comprising: capturing a first image of the electronic device using the mobile device and transmitting the captured first image to a remote server so that the remote server can process the captured first image to determine the identity of the electronic device; After determining the identity of the electronic device, downloading information containing specific wiring instructions for connecting the electronic device to the mobile device; superimposing the specific wiring instructions onto a currently displayed real-time image of the electronic device as displayed on a display of the mobile device to guide a user of the mobile device to wire the electronic device; After the electronic device has been at least partially wired, subsequently capturing a second image of the at least partially wired electronic device using the mobile device, and transmitting the second image captured of the wired electronic device to the remote server; The remote server processes the captured second image to detect whether there are any wiring errors; and When the remote server discovers a wiring error, the mobile device receives a message from the remote server indicating the discovered wiring error; Wherein the electronic device comprises a replacement electronic device for replacing an old electronic device, the method further comprises: capturing an image of the old electronic device using the mobile device and transmitting the captured image of the old electronic device to the remote server; processing, via the remote server, the captured image to identify the old electronic device; and Removal instructions for removing the old electronic device are provided to the mobile device, wherein the removal instructions also include an identification of the replacement electronic device and wiring instructions for wiring the replacement electronic device using existing wiring of the old electronic device, wherein the mobile device superimposes the removal instructions on a captured image of the old electronic device and superimposes the wiring instructions for the replacement electronic device on a captured image of the replacement electronic device. 2 . The method of claim 1 , wherein the remote server processes the captured first image to determine the identity of the electronic device by identifying a unique identifier contained within the captured first image.

3. The method according to any one of claims 1 or 2, further comprising: capturing an image of another electronic device to be connected to the electronic device via one or more of the different wires; processing the image to ascertain the identity of the other electronic device; and Verify that the other electronic device is the correct device type.

4. The method of any one of claims 1 or 2, wherein the remote server is configured to use machine learning to identify common wiring errors of users, and the specific wiring instructions provided by the remote server include proactive guidance to avoid the identified common wiring errors.

5. The method according to any one of claims 1 or 2, further comprising: Prior to being positioned at the installation location of the electronic device, the mobile device retrieves information from the remote server identifying the location where the electronic device is to be installed; and Directions are displayed on the display of the mobile device to assist the user of the mobile device in physically locating the installation location of the electronic device.

6. The method according to any one of claims 1 or 2, further comprising: The mobile device retrieves configuration information for configuring the electronic device; as well as The configuration information is displayed on the display of the mobile device so that the user of the mobile device can configure the electronic device using the configuration information.

7. A non-transitory computer-readable medium having executable instructions thereon for instructing a user how to wire an electronic device, the executable instructions, when executed by one or more processors of a remote server, causing the remote server to: receiving a first image of the electronic device captured by a user using a mobile device; processing the received first image to ascertain the identity of the electronic device; downloading information containing specific wiring instructions for wiring the electronic device to the mobile device; The mobile device superimposes the specific wiring instructions onto a real-time image of the electronic device captured by the mobile device; receiving a second image of the electronic device after routing; processing the received second image to detect if there are any wiring errors; as well as When a wiring error is found, sending a message to the mobile device notifying the user that a wiring error has been found; The electronic device includes a replacement electronic device for replacing an old electronic device, and the executable instructions further include: capturing an image of the old electronic device using the mobile device and transmitting the captured image of the old electronic device to the remote server; processing, via the remote server, the captured image to identify the old electronic device; and Removal instructions for removing the old electronic device are provided to the mobile device, wherein the removal instructions also include an identification of the replacement electronic device and wiring instructions for wiring the replacement electronic device using existing wiring of the old electronic device, wherein the mobile device superimposes the removal instructions on a captured image of the old electronic device and superimposes the wiring instructions for the replacement electronic device on a captured image of the replacement electronic device. 8 . The non-transitory computer readable medium of claim 7 , wherein when the remote server finds no wiring error, a message indicating that the electronic device is correctly wired without a wiring error is sent to the mobile device.

9. The non-transitory computer readable medium of any one of claims 7 or 8, further comprising tracking discovered wiring errors over time to identify common wiring errors of users, and wherein the specific wiring instructions include proactive guidance to avoid the identified common wiring errors.

Images