Information display system, information display method, and information display program

Abstract

To provide easy-to-understand information while suppressing constraint on an operation device.SOLUTION: An information display device includes: an operation device 2 for operating a device 1; a smart device 10; a storage unit 34 storing a screen description information database 41 obtained by associating a state screen with screen description information 38 related to the state screen, and a specific information database 42 obtained by associating a specific part 8 of the state screen with the state screen; a screen extraction unit 29 which acquires a screen image 37 obtained by an imaging device 21 imaging the state screen displayed on a display unit of the operation device 2 and extracts the specific part 8 from the screen image 37; a screen specifying unit 30 which specifies a state screen corresponding to the extracted specific part 8 from the specific information database 42; and a screen description information control unit 32 which extracts screen description information 38 relating to the specified state screen from the screen description information database 41, to be displayed on a display screen 22 of the smart device 10 together with the state screen.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information display system, an information display method, and an information display program for an operating device that remotely operates a device. [Background technology]

[0002] An operating device (remote control device) remotely controls a device (controlled device). As disclosed in Patent Document 1, there is an operating device that has a display unit and systematically organizes and displays an explanation of the operating method of the operating device and the functions of the device on the display unit to easily inform the user of the operating method.

[0003] In addition, as disclosed in Patent Document 2, there are smart devices (mobile phones) that can display operating instructions for a device (navigation device) (status information of the operation buttons and a help center URL) by photographing a two-dimensional barcode displayed on the device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-253946 A [Patent Document 2] JP 2006-125871 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in order to display the instructions on the operation device, the display unit of the operation device needs to be large and have a certain level of performance. Also, even if the instructions are organized, it is not easy to intuitively understand the operation method and the displayed contents when the instructions are displayed in a systematic manner.

[0006] Furthermore, when displaying identification information such as a two-dimensional barcode on an operating device, it is necessary to secure a display area, which makes it difficult to freely and efficiently design the screen of the operating device.

[0007] An object of the present invention is to provide easily understandable information while minimizing restrictions on an operating device. [Means for solving the problem]

[0008] In order to achieve the above object, a characteristic configuration of an information display system according to one embodiment of the present invention includes an apparatus, an operation device having a display unit that displays a status screen that changes depending on a status of the apparatus and that operates the apparatus, a smart device having an imaging device and a display screen, a storage unit that stores a screen description information database in which screen description information related to the status screen is associated with the status screen and a specific information database in which a distinctive portion of the status screen is associated with the status screen, a screen extraction unit that acquires a screen image of the status screen displayed on the display unit of the operation device captured by the imaging device and extracts the distinctive portion from the screen image, a screen identification unit that identifies the status screen corresponding to the extracted distinctive portion from the specific information database, and a screen description information database that extracts the screen description information related to the identified status screen from the screen description information database. Identified The present invention is characterized in that it includes a screen description information control unit that displays the status screen on the display screen of the smart device together with the status screen.

[0009] Furthermore, a characteristic configuration of an information display method according to an embodiment of the present invention includes a step of capturing a status screen of the device displayed on an operation device for operating the device as a screen image with a smart device, a step of extracting a characteristic portion from the screen image, a step of identifying the status screen corresponding to the extracted characteristic portion from a specific information database in which the status screen and the characteristic portion of the status screen are associated, a step of extracting the screen description information related to the identified status screen from a screen description information database in which the status screen and screen description information related to the status screen are associated, and a step of displaying the extracted screen description information. Identified and displaying the status screen on a display screen of the smart device together with the status screen.

[0010] Furthermore, a characteristic configuration of an information display program according to an embodiment of the present invention includes a function of acquiring a screen image of a status screen of the device displayed on an operation device for operating the device, the screen image being captured by a smart device; a function of extracting a characteristic portion from the screen image; a function of identifying the status screen corresponding to the extracted characteristic portion from a specific information database in which the status screen and the characteristic portion of the status screen are associated; a function of extracting the screen description information related to the identified status screen from a screen description information database in which the status screen and screen description information related to the status screen are associated; and a function of displaying the extracted screen description information. Identified The present invention has a feature of causing a computer to execute a function of displaying the status screen on a display screen of the smart device together with the status screen.

[0011] With this configuration, the specific information database can easily identify the status screen (captured screen image) displayed on the operation device, and the screen description information database can easily extract screen description information corresponding to the identified status screen. Furthermore, by displaying the screen description information on the smart device rather than on the operation device, the screen description information can be displayed in an easy and easy-to-understand manner while preventing the information display area (display unit) of the operation device from being restricted. As a result, the user can Identified It becomes possible to easily understand the meaning of the status screen and the operation method, and to operate appropriately.

[0012] In addition, the status screen may change to any of a plurality of status display screens, and may include a fixed portion that is fixed in any of the status display screens, a unique portion that is fixed in the same status display screen, and a variable portion that changes depending on the status of the device.

[0013] With this configuration, it becomes clear which part of the screen image is used to identify the status screen displayed on the operating device.

[0014] Furthermore, at least one of the memory unit, the screen extraction unit, the screen identification unit, and the screen description information control unit may be mounted on the smart device.

[0015] With this configuration, the system can be easily constructed.

[0016] The system may further include a server capable of communicating with the operating device and the smart device, and at least one of the memory unit, the screen extraction unit, the screen identification unit, and the screen description information control unit may be mounted on either the smart device or the server.

[0017] Such a configuration makes it easy to optimize the configuration of the smart device or the operating device.

[0018] The screen explanatory information may include at least one of an explanation of the content of the status screen and an explanation of a method of operating the operating device.

[0019] With this configuration, the user can easily and accurately operate the operating device.

[0020] The smart device may also be a mobile terminal.

[0021] With this configuration, a remote control configuration can be easily constructed.

[0022] The smart device may also include two separable devices, one of the devices having the imaging device and the other of the devices having the display screen.

[0023] With this configuration, the capture of the screen image and the display of the screen explanatory information can be performed with a higher degree of freedom, and the screen explanatory information can be displayed in a more easily understandable configuration. [Brief description of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram illustrating a relationship between devices and a remote control for operating the devices. [Diagram 2] FIG. 13 is a diagram illustrating an example of a status screen. [Diagram 3] FIG. 13 is a diagram illustrating fixed parts, unique parts, and variable parts in a status screen. [Figure 4] FIG. 1 is a diagram illustrating an example of a configuration of an information display system. [Diagram 5] FIG. 13 is a diagram showing a process flow for displaying screen explanation information related to a status screen. [Figure 6] FIG. 11 is a diagram illustrating an example of screen description information. [Figure 7] FIG. 11 is a diagram illustrating an example of screen description information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] [Remote control] First, the remote control 2 for operating the device 1 will be described with reference to FIG.

[0026] As shown in FIG. 1, a device 1 such as a water heater is operated by a remote control (corresponding to an "operation device") 2 connected by wire or wirelessly. The remote control 2 is associated with the device 1 to be operated, and can operate the corresponding device 1. The remote control 2 includes a display unit 3 and various switches 4. The display unit 3 is a touch panel, and displays a status screen 6 (see FIG. 2) that changes depending on the state of the device 1. The remote control 2 is operated by operating the buttons and switches 4 displayed on the status screen 6 while referring to the status screen 6 displayed on the display unit 3.

[0027] The remote control 2 is connected to the server 5 via a communication line such as the Internet in a state in which wired or wireless data communication is possible. The server 5 can communicate with multiple remote controls 2, and can collect information from each remote control 2 and / or the device 1 to which each remote control 2 is connected, and can provide the information to each remote control 2 and / or the device 1.

[0028] Next, the status screen 6 displayed on the display unit 3 of the remote control 2 will be described using Fig. 2 and Fig. 3 while referring to Fig. 1. Here, a hot water supply system capable of hot water supply and heating will be described as an example of the device 1. This hot water supply system is capable of using electricity generated by a generator, selling the generated electricity to a power company, and purchasing electricity from the power company in the event of a power shortage. The hot water supply system may also be equipped with a storage battery capable of charging and discharging electricity.

[0029] 2, the status screen 6 displayed on the display unit 3 of the remote control 2 changes to a number of types of status display screens. For example, the status screen 6 changes to a status display screen 6A (FIG. 2(a)) indicating that power is being generated and power is being purchased, a status display screen 6B (FIG. 2(b)) indicating that power is being generated and power is being sold, a status display screen 6C (FIG. 2(c)) indicating that power generation has stopped and power is being purchased, a status display screen 6D (FIG. 2(d)) indicating that a malfunction has occurred, and the like.

[0030] The status display screen 6A (FIG. 2(a)), which indicates that electricity is being generated and purchased, indicates that the generated electricity and the purchased electricity are being used, and also shows the amount of electricity being generated and the amount of electricity being purchased.

[0031] The status display screen 6B (FIG. 2(b)) indicating that electricity is being generated and sold indicates that the generated electricity is being used and sold, and also displays the amount of power being generated and the amount of power being sold.

[0032] The status display screen 6C (FIG. 2(c)), which indicates that power generation is stopped and electricity is being purchased, indicates that power generation is stopped and the purchased electricity is being used, and also indicates the amount of power being purchased.

[0033] The status display screen 6D (FIG. 2(d)) indicating that the power generation system of the hot water supply system is in a breakdown state indicates that purchased electricity is being used and also indicates the amount of purchased electricity.

[0034] Each of these status display screens 6A-6D is comprised of a fixed portion 7, a unique portion 8, and a variable portion 9. The fixed portion 7 is a portion of the status screen 6 that is displayed in the same way on all status display screens 6A-6D, and is a portion that does not change even when the status screen 6 changes. The unique portion 8 is a fixed portion that is different on each of the status display screens 6A-6D, and does not change within the same status display screen 6A-6D (each of the status display screens 6A-6D), and is, for example, a portion that is displayed as an animation. The variable portion 9 is a portion that changes depending on the status of the hot water supply system, and is, for example, a portion that shows power.

[0035] A specific configuration of the status display screen will be described with reference to Fig. 3, taking the status display screen 6A shown in Fig. 2(a) as an example. The fixed portion 7 is the portion shown in Fig. 3(a) in the status display screen 6A (status screen 6) shown in Fig. 2(a), and is the same in the other status display screens 6B to 6D. For example, the fixed portion 7 shown in Fig. 3(a) is a pattern 12 that represents a facility such as a house that uses electricity, a pattern 13 that represents a power generation and storage system, and a pattern 14 that represents a utility pole of a power company.

[0036] The unique portion 8 is the portion shown in Fig. 3(b) in the status display screen 6A (status screen 6) shown in Fig. 2(a), and is different for each of the status display screens 6A to 6D, but is a fixed portion in each of the status display screens 6A to 6D. For example, the unique portion 8 shown in Fig. 3(b) is the characters "Generating electricity" 15, the arrow 16 indicating the direction of electricity supply, and the unit of power 17.

[0037] The changing portion 9 is the portion shown in Fig. 3(c) in the status display screen 6A (status screen 6) shown in Fig. 2(a), and is the portion that changes depending on the status of the hot water supply system. For example, the changing portion 9 shown in Fig. 3(c) is a numerical value 11 that corresponds to the total power generated, purchased, and used in the home.

[0038] As described above, server 5 collects information from the hot water supply system, which is device 1, via remote control 2. The information includes various types of information indicating the status of the hot water supply system as display information 36 (see FIG. 4) for displaying status screen 6.

[0039] The remote control 2 receives the display information 36 via the server 5 or directly from the device 1, and first determines which of the status display screens 6A to 6D the current operating state of the device 1 corresponds to as the status screen 6. Then, the remote control 2 displays the fixed portion 7 on the display unit 3, and also displays the unique portion 8 on the display unit 3, which corresponds to one of the status display screens 6A to 6D determined according to the display information 36. At the same time, the remote control 2 extracts the display content of the variable portion 9 from the display information 36, and displays the extracted display content on the display unit 3. In this way, the remote control 2 combines the fixed portion 7, unique portion 8, and variable portion 9 corresponding to the received display information 36, and displays them on the display unit 3.

[0040] [Information display] As described above, with the status screen 6 displayed on the display unit 3 of the remote control 2, the user of the hot water supply system operates the hot water supply system via the remote control 2 while checking the status screen 6. At this time, the user needs to understand the meaning of the status screen 6 and how to operate it. For this reason, it is required to easily and clearly explain to the user the meanings of the patterns 12 to 14, characters 15, arrows 16, units 17, numbers 11, etc. displayed on the status screen 6 (status display screens 6A to 6D), and how to operate the buttons. However, the area of ​​the display unit 3 of the remote control 2 is limited, and the display of various explanations is restricted by the display performance of the remote control 2. In order to properly display various explanations on the display unit 3 of the remote control 2, the display performance of the remote control 2 needs to be improved, and it may be difficult to properly display them.

[0041] Therefore, in this embodiment, the explanation (information display) of the status screen 6 is displayed on the display screen 22 of a separately provided mobile terminal (corresponding to a "smart device") 10.

[0042] Hereinafter, an information display system, an information display method, and an information display program for displaying an explanation of the status screen 6 on the mobile terminal 10 will be described with reference to FIGS. 1 to 3 and with reference to FIGS.

[0043] First, the information display system according to this embodiment includes a device 1, a remote control 2 for operating the device 1, a mobile terminal 10, and a server 5, as shown in FIG.

[0044] Although the device 1 may be any device having various functions, the following example will be described as a hot water supply system. As described above, the remote control 2 is connected to the device 1 by wire or wirelessly, and is usually associated with the device 1 on a one-to-one basis, and performs operations on the corresponding device 1. The remote control 2 includes a display unit 3 and various switches 4. The display unit 3 is, for example, a touch panel, and can display a status screen 6, and the status screen 6 is displayed including buttons for performing various operations.

[0045] The mobile terminal 10 may be any of a variety of smart devices, such as a mobile phone or a smartphone, that can be connected wirelessly or with a wire to a communication line such as the Internet. The mobile terminal 10 includes a display screen 22 that can display various kinds of information, and an imaging device 21. In this embodiment, the mobile terminal 10 uses the imaging device 21 to capture a status screen 6 displayed on the display unit 3 of the remote control 2 as a screen image 37.

[0046] The server 5 is connected to a plurality of devices 1 directly or via the remote control 2 in a state in which data communication is possible through a communication line such as the Internet. In the following description, it is assumed that various information of the devices 1 or the remote control 2 is acquired through the remote control 2. The server 5 can acquire information indicating the operation state of the devices 1 and information related to the status screen 6 displayed on the remote control 2 through the remote control 2, and transmit various information to the remote control 2 or the devices 1. The server 5 is also connected to the mobile terminal 10 in a state in which data communication is possible through a communication line such as the Internet. The server 5 can transmit various information to the mobile terminal 10, and can acquire various information from the mobile terminal 10. For example, the server 5 acquires a screen image 37 captured by the mobile terminal 10, and acquires display information 36 related to the status screen 6 from the remote control 2.

[0047] The server 5 includes a control unit 25, a communication unit 26, a screen extraction unit 29, a screen identification unit 30, a screen description information control unit 32, and a storage unit .

[0048] The control unit 25 includes a processor such as a CPU, and controls the communication unit 26, screen extraction unit 29, screen identification unit 30, screen description information control unit 32, and storage unit 34, thereby controlling the operation of the server 5. The communication unit 26 controls data communication with the remote control 2 and the mobile terminal 10, receives information from the remote control 2 and the mobile terminal 10, and transmits information to the remote control 2 and the mobile terminal 10. The storage unit 34 stores various types of information.

[0049] The screen extraction unit 29 extracts the characteristic part 8, or the fixed part 7 and the characteristic part 8, from the fixed part 7, the characteristic part 8, and the changing part 9 of the status screen 6 (status display screens 6A to 6D) displayed on the remote control 2, from the screen image 37 captured by the mobile terminal 10 and received via the communication unit 26.

[0050] The screen identification unit 30 identifies the status screen 6 (status display screens 6A to 6D) of the remote control 2 photographed by the mobile terminal 10, using at least the unique part 8 extracted by the screen extraction unit 29 from the identification information database 42 described later.

[0051] The screen description information control unit 32 extracts screen description information 38 related to the status screen 6 (status display screens 6A to 6D) identified by the screen identification unit 30 from a screen description information database 41 described below. The screen description information control unit 32 combines the extracted screen description information 38 with the status screen 6 (status display screens 6A to 6D) identified by the screen identification unit 30, transmits the combined image to the display screen 22 of the mobile terminal 10 via the communication unit 26, and causes the combined image to be displayed on the display screen 22.

[0052] Here, the screen description information 38 is information explaining the meanings of the patterns 12 to 14, letters 15, arrows 16, units 17, numbers 11, etc. displayed on the status screen 6, as well as how to operate the buttons, etc., and is associated with each of the displayable status screens 6, described in the screen description information database 41, and stored in the memory unit 34.

[0053] Such screen explanatory information 38 is combined with the status screen 6 displayed on the remote control 2 and displayed on the mobile terminal 10, so that the user can operate the remote control 2 while checking the screen explanatory information 38 displayed on the mobile terminal 10. This makes it possible to provide easy-to-understand information in a manner displayed on the mobile terminal 10 while minimizing restrictions on the remote control 2.

[0054] Next, the processing flow of the information display system will be described with reference to Fig. 5. The display of information according to this embodiment may be implemented by the information display system shown in Fig. 4, or may be implemented by an information display method and an information display program that perform the processing flow shown in Fig. 5. Such an information display method and information display program may be implemented by using the information display system shown in Fig. 4, but can be implemented by any configuration. The information display program is stored in the storage unit 34 and executed by the control unit 25 of the server 5 or a processor provided in the mobile terminal 10.

[0055] First, the remote control 2 continuously obtains, from the corresponding device 1, display information 36 that changes according to the state of the device 1, and displays, on the display unit 3, a status screen 6 according to the obtained display information 36 as needed. The remote control 2 accepts operations of the device 1 while displaying this status screen 6 (step #1 in FIG. 5).

[0056] Specifically, the display information 36 includes information on whether power is being generated, whether power generation is stopped, whether the device 1 is broken, information on whether power is being purchased or sold, the power being purchased, the power being sold, the power being generated, etc. Then, the remote control 2 determines, according to the display information 36, which of the status display screens 6A to 6D to display as the status screen 6, and what display to display as the changed portion 9.

[0057] The display information 36 is also transmitted to the server 5 via the communication unit 26 and stored in the storage unit 34. At this time, the status screen 6 displayed together with the display information 36 is also transmitted to the server 5. The server 5 extracts the unique portion 8 in the status screen 6 from the status screen 6, generates a specific information database 42 in which the unique portion 8 or the unique portion 8 and the fixed portion 7 are associated with the status screen 6, and stores the specific information database 42 in the storage unit 34. Note that the specific information database 42 may be generated in advance by associating combinations of the unique portion 8 and the fixed portion 7 with the status screen 6, and stored in the storage unit 34.

[0058] Further, a screen description information database 41 is stored in advance in the storage unit 34 of the server 5. The screen description information database 41 is a database in which screen description information 38 relating to each of a plurality of status screens 6 assumed to be displayed on the remote control 2 is associated with the status screen 6. As described above, the screen description information 38 is information that easily and clearly explains to the user the meanings of the patterns 12 to 14, characters 15, arrows 16, units 17, numerical values ​​11, etc. displayed on the status screen 6, as well as the operation methods of buttons, etc.

[0059] Next, when the user wants to display an explanation of the status screen 6, the user uses the imaging device 21 of the mobile terminal 10 to capture the status screen 6 displayed on the remote control 2 as a screen image 37 (step #2 in FIG. 5). The captured screen image 37 is transmitted to the server 5 via the communication unit 26 and stored in the storage unit 34.

[0060] When the control unit 25 of the server 5 receives the screen image 37 from the mobile terminal 10, it controls the screen extraction unit 29 to extract the fixed portion 7 from the screen image 37 by referring to the display information 36 (step #3 in FIG. 5), and extract the unique portion 8 from the image remaining after removing the fixed portion 7 from the screen image 37 (step #4 in FIG. 5). Note that the unique portion 8 may or may not include the changing portion 9.

[0061] Next, the screen identification unit 30 identifies the status screen 6 corresponding to the screen image 37, i.e., the status screen 6 displayed on the remote control 2, using the specific information database 42 and the unique part 8 extracted by the screen extraction unit 29 (step #5 in FIG. 5).

[0062] Specifically, the screen identification unit 30 compares the characteristic portion 8 extracted by the screen extraction unit 29 with the characteristic portion 8 associated with the specific information database 42, and if a matching characteristic portion 8 is found in the specific information database 42, it identifies the status screen 6 associated with the matching characteristic portion 8 as the status screen 6 corresponding to the screen image 37, that is, the status screen 6 displayed on the remote control 2. In other words, the screen identification unit 30 uses the characteristic portion 8 unique to the status display screens 6A to 6D of the screen image 37, excluding the fixed portion 7 that does not change in the status screen 6 and the changing portion 9 that changes irregularly in the operating state of the device 1, as a comparison target, and identifies which status screen 6 (status display screen 6A to 6D) it is from the specific information database 42.

[0063] If no matching unique portion 8 is found in the comparison in screen identification section 30, a notice to that effect is sent to the user, and the user again uses imaging device 21 of mobile terminal 10 to capture status screen 6 displayed on remote control 2 as screen image 37 (step #6 in FIG. 5). Such notice can be sent by any method, but for example, the notice is sent by displaying a notice on display screen 22 of mobile terminal 10 or by playing a voice notifying the user of the notice from mobile terminal 10.

[0064] When a matching unique portion 8 is found by the comparison in the screen identification unit 30, the screen description information control unit 32 extracts screen description information 38 corresponding to the identified status screen 6 from the screen description information database 41.

[0065] Specifically, the screen description information control unit 32 compares the identified status screen 6 with the status screen 6 associated with the screen description information database 41, and extracts the screen description information 38 associated with the matching status screen 6 in the screen description information database 41 as the screen description information 38 related to the identified status screen 6, i.e., the status screen 6 displayed on the remote control 2.

[0066] Then, the screen description information control unit 32 combines the extracted screen description information 38 with the status screen 6, and displays it on the display screen 22 of the mobile terminal 10. Specifically, the screen description information control unit 32 transmits the combined composite image to the mobile terminal 10 via the communication unit 26, and the mobile terminal 10 displays the received composite image on the display screen 22 (step #7 in FIG. 5).

[0067] 6 and 7 show examples of composite images obtained by combining the screen explanation information 38 and the status screen 6. Fig. 6(a) is a composite image when the status display screen 6A (Fig. 2(a)) is displayed on the remote control 2, Fig. 6(b) is a composite image when the status display screen 6B (Fig. 2(b)) is displayed on the remote control 2, Fig. 7(c) is a composite image when the status display screen 6C (Fig. 2(c)) is displayed on the remote control 2, and Fig. 7(d) is a composite image when the status display screen 6D (Fig. 2(d)) is displayed on the remote control 2. In this way, the composite image displays explanations of the various information displayed on the status screen 6, guidance for the status, and the like.

[0068] In this way, by displaying on the mobile terminal 10 the status screen 6 displayed on the remote control 2 and the screen explanation information 38 which is an explanation related to the status screen 6, the user can easily understand the meanings of the patterns 12-14, letters 15, arrows 16, units 17, numbers 11, etc. displayed on the status screen 6 (corresponding to "an explanation of the contents of the status screen 6"), how to operate the buttons (corresponding to "how to operate the operating device (remote control 2)"), etc.

[0069] Furthermore, the status screen 6 displayed on the remote control 2 is photographed by the mobile terminal 10, the displayed status screen 6 is identified from the unique portion 8 of the screen image 37, and screen description information 38 related to the status screen 6 is extracted and displayed together with the status screen 6 on the mobile terminal 10, thereby making it possible to easily display the screen description information 38 related to the status screen 6 displayed on the remote control 2 on the mobile terminal 10.

[0070] [Another embodiment] (1) In the above-described information display system, each functional block is not limited to the configuration shown in the functional block in FIG. 4, and any functional block may be integrated, or any of the functional blocks may be further divided into multiple functional blocks.

[0071] (2) In each of the above embodiments, the smart device that captures the status screen 6 and displays the screen description information 38 is not limited to the mobile terminal 10, and various smart devices can be used. The smart device may be configured to include at least two devices, each of which is configured to be separable, with one device provided with the display screen 22 and the other device provided with the imaging device 21. In this way, the screen image 37 is captured by the imaging device 21 and the screen description information 38 is displayed on the display screen 22 of the other device, so that the screen description information 38 can be displayed in a state that is easier to confirm.

[0072] (3) In each of the above embodiments, the screen extraction unit 29, the screen identification unit 30, the screen description information control unit 32, and the storage unit 34 are not limited to being provided in the server 5, but some or all of them may be provided in the mobile terminal 10 or the remote control 2, or may be distributed among the server 5, the mobile terminal 10, and the remote control 2. Also, without providing a server 5, the screen extraction unit 29, the screen identification unit 30, the screen description information control unit 32, and the storage unit 34 may be distributed among the mobile terminal 10 and the remote control 2.

[0073] (4) In each of the above embodiments, the image in which the status screen 6 and the screen description information 38 are combined may be displayed not only on the mobile terminal 10 but also on a separately provided display device. Also, the screen description information 38 is not limited to being combined with the status screen 6 and displayed, and only the screen description information 38 may be displayed in a state in which the information corresponding to the status screen 6 is clear. Also, the screen description information 38 displayed on the display screen 22 is not limited to being combined with the status screen 6 and displayed, and may be displayed in any manner in which the description of the status screen 6, etc., can be understood. For example, the screen description information 38 and the status screen 6 may be displayed side by side, vertically or horizontally. Also, the screen description information 38 may be combined with the screen image 37 captured by the mobile terminal 10 and displayed. With such a configuration, the degree of freedom of the display manner of the screen description information 38 is improved, and the screen description information 38 can be displayed in a state in which it is easier to confirm depending on the situation.

[0074] In addition, the configurations disclosed in the above embodiments (including other embodiments, the same applies below) can be applied in combination with configurations disclosed in other embodiments, provided that no contradiction arises. Furthermore, the embodiments disclosed in this specification are illustrative, and the embodiments of the present invention are not limited thereto, and can be appropriately modified within the scope that does not deviate from the purpose of the present invention. [Industrial Applicability]

[0075] The information display system, the information display method, and the information display program of the present invention can be used to display information on a screen displayed on an operating device such as a remote control that operates various devices. [Explanation of symbols]

[0076] 1 equipment 2 Remote control (operating device) 3 Display section 6 Status screen 7 Fixed part 8 Unique parts 9. Changes 10 Mobile terminals (smart devices) 21 Imaging device 22 Display screen 29 Screen extraction part 30 Screen specific part 32 Screen description information control section 34 Storage section 36 Display information 37 Screen Image 38 Screen Description Information 41 Screen Description Information Database 42 Specific Information Database

Claims

1. Equipment and an operation device having a display unit for displaying a status screen that changes according to a status of the device and for operating the device; A smart device having an imaging device and a display screen; a storage unit that stores a screen description information database in which screen description information related to the status screen is associated with the status screen, and a specific information database in which a unique portion of the status screen is associated with the status screen; a screen extraction unit that acquires a screen image of the status screen displayed on the display unit of the operation device by the imaging device, and extracts the unique portion from the screen image; a screen identification unit that identifies the status screen corresponding to the extracted unique portion from the specific information database; An information display system comprising: a screen description information control unit that extracts the screen description information related to the identified status screen from the screen description information database, synthesizes it with the identified status screen, and displays it on the display screen of the smart device.

2. 2. The information display system of claim 1, wherein the status screen changes to one of a plurality of status display screens, and includes a fixed portion that is fixed in all of the status display screens, a unique portion that is fixed in the same status display screen, and a variable portion that changes depending on the status of the device.

3. The information display system according to claim 1 or 2, wherein at least one of the storage unit, the screen extraction unit, the screen specification unit, and the screen description information control unit is mounted on the smart device.

4. a server capable of communicating with the operation device and the smart device; The information display system according to claim 1 or 2, wherein at least one of the memory unit, the screen extraction unit, the screen identification unit, and the screen description information control unit is mounted on either the smart device or the server.

5. The information display system according to claim 1 , wherein the screen instruction information includes at least one of an explanation of the contents of the status screen and an explanation of a method of operating the operating device.

6. The information display system according to claim 1 , wherein the smart device is a mobile terminal.

7. A step of capturing a status screen of the device displayed on an operation device for operating the device as a screen image using a smart device; extracting a characteristic portion from the screen image; identifying the status screen corresponding to the extracted unique portion from a specific information database in which the status screen and the unique portion of the status screen are associated with each other; extracting screen description information related to the specified status screen from a screen description information database in which the status screen and screen description information related to the status screen are associated with each other; and combining the extracted screen description information with the specified status screen and displaying the combined information on a display screen of the smart device.

8. A function of acquiring a screen image of a status screen of the device, which is displayed on an operation device that operates the device, captured by a smart device; extracting a distinctive portion from the screen image; a function of identifying the status screen corresponding to the extracted characteristic portion from a specific information database in which the status screen and the characteristic portion of the status screen are associated with each other; A function of extracting screen description information related to the specified status screen from a screen description information database in which the status screen and screen description information related to the status screen are associated with each other; and a function of combining the extracted screen description information with the specified status screen and displaying the combined information on a display screen of the smart device.

Images