Device management apparatus, device management system, input / output device, device management method, and program

Abstract

To easily select a desired device from a plurality of available devices.SOLUTION: A device management device includes a display processing part for causing a display part to display a display screen where display elements indicating each device are arranged at different positions, and changing at least one of the positions of the display elements, and a range specification figure indicating a selection region constituting a part of the display screen according to operation, and a device management part for selecting a device corresponding to the display element in which a positional relation between the range specification figure and the display element varies as an object device, and controlling an operation state of the object device, wherein the display processing part determines a display mode of the display element of the object device as a display mode corresponding to the operation state of the object device.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a device management apparatus, a device management system, an input / output device, a device management method, and a program. For example, the present invention relates to a method for providing a graphical user interface for selecting a device to actually use for playback from among playback-capable devices capable of playing information distributed by media. [Background technology]

[0002] The devices that can acquire information distributed via media such as broadcasting and the Internet are becoming more diverse. For example, while radio receivers for receiving radio broadcasts and television receivers for receiving television broadcasts have traditionally been widely used, smartphones have become increasingly popular in recent years. Furthermore, IoT (Internet of Things) devices capable of displaying various types of information, including smartwatches and smart speakers, are expected to become more widespread. Given this trend, the diversification of devices used by viewers is expected to continue.

[0003] As devices become more diverse, information presentation is not limited to a single device, but also to various forms of information presentation that link various devices together. For example, music can be played using earphones wirelessly connected to a smartphone, or a video playing on the smartphone can be played on the television receiver from where it left off by linking the smartphone and a television receiver connected to a home local area network (LAN).

[0004] Therefore, a method for selecting a device to be used for presenting information from a variety of devices is expected. For example, an information processing device described in Patent Document 1 grasps a user's privacy level based on information about the user, and when it is determined that a preset output form is not suitable based on the privacy level of the preset output form, selects an optimal output form from among multiple output forms based on the user's privacy level and the privacy levels corresponding to each of the multiple output forms. The control method described in Patent Document 2 includes a step of performing control corresponding to a second predetermined operation related to control of a smart hardware device when a first predetermined operation is performed on a control key on the screen lock screen of a terminal device.

[0005] The information processing device described in Patent Document 3 performs a process of guiding the user's line of sight to the location of a device to be controlled, and a process of controlling the timing of executing an operation command in the device to be controlled according to the timing of guiding the line of sight. The control method described in Patent Document 4 includes a step of adding a chat room with a bot to a messenger to link the user with an IoT server, and a step of controlling an IoT device by sending a control signal of the IoT device to the IoT server in accordance with a control input entered by the user in the chat room. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-91221 [Patent Document 2] Special Publication No. 2017-509277 [Patent Document 3] Japanese Patent Application Publication No. 2019-113928 [Patent Document 4] Japanese Patent Publication No. 2020-61131 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the above method does not take into consideration the overall availability of devices for the user, and only selects a specific device based on objective criteria, so the device desired by the user may not necessarily be used to present information.

[0008] The present invention has been made in consideration of the above points, and aims to provide a device management apparatus, a device management system, an input / output device, a device management method, and a program that allow a desired device to be easily selected. [Means for solving the problem]

[0009] The present invention has been made to solve the above problems, [1] One aspect of the present invention includes a display processing unit that displays a display screen on a display unit in which display elements representing individual devices are arranged in different positions, and that changes, in response to an operation, at least one of the positions of the display elements and a range designation figure that indicates a selection range and that forms part of the display screen; and a device management unit that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation figure has changed, and controls the operating state of the target device, and the display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. When a type of information is specified in response to an operation, the device management data indicating the type of information that can be presented for each device is referenced, a device that can present the specified type of information is identified as a presentable device, and display elements of the presentable device in a display mode that is different from display elements of a device separate from the presentable device. It is a device management system. According to the configuration of [1], the operating state of the target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can easily select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. In addition, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices. [2] One aspect of the present invention includes a display processing unit that displays a display screen on a display unit in which display elements representing individual devices are arranged in different positions, and that changes, in response to an operation, at least one of the positions of the display elements and a range designation figure that indicates a selection range and that is part of the display screen; and a device management unit that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation figure has changed, and controls the operating state of the target device, and the display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. For each device, refer to device management data indicating other devices that can be connected, and for each device, display a graphic indicating the possibility of connection with the other devices between the display elements corresponding to the one device and the other devices. It is a device management system. According to the configuration of [2], the operating state of the target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can easily select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. Furthermore, the user can easily understand that the devices corresponding to the display elements shown on both ends of the graphic indicating the connectability are connectable to each other.

[0010] [3] One aspect of the present invention is the above-mentioned device management system, wherein on the display screen, a user display element, which is a display element indicating a user, is positioned at a different position from a device display element, which is a display element indicating the device, and the range designation figure is displayed in an area including the user display element, and the device management unit may select as the target device a device corresponding to a device display element entering the selection range or a device display element exiting the selection range. [3] According to this configuration, the operating state of the device corresponding to the user display element can be intuitively controlled by operating the user display element to enter or exit the range specification figure.

[0011] [4]One aspect of the present invention is the above-mentioned device management system, wherein the display processing unit displays the range specification figure in a selection range indicated in accordance with an operation, and the device management unit selects a device corresponding to a display element included in the selection range as a target device. [4] According to the configuration of (1), a device whose operational state is to be controlled can be intuitively selected by a selection range defined by an operation using a range designation figure.

[0013] [5] One aspect of the present invention is the device management system described above, wherein when an installation location is specified in response to an operation, the display processing unit refers to device management data indicating the installation location of each device, identifies devices that can be used in the specified location as available devices, and displays display elements of the available devices in a display manner different from display elements of devices separate from the available devices. According to the configuration of [5], by specifying the installation location, devices that can be used in the specified installation location are selected. Therefore, by specifying the installation location, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[0015] [6] One aspect of the present invention is a device management apparatus comprising: a display processing unit that displays a display screen on a display unit in which display elements representing individual devices are arranged in different positions, and that changes, in response to an operation, at least one of the positions of the display elements and a range designation graphic that indicates a selection range and that forms part of the display screen; and a device management unit that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device, wherein the display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device; When a type of information is specified in response to an operation, the device management data indicating the type of information that can be presented for each device is referenced, a device that can present the specified type of information is identified as a presentable device, and display elements of the presentable device are displayed in a display mode that differs from display elements of a device separate from the presentable device. It is a device management device. [6]According to this configuration, the operating state of a target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can immediately select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. In addition, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[0016] [7] One aspect of the present invention is an input / output device that includes a display processor that displays a display screen on a display unit in which display elements representing individual devices are arranged in different positions, and that changes at least one of the positions of the display elements and a range designation graphic that indicates a selection range and that forms part of the display screen in response to an operation, and the display processor selects, as a target device, a device corresponding to a display element that has changed in its positional relationship with the range designation graphic, and determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. When a type of information is specified in response to an operation, the device management data indicating the type of information that can be presented for each device is referenced, a device that can present the specified type of information is identified as a presentable device, and display elements of the presentable device in a display mode that is different from display elements of a device separate from the presentable device. . [7] According to the configuration, detected devices in operation become candidates for providing devices related to providing distribution information, and non-detected devices are excluded as providing devices. Therefore, a providing device can be efficiently selected from the active devices, and unnecessary processing related to providing distribution information to non-detected devices can be avoided. In addition, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[0017] [8] One aspect of the present invention is to [6] Device management equipment or [7] The program may be a program for causing the device to function as an input / output device. [8] According to this configuration, the operating state of a target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can immediately select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. In addition, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[0018] [9] One aspect of the present invention is a device management method that executes the following steps: a first step in which a display processing unit displays a display screen on a display unit, in which display elements representing individual devices are arranged in different positions; a second step in which the display processing unit changes, in response to an operation, at least one of the positions of the display elements and a range designation graphic that indicates a selection range and that is part of the display screen; a third step in which a device management unit selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device; and a fourth step in which the display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. wherein the fourth step includes a step of, when a type of information is specified in response to an operation, referring to device management data indicating the type of information that can be presented for each device, specifying a device that can present the specified type of information as a presentable device, and displaying display elements of the presentable device in a display mode that is different from display elements of a device separate from the presentable device. . [9] According to this configuration, the operating state of a target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can immediately select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. In addition, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[10] One aspect of the present invention is a device management method that executes the following steps: a first step in which a display processing unit displays a display screen on a display unit, in which display elements representing individual devices are arranged in different positions; a second step in which the display processing unit changes, in response to an operation, at least one of the positions of the display elements and a range designation graphic that indicates a selection range and that is part of the display screen; a third step in which a device management unit selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device; and a fourth step in which the display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. wherein the fourth step includes a step of referring to device management data indicating other devices that can be connected for each device, and displaying, for each device, a graphic indicating the possibility of connection with the other devices between display elements corresponding to the one device and the other devices. .

[10] According to this configuration, the operating state of a target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape through operation, and the display element is displayed in a display mode corresponding to that operating state. The user can immediately select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen. Furthermore, the user can easily understand that the devices corresponding to the display elements shown on both ends of the graphic indicating the connectability are connectable to each other. [Effects of the Invention]

[0019] According to the present invention, a desired device can be easily selected from a plurality of available devices. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is an explanatory diagram illustrating an overview of a device management system according to a first embodiment. [Figure 2] 1 is a schematic block diagram illustrating an example of the functional configuration of a device management system according to a first embodiment. [Figure 3] FIG. 2 is a diagram showing a first display example of a display screen according to the first embodiment. [Figure 4] FIG. 10 is a diagram showing a second display example of the display screen according to the first embodiment. [Figure 5] FIG. 3 is a diagram showing a first example of device management data according to the first embodiment. [Figure 6] 10 shows an example of user-selected range data indicating the display range of the range designation graphic according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing a second example of device management data according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing a third example of device management data according to the first embodiment. [Figure 9] FIG. 4 is a sequence diagram illustrating an example of a device selection process according to the first embodiment. [Figure 10] FIG. 10 is an explanatory diagram illustrating an overview of a device management system according to a second embodiment. [Figure 11] FIG. 10 is a schematic block diagram illustrating an example of the functional configuration of a device management system according to a second embodiment. [Figure 12] FIG. 10 is a diagram showing a first example of device management data according to the second embodiment. [Figure 13] FIG. 10 is a diagram showing an example of user-selected range data according to the second embodiment. [Figure 14] FIG. 10 is a diagram showing a first display example of a display screen according to the second embodiment. [Figure 15] FIG. 10 is a diagram showing a second display example of the display screen according to the second embodiment. [Figure 16] FIG. 10 is a diagram showing a third display example of the display screen according to the second embodiment. [Figure 17] FIG. 10 is a diagram showing a fourth display example of the display screen according to the second embodiment. [Figure 18] FIG. 10 is a diagram showing a fifth display example of the display screen according to the second embodiment. [Figure 19] FIG. 10 is a diagram showing a sixth display example of the display screen according to the second embodiment. [Figure 20] FIG. 13 is a diagram showing a seventh display example of the display screen according to the second embodiment. [Figure 21] FIG. 13 is a diagram showing an eighth display example of the display screen according to the second embodiment. [Figure 22] FIG. 10 is a diagram showing a second example of device management data according to the second embodiment. [Figure 23] FIG. 13 is a diagram showing a ninth display example of the display screen according to the second embodiment. [Figure 24] FIG. 11 is a diagram showing a third example of device management data according to the second embodiment. [Figure 25] FIG. 10 is a sequence diagram showing a first example of a device selection process according to the second embodiment. [Figure 26] FIG. 10 is a sequence diagram showing a second example of a device selection process according to the second embodiment. [Figure 27] FIG. 10 is a sequence diagram showing a third example of the device selection process according to the second embodiment. [Figure 28] FIG. 10 is a sequence diagram showing a fourth example of the device selection process according to the second embodiment. [Figure 29]FIG. 10 is a sequence diagram illustrating an example of a device deselection process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First Embodiment FIG. 1 is an explanatory diagram showing an overview of a device management system 1 according to the first embodiment. The device management system 1 includes a control device 100 and an input / output device 200. In the illustrated example, the device management system 1 is installed in an indoor living space. Here, an IoT gateway corresponds to the control device 100, and a tablet PC (Personal Computer) corresponds to the input / output device 200.

[0022] The control device 100 detects devices installed in a specified living space and manages the functions and operating states of the detected devices. The control device 100 outputs the operating states of the detected devices to the input / output device 200. The devices to be detected are devices that have a function of presenting distribution information distributed to the device itself. In the illustrated example, in addition to a tablet PC, a smartphone, a smart speaker, and a TV (television set) can be detected. In this application, a device that has a function of presenting distribution information may be referred to as a "presentation device." The input / output device 200 may also be configured to present information as a presentation device. The number of presentation devices can be two or more.

[0023] The input / output device 200 receives input of device operating states from the control device 100. The input / output device 200 generates a display screen in which device icons are arranged in different positions as examples of display elements representing individual devices, and displays the generated display screen on the display unit. In this embodiment, a user icon representing the user of the input / output device 200 is also arranged in a different position from the device icon on the display screen, and a range designation graphic is arranged in a position whose area includes the user icon. In the example of FIG. 1, the device icons of a smart speaker, a tablet PC (the input / output device 200 itself), a TV, and a smartphone are each represented by a circle. The user icon is a circle arranged near the character string "you."

[0024] In the input / output device 200, a circle centered on the user icon corresponds to a range designation graphic indicating a selection range for selecting a device icon. The input / output device 200 accepts a user operation and makes it possible to change the position of each device icon in accordance with the operation. The input / output device 200 notifies the control device 100 of operation information indicating the position of the specified device icon.

[0025] The control device 100 selects, as a target device, a device corresponding to a device icon whose positional relationship with the range designation figure has changed, and controls the operating state of the target device. The control device 100 selects, as a target device, a device corresponding to a device icon that enters the selection range from outside, and causes the selected target device to begin presenting distribution information. The input / output device 200 displays the device icon of the target device for which distribution information has been presented in a darker color than before the distribution information was presented. The input / output device 200 selects, as a target device, a device corresponding to a device icon that exits from within the selection range to outside the selection range. The control device 100 selects, as a target device, a device corresponding to a device icon that enters from within the selection range to outside the selection range, and stops presenting distribution information to the selected target device. The input / output device 200 displays the device icon of the target device for which presentation of distribution information has been stopped in a brighter color than before the stop. In the illustrated example, the TV and smartphone are activated, and the smart speaker and tablet PC (information presentation function) are stopped. This allows the user to immediately check the presentation state of the distributed information by each device and adjust the presentation state with a simple operation. In this application, a device having a function of presenting distributed information may be referred to as a "presentation device." In the example shown in FIG. 1, the smart speaker and the TV correspond to the presentation devices. In addition, the input / output device 200 (smartphone) also functions as a presentation device.

[0026] Next, an example of the functional configuration of the device management system 1 according to this embodiment will be described. FIG. 2 is a schematic block diagram showing an example of the functional configuration of the device management system 1 according to this embodiment. The device management system 1 includes a control device 100 and an input / output device 200. The control device 100 and the input / output device 200 are connected via a network NW so that various types of information can be input and output. The network NW may be either wireless or wired. The network NW is, for example, a LAN that can transmit information wirelessly using a predetermined communication method such as IEEE802.11 or IEEE802.15. Devices separate from the control device 100 and the input / output device 200, such as a presentation device 300, can also be connected to the network NW. The number of presentation devices 300 is not limited to one and is typically two or more.

[0027] The control device 100 detects presentation devices in a predetermined management area and controls the operating status of the detected presentation devices. The control device 100 may be a dedicated device whose primary function is device management, or may be a device having another primary function. The control device 100 may be configured as another type of gateway device (e.g., a router device), a meter (e.g., a power meter, a gas meter, etc.), or may have the function of a presentation device (e.g., a television receiver, a recording device), and may manage the operating status of the control device itself and / or the input / output device 200. In the example shown in FIG. 1 , the management area is a residence where one or more people reside. The residents are users who receive distributed information and may be users of the input / output device 200 and the control device 100. One management area does not necessarily correspond to one building. For example, the management area may be one or more dwelling units that are part of an apartment building.

[0028] The control device 100 includes a storage unit 122 , a device management unit 124 , and a communication unit 126 . The storage unit 122 temporarily or permanently stores various types of data. The storage unit 122 includes storage media such as RAM (Random Access Memory) and ROM (Read Only Memory). The stored data includes, for example, device management data and user selection range data. The device management data includes various types of information for each presentation device. The device management data may include presentable information and operating status information for each presentation device. The device management data may also include detection information for each presentation device. The user selection range data is data indicating a range designation graphic that allows the user to select an icon for displaying information. Specific examples of device management data will be described later.

[0029] The device management unit 124 manages information about presentation devices located within its management area. The device management unit 124 may detect powered presentation devices connected to its own device at predetermined detection intervals (e.g., every few seconds to every few minutes). When detecting a presentation device, the device management unit 124, for example, causes the communication unit 126 to broadcast a notification signal (beacon) to the network NW connected to its own device according to a predetermined communication method. When the device management unit 124 receives a response signal to the notification signal from any presentation device within a predetermined detection time from the transmission of the notification signal, it can determine that the presentation device that is the sender of the response signal has been detected. When the presentation device does not receive a response signal to the notification signal or receives a negative response signal to the notification signal, it can determine that the presentation device has not been detected. The device management unit 124 extracts the identification information of the presentation device from the response signal and uses the extracted identification information to identify the presentation device that sent the response signal. Even if a managed device is installed within the management area, it will not be detected if it is not receiving power and is not operating. Therefore, the number of managed devices that are operating may change from time to time.

[0030] The device management unit 124 associates detected device information indicating the detection of a newly identified device with the presentation device and stores it in the storage unit 122. This updates the device management data. When a presentation device whose detection information is stored in the storage unit 122 is not transmitted by the input detected device information, the device management unit 124 determines that the presentation device has not been detected. The device management unit 124 associates detection information indicating the non-detection of the presentation device with the presentation device and stores it. This updates the device management data.

[0031] The device management unit 124 monitors operation information received from the input / output device 200 using the communication unit 126 and determines the positional relationship between the position of a device icon indicated by the operation information and the selection range defined by a predetermined range selection graphic. The device management unit 124 identifies a device icon that has entered the selection range from outside and causes the presentation device corresponding to the identified device icon to start presenting information. The device management unit 124 uses the communication unit 126 to send a control signal indicating the start of information presentation to the presentation device. When an acknowledgement response to the control signal is input within a predetermined time after transmission of the control signal, the device management unit 124 associates operation status information indicating that information has been presented by the presentation device with the presentation device and stores it in the storage unit 122. The device management unit 124 sends device management data including the updated operation status information to the input / output device 200 using the communication unit 126.

[0032] The device management unit 124 identifies a device icon that has exited from within the selection range and causes the presentation device corresponding to the identified device icon to stop presenting information. The device management unit 124 uses the communication unit 126 to send a control signal indicating that information presentation should be stopped to the presentation device. When an acknowledgement (ACK) to the control signal is input within a predetermined time from the transmission of the control signal, operation status information indicating that information will not be presented by the presentation device is stored in the storage unit 122 in association with the presentation device. The device management unit 124 sends device management data including the updated operation status information to the input / output device 200 using the communication unit 126. If an acknowledgment response to a control signal is not received within a predetermined time from the transmission of the control signal, or if a negative acknowledgment response (NACK) is received in response to the control signal, the device manager 124 does not update the device management data, as it is assumed that the change in the operating state instructed by the control signal has failed.

[0033] The communication unit 126 is capable of communicating with other devices wirelessly or via a wired connection using a predetermined communication method. The communication unit 126 includes, for example, a communication interface. The other devices with which the communication unit 126 can communicate include the input / output device 200 and the presentation device 300.

[0034] The input / output device 200 displays a display screen on which device icons of the respective presentation devices are arranged, and accepts user operations to instruct the operating state of any of the presentation devices. The input / output device 200 may be any type of device as long as it can visibly display information and accept user operations. The input / output device 200 is not limited to information and communication devices such as a general-purpose tablet PC or a smartphone, and may also be a dedicated controller (remote control) or the like.

[0035] The input / output device 200 includes a storage unit 222 , a communication unit 226 , a display processing unit 228 , a display unit 234 , and an operation unit 236 . The storage unit 222 stores various types of data. The storage unit 222 includes storage media such as RAM, ROM, etc. The data stored in the storage unit 222 will be described together with the function of the display processing unit 228. The communication unit 226 is capable of communicating with other devices wirelessly or via a wire using a predetermined communication method. The communication unit 226 is configured to include, for example, a communication interface. The other devices with which the communication unit 226 can communicate include the control device 100.

[0036] The display processing unit 228 generates a display screen and displays the generated display screen on the display unit 234. A range specification figure is arranged on the display screen. A user icon representing the user is arranged at a representative point (e.g., the center, center of gravity, etc.) of the range specification figure. The display range of the range specification figure is set as a selection range, and device icons of presentation devices that will present information are displayed within the selection range, and device icons of presentation devices that will not present information are displayed outside the selection range. The display processing unit 228 refers to the device management data received from the control device 100, identifies the presentation devices detected by the control device 100, and determines the identified presentation devices as devices on which device icons are to be placed (see FIGS. 3 and 5). The display processing unit 228 describes, in the device management data, coordinates indicating the display position on the display screen of the user icon for each presentation device.

[0037] The display processing unit 228 may randomly determine the coordinates of the initial display position of the device icon within or outside the display range using pseudo-random numbers, or may use predetermined coordinate values. The display processing unit 228 may specify the coordinates of the initial display position of the device icon of the presented device by referring to the latest device management data stored in the storage unit 222. The closer the installation location of a presented device is to the input / output device 200, the closer the display position of the presented device may be to the user icon. The display processing unit 228 may specify the installation location of each presented device by referring to the device management data (FIG. 12).

[0038] The display processing unit 228 varies the display position of a device icon designated in response to a predetermined type of operation (e.g., a drag operation). A drag operation is an operation for continuously moving a pressed position on the display screen in space while pressing the position. In this application, "pressing" may mean not only actually pressing a certain area or position, but also inputting operation information indicating that area or position. The display processing unit 228 sequentially updates the device icon designated by operation information input from the operation unit 236 to the position on the display screen designated by the operation information. In other words, the operation information designates the device icon (sometimes referred to as an "operation icon") and display position of the presented device to be operated. The display processing unit 228 transmits the input operation information to the control device 100 using the communication unit 226.

[0039] The display processing unit 228 monitors whether the device management data received from the control device 100 using the communication unit 226 has been updated. The display processing unit 228 can detect the addition or removal of a presented device by comparing the latest device management data with the immediately previous device management data. The display processing unit 228 adds a device icon representing an added presented device to the display screen. The display processing unit 228 erases a device icon representing a removed presented device from the display screen.

[0040] The display processing unit 228 can detect a change in the operating state of a maintained presentation device by comparing the latest device management data with the immediately preceding device management data. The display processing unit 228 may change the display mode of the device icon of a presentation device whose operating state has changed from no information presented to information presented to a display mode corresponding to information presented. The display mode may be any one or a combination of color, brightness, shape, pattern, etc. For a presentation device whose operating state does not change to information presented even though the display position of the device icon moves from outside the selection range to within the selection range, the display processing unit 228 may return the display position of the device icon to a position outside the selection range. This is because it is presumed that the change in the operating state of the presentation device has failed.

[0041] The display processing unit 228 may change the display mode of the device icon of a presentation device whose operating state has changed from presenting information to not presenting information to a display mode corresponding to not presenting information. For a presentation device whose operating state does not change to not presenting information even though the display position of the device icon has moved from within the selection range to outside the selection range, the display processing unit 228 may return the display position of the device icon to within the selection range.

[0042] Each time a device icon is changed, that is, each time any one or a combination of processes such as addition, deletion, change in position, and change in display mode is performed, the display processing unit 228 outputs display data showing a display screen reflecting the process to the display unit 234. The display unit 234 displays a display screen reflecting the change in the device icon due to the operation, the detection state of the presented device, the operating state, etc.

[0043] The display unit 234 displays various display information based on the display data input from the display processing unit 228. The display unit 234 may be, for example, a liquid crystal display, an organic electroluminescence display, or the like. The operation unit 236 receives a user operation, generates operation information indicating information instructed by the received operation, and outputs the generated operation information to the display processing unit 228. The operation unit 236 may have general-purpose components such as a touch sensor, a mouse, a slider, or a joystick, or may have dedicated components such as a button or a knob. The touch sensor functioning as the operation unit 236 and the display functioning as the display unit 234 may be configured as a touch panel by being integrally overlapped with each other.

[0044] The presentation device 300 includes a communication unit 326 , a presentation information processing unit 332 , and an information presentation unit 334 . The communication unit 326 is capable of communicating with other devices wirelessly or via a wire using a predetermined communication method. The communication unit 326 is configured to include, for example, a communication interface. The other devices with which the communication unit 326 communicates include the control device 100. When the communication unit 326 receives a detection signal (beacon) from the control device 100, it transmits a response signal to the control device 100 in response to the detection signal. The communication unit 326 can extract identification information of the control device 100 from the detection signal and identify the control device 100 that sent the signal using the extracted identification information.

[0045] The communication unit 326 may have a function as an example of a distribution information acquisition unit that receives distribution information to be presented via a communication transmission path or a broadcast transmission path. The communication transmission path is a transmission path that allows the presentation device to both send and receive various types of data. The communication transmission path includes any one of the Internet, a public network, a LAN, a dedicated line, etc., or a combination thereof. The broadcast transmission path is a transmission path that allows the presentation device to receive data but not transmit data. The communication unit 326 outputs the acquired distribution information to the presentation information processing unit 332. The presentation device 300 may be provided with a distribution information acquisition unit separate from the communication unit 326. In that case, the communication unit 326 may omit the function of acquiring distribution information.

[0046] The presentation information processing unit 332 executes processing related to the presentation of the distribution information. The presentation information processing unit 332 monitors a control signal input from the control device 100 using the communication unit 326. When the input control signal indicates the start of information presentation, the presentation information processing unit 332 starts presenting the distribution information. At this time, the presentation information processing unit 332 starts receiving the distribution information instructed by the control signal using the communication unit 326. The presentation information processing unit 332 generates presentation data for presenting the distribution information. The processing for generating the presentation data may depend on the type of distribution information and the encoding method used to encode the distribution information. The presentation information processing unit 332 outputs the generated presentation data to the information presentation unit 334. When the presentation information processing unit 332 is successful in obtaining the distribution information and outputting the presentation data, it transmits an acknowledgment to the control device 100. On the other hand, when the presentation information processing unit 332 fails to obtain the distribution information or output the presentation data, it transmits a negative acknowledgment to the control device 100.

[0047] If the distribution information is video or auxiliary video, the presentation information processing unit 332 generates video data as presentation data using a video decoding method corresponding to the video encoding method used to encode the distribution information. If the distribution information is audio or auxiliary audio, the presentation information processing unit 332 generates audio data as presentation data using a decoding method corresponding to the audio encoding method used to encode the distribution information. If the distribution information is notification information, the presentation information processing unit 332 generates a drive signal according to a notification pattern indicating the distribution information as presentation data. Notification information is information expressed using an information transmission medium other than audio or video. Such information transmission media include light emission, vibration, and warning sounds. Device notification information is typically information that conveys non-verbal information such as cautions, warnings, and signals. The notification pattern is expressed as the presence or absence of presentation, or a time series consisting of repetition of the presence or absence of presentation at predetermined intervals. The drive signal for presenting the notification information may use power supplied to components related to the information presentation. The drive signals for light emission, vibration, and warning sound are expressed by the presence or absence or repetition of power supply to a light emitter (for example, a light emitting diode), vibrator, and sound generator (for example, a speaker), respectively.

[0048] Auxiliary video is video that shows content (for example, incidental, complementary, or alternative content) related to the main video (hereinafter referred to as "main video" and sometimes distinguished from auxiliary video). For example, if the main video is video from a drama, auxiliary video could include explanatory video or text about the characters and historical background, or subtitles in other languages. Auxiliary audio is audio that shows content (for example, incidental, complementary, or alternative content) related to the main audio (hereinafter referred to as "main audio" and sometimes distinguished from auxiliary audio). For example, if the main audio is music, auxiliary audio could include a specific instrument, a specific singing part, audio commentary for the visually impaired, audio commentary in another language, etc.

[0049] When the input control signal indicates that information presentation should be stopped, presentation information processing unit 332 stops presenting the distribution information. At this time, presentation information processing unit 332 stops acquiring distribution information and outputting it to information presenter 334. When the presentation information processing unit 332 is successful in acquiring the distribution information or stopping the output of the presentation data, it transmits an acknowledgment to the control device 100. On the other hand, when the presentation information processing unit 332 is unsuccessful in acquiring the distribution information or stopping the output of the presentation data, it transmits a negative acknowledgment to the control device 100.

[0050] The information presenter 334 presents presentation information based on the presentation data input from the presentation information processor 332. The information presenter 334 includes a component capable of presenting one or more types of information. The information presenter 334 includes, for example, a display for displaying video or auxiliary video based on video data as part or all of the presentation data. The information presenter 334 may also include a speaker for playing audio or auxiliary audio based on audio data as part or all of the presentation data. The information presenter 334 may also include any one or a combination of a light emitter, a vibrator, and a speaker for presenting notification information in response to a drive signal.

[0051] When the input / output device 200 has a function as a presentation device, it includes components corresponding to the presentation information processing unit 332 and the information presentation unit 334. For example, a tablet PC used as the input / output device 200 may include a tuner that can receive presentation information distributed by television broadcasting. The tablet PC may include, together with or instead of the tuner, one or both of a light emitter and a vibrator for presenting notification information.

[0052] Next, a display example of the display screen displayed on the display unit 234 of the input / output device 200 will be described. FIG. 3 is a diagram showing a first display example of the display screen according to this embodiment. In the example of FIG. 3, four device icons and one user icon are arranged in different positions. Characters indicating the name of the corresponding presented device (e.g., "smart speaker") are displayed near each device icon. The character "you" is displayed near the user icon. Display of these characters may be omitted. A circle centered on the position of the user icon is an example of a range designation shape that occupies the selection range. The inside of the range designation shape is filled with a lighter tone in a manner different from the outside.

[0053] In the example of FIG. 3, the TV is a device presenting information (e.g., video and audio of a broadcast program), and the other devices are not presenting information. The TV device icon is painted in a darker gradation to have a different display mode from the other device icons. The user icon is also painted in a darker gradation than its surroundings. This allows the user to immediately understand that the TV corresponding to the device icon displayed in the display area of ​​the range designation shape displayed based on the position of the user icon is presenting information, and that the other devices are not presenting information. In the following description, a display mode corresponding to the presentation of presented information, such as the TV device icon, may be referred to as a presenting display. A presenting display corresponds to an active display indicating that presented information is being presented.

[0054] FIG. 5 is a diagram illustrating an example of device management data according to this embodiment. The device management data illustrated in FIG. 5 indicates the operating status of the presentation device displayed on the display screen illustrated in FIG. 3 and the display position of the device icon or user icon. The device management data includes, for each presentation device, a type, presentable information, detection information, operating status information, and a display position, which are associated with each other. The presentation device field contains text to be displayed in association with each presentation icon and user icon. The type indicates the type of information described in the presentation device field, whether it is a presentation device or a user. The presentable information field contains presentable types of presentation information. The detection information indicates whether the device is detected by the control device 100. A circle indicates a detected device. Although not shown in FIG. 5, an x ​​indicates an undetected device. The operating status information indicates whether information is presented. A circle indicates that information is presented. An x indicates that information is not presented. The display position indicates the two-dimensional coordinates at which the device icon or user icon is displayed on the display screen. Note that a - indicates no match.

[0055] 6 shows an example of user-selected range data indicating the display range of a range specification shape according to this embodiment. The example in FIG. 6 shows the two-dimensional coordinates of the center of a circle used as the range specification shape and the radius of the circle. The shape of the range specification shape is not necessarily limited to a circle, and can be any shape and size that allows multiple device icons and one user icon to be displayed within the display range without overlapping each other.

[0056] Here, let us assume that the device icon representing a smartphone is moved by dragging within the selected area of ​​the graphic on the display screen exemplified in Fig. 3. Fig. 5 shows an example of the display screen that is displayed in this case. In the example of Fig. 5, information is presented for the smartphone as well as the TV. The display mode of the smartphone device icon is also set to "presenting" and is filled in with a darker gradation than the device icons for the tablet PC and smart speaker, which are not presenting information. The device management data illustrated in Fig. 7 indicates the operation status information of the presented device, the display position of the device icon, and the display position of the user device displayed on the display screen illustrated in Fig. 4. That is, it differs from the example in Fig. 5 in that a circle is written in the item for the operation status information of the smartphone, and the coordinates of the device icon after it has been moved are written in the item for the display position.

[0057] The data format of the device management data exemplified in Figures 5 and 7 is a table, but is not limited to this. Any format that expresses the relationships between information elements for each presentation device is applicable. For example, the data format of the device management data may be the Resource Description Framework (RDF) format, as exemplified in Figure 8. RDF is a recommendation of the World Wide Web Consortium (W3C). In RDF, the relationships between resources (information elements) are expressed as a combination of three elements (triples): subject, predicate, and object. The subject corresponds to the resource being described. The predicate indicates the characteristics of the subject or the relationship between the subject and object. The object indicates something related to the subject or the value of the predicate. A set of triples can be expressed using a node indicating the subject, a node indicating the object, and an edge (arrow) indicating the relationship between the subject and object.

[0058] The device management data illustrated in FIG. 8 shows substantially the same information as the tabular device management data illustrated in FIG. 7. More specifically, each node for the presentation devices ("smart speaker," "tablet PC," "TV," and "smartphone") is associated with the coordinates of its individual display position, some or all of the corresponding options for presentable information ("video," "audio," and "notification"), and either "○" or "×" (a circle or an error) for operation status information. The node for the user ("you") is associated with the coordinates of the display position, the coordinates of the center, and the corresponding option for type ("user"). The node for the selection range shape (user selection range) is associated with the coordinates of the center and the radius. According to RDF, the relationship between element information is expressed exactly by associating different element information (subject) with element information (predicate).

[0059] Next, an example of device selection processing in the device management system 1 according to this embodiment will be described. Fig. 9 is a sequence diagram showing an example of device selection processing according to this embodiment. However, as exemplified in Figs. 3 and 4, it is assumed that the device icon of a presentation device that is not presenting information is moved by a drag operation into the selection range of the range designation figure. (Step S102) The device management unit 124 of the control device 100 detects the active presentation devices connected to the control device itself. (Step S104) The device management unit 124 stores detection information indicating the detection of a newly detected presentation device in the storage unit 122 of its own device in association with the presentation device. The device management unit 124 transmits the detection information to the input / output device 200. The storage unit 222 of the input / output device 200 stores the detection information received from the control device 100 in association with the presentation device. This updates the device management data.

[0060] (Step S106) The display processing unit 228 of the input / output device 200 reads out the updated device management data and user selection range data from the storage unit 222. (Step S108) The display processing unit 228 identifies the detected presented devices based on the device management data, places device icons in different positions for each presented device, and generates a display screen showing the user icon and selection range graphic in the position indicated by the device management data. (Step S110) The display processing unit 228 outputs display data indicating the generated display screen to the display unit 234. The display unit 234 displays a display screen based on the display data input from the display processing unit 228.

[0061] (Step S114) Operation unit 236 accepts a drag operation by the user and outputs operation information indicating the position indicated by the drag operation to display processing unit 228. Display processing unit 228 identifies the display position after the operation indicated by the position indicated by the operation information input from operation unit 236 and the device icon as the operation icon. (Step S116) The display processing unit 238 outputs to the control device 100 operation information indicating the coordinates of the display position of the identified operation icon. (Step S118) The display processing unit 238 updates the display screen so that the operation icon is moved to the specified display position, thereby reflecting the drag operation. (Step S120) Display processing unit 238 outputs display data showing the updated display screen to display unit 234. Display unit 234 displays the updated display screen.

[0062] (Step S122) The device management unit 124 of the control device 100 detects an operation icon that has moved from outside the selection range into the selection range by referring to the user selection range data, based on the operation information input from the input / output device 200. The device management unit 124 transmits a control signal indicating the start of information presentation to the presentation device corresponding to the detected operation icon (presentation device control). (Step S124) When an acknowledgement to the control signal is input, the device management unit 124 stores the operation status information related to the presentation device, which is operation status information indicating that information is being presented, and display position information indicating the display position after movement, in association with the identified presentation device in the storage unit 222. This updates the device management data. (Step S126) The device management unit 124 outputs the updated device management data to the input / output device 200.

[0063] (Step S128) The display processing unit 228 of the input / output device 200 receives device management data from the control device 100 and stores the received device management data in the storage unit 222. The display processing unit 228 identifies a presentation device corresponding to the identified operation icon and identifies operation status information of the identified presentation device by referring to the device management data. The display processing unit 228 reflects on the display screen, as the display mode of the operation icon corresponding to the identified presentation device, a display mode corresponding to "information presented" as the operation status indicated by the identified operation status information. (Step S130) The display processing unit 228 outputs display data showing the updated display screen to the display unit 234. The display unit 234 displays a display screen showing the moved operation icon with the display mode changed.

[0064] As described above, the device management system 1 according to this embodiment includes a display processing unit 228 and a device management unit 124. The display processing unit 228 displays a display screen on the display unit 234, in which device icons as display elements representing individual presented devices and user icons as display elements representing users are arranged in different positions, and a range designation figure is arranged in an area including the user icons, and changes the position of the device icons in response to an operation. The device management unit 124 selects, as a target device, a device corresponding to a device icon entering or exiting a selection range occupying the range designation figure, and controls the operating state of the target device. The display processing unit 228 determines the display mode of the device icon of the target device to a display mode corresponding to the operating state of the target device. According to this configuration, the user can intuitively control the operating state of the device corresponding to the device icon by operating the device icon to enter or exit the range specification figure.

[0065] <Second embodiment> Next, a second embodiment will be described. The following description will mainly focus on the differences from the first embodiment, and unless otherwise specified, common reference numerals will be used to designate processes and configurations common to the first embodiment, and the description thereof will be incorporated herein. FIG. 10 is an explanatory diagram showing an overview of a device management system 1 according to the second embodiment. A notebook PC (sometimes referred to as a "notebook PC" in this application) corresponds to the input / output device 200. An apparatus corresponding to the control device 100 is not shown in the drawing.

[0066] In the example of FIG. 10, the management area is divided into multiple installation locations (rooms), and the location of presentation devices is managed for each installation location. As presentation devices, a bedroom TV and an IoT light are installed in the bedroom, and a TV, a smart speaker, an additional speaker, a headset, a smartphone, and a laptop PC are installed in the living room. The display screen displayed on the input / output device 200 displays a device icon for each presentation device and four edges (also called "links"). A device icon is connected to each end of each edge. Each edge indicates that the presentation devices represented by the two connected device icons can be connected to input or output presentation information. In the example of FIG. 10, the TV and the additional speaker, the TV and the headset, the TV and the smartphone, and the smartphone and the headset can be connected. This embodiment differs from the first embodiment in that user icons and range selection shapes are not displayed on the display screen.

[0067] In this embodiment, the display processing unit 228 of the input / output device 200 displays a range specification graphic in a selection range specified in response to an operation. The device management unit 124 of the control device 100 selects a presentation device corresponding to a device icon located within the specified selection range as a target device whose operating state is to be controlled. When an installation location is specified in response to an operation, the display processing unit 228 displays the device icon of the presentation device installed in the specified installation location in a display mode different from the device icons of the other presentation devices, and determines it as a candidate target device. The display processing unit 228 determines the other presentation devices as non-candidate target devices. When a type of information is specified in response to an operation, the display processing unit 228 identifies the device as a presentable device that can present the specified type of information, and displays the device icon of the identified presentable device in a display mode different from the device icons of the other presentation devices, and determines it as a candidate target device. The display processing unit 228 determines the other presentation devices as non-candidate target devices.

[0068] Next, an example of the functional configuration of the device management system 1 according to this embodiment will be described. FIG. 11 is a schematic block diagram showing an example of the functional configuration of the device management system 1 according to this embodiment. The device management system 1 according to this embodiment includes a control device 100 and an input / output device 200. In the illustrated example, the control device 100 is connected to the input / output device 200 and the presentation device 300 installed in the management area via the Internet and a router 400. The control device 100 may be configured as a server device such as a web server or a database server. The device management unit 124 of the control device 100 can detect the input / output device 200 and the presentation device 300 that are in operation and detected in the management area based on the detected device information received from the router 400.

[0069] The router 400 is connected to the Internet and a LAN, and relays data transmitted between the control device 100 connected to the Internet and the input / output device 200 or the presentation device 300 connected to the LAN. The router 400 also relays data transmitted between the input / output device 200 and the presentation device 300. The router 400 does not necessarily have to be a device independent of the input / output device 200 or the presentation device 300. At least one of the input / output device 200 and the presentation device 300 may have the same functions as the router 400, and the router 400 may be omitted.

[0070] Next, an example of device management data according to this embodiment will be described. FIG. 12 shows a first example of device management data according to this embodiment. The device management data according to this embodiment includes, for each presentation device, presentable information, detection information, operating status information, connectable device information, installation location information, and display position, and associates these information. The operating status information indicates whether or not each type of presentation information is presented. A circle indicates that the information is presented, and no circle indicates that the information is not presented. A dash (-) indicates that the information is not applicable. The connectable device information indicates other devices that can be connected. The example of FIG. 12 indicates that none of the presentation devices are presenting distribution information. The connectable device information describes information about presentation devices to which the device itself receives presentation information or presentation devices that provide presentation information to the device itself. The display processing unit 228 of the input / output device 200 references the presentable information, identifies connectable devices indicated in the connectable device information of the detected presentation device, and adds an edge connecting the device icon corresponding to the detected presentation device and the device icon corresponding to the identified connectable device to the display screen.

[0071] The installation location information indicates the installation location where the corresponding presentation device is installed. However, in the example shown in FIG. 12 , for a presentation device that is not fixed in its installation location but is movable, all possible locations are described as installation location information. Specifically, for a smartphone, a laptop PC, and a headset, the living room and the bedroom are described. The installation location information item may include information on the installation location of the router 400 directly connected to the presentation device or the installation location of another gateway device (e.g., an access point) directly connected to the presentation device via the router 400. In this case, all possible locations do not need to be described in advance as installation location information. Therefore, the router 400 may include information on the installation location of the router 400 or a gateway device directly connected to each presentation device in device detection information and transmit the information to the control device 100. The device management unit 124 of the control device 100 associates information indicating the detection location included in the device detection information received from the router 400 with the presentation device and describes it as installation location information.

[0072] Next, an example of user-specified range data according to this embodiment will be described. FIG. 13 shows an example of user-selected range data according to this embodiment. FIG. 13 illustrates an example in which the range specification figure is a polygon with three or more vertices. The user-selected range data indicates the display positions of each vertex of the polygon. The display processing unit 228 of the input / output device 200 determines the display positions of each vertex based on operation information input from the operation unit 236, stores user-specified range data indicating the display positions determined for each vertex in the storage unit 222, and transmits the data to the control device 100. The display processing unit 228 displays a display screen including the range specification figure on the display unit 234 by sequentially adding edges connecting two vertices to the display screen. When receiving device management data in response to the user-specified range data, the display processing unit 228 compares the received device management data with the immediately preceding device management data to identify the presentation device whose operating status information has been updated and the updated operating status information. The display processing unit 228 changes the display mode of the device icon for the identified presentation device to a display mode corresponding to the operating state (whether or not information is presented) indicated in the identified operating state information. The display processing unit 228 erases the range selection figure from the display screen regardless of whether the operating state information in the received device management data has been updated. The range selection figure is not limited to a polygon, but may be a curve that passes through multiple positions indicated by the operation information. Examples of a predetermined function that provides coordinate points forming such a curve include a Bernstein polynomial that provides a Bezier curve and a spline function that provides a spline curve. The display processing unit 228 may identify positions on the display screen indicated by the operation information as control points and determine parameters of the function to provide a curve that passes through the identified control points.

[0073] Next, display examples of the display screen according to this embodiment will be described. FIG. 14 is a diagram illustrating a first display example of the display screen according to this embodiment. In the example of FIG. 14, in addition to nine device icons, a presentation information selection button is arranged near the upper right corner of the display screen, and a location selection button is arranged near the lower right corner of the display screen. The presentation information selection button is a button for selecting a type of presentation information by pressing it. The word "Notification" displayed on the presentation information selection button indicates that notification information is selected as the type of presentation information. The display processing unit 228 switches the type of presentation information in a predetermined order each time it detects a press of the presentation information selection button based on operation information input from the operation unit 236. The display processing unit 228 can refer to the device management data and determine whether the type of presentation information selected is a type of presentation information that can be presented by the presentation device based on whether the type of presentation information is included in the types of presentation information indicated in the presentable information for each presentation device. The display processing unit 228 may display the display mode of a device icon corresponding to a presentation device that can present the selected type of presentation information more clearly than the display mode of device icons corresponding to other presentation devices. In the following description, the display mode of a device icon that accepts selection based on operation information, such as device icons corresponding to other presentation devices, may be referred to as a presentable display. The presentable display corresponds to an active display indicating that selection is possible according to operation information. The display processing unit 228 may accept selection of a device icon corresponding to a presentation device that can present the selected type of presentation information, but may not accept selection of device icons corresponding to other presentation devices. In the following description, the display mode of a device icon that does not accept selection based on operation information, such as device icons corresponding to other presentation devices, may be referred to as a non-presentable display. In the example shown in FIG. 14, all nine presentation devices can present notification information, so the display mode of the corresponding device icons is a presentable display.

[0074] The location selection button is a button for selecting the location where the presentation device is installed by pressing it. The words "ROOM SELECTION" displayed on the location selection button inform the user that the room (installation location) can be selected by pressing it. In the example of FIG. 14, no location is specified, and any presentation device detected in the management area can be selected. The functions and processes related to the location selection button will be described later.

[0075] FIG. 15 is a diagram showing a second display example of a display screen according to this embodiment. The example of FIG. 15 shows a state in which video is selected as the type of presentation information by pressing the presentation information selection button. In this state, the word "video" is displayed on the presentation information selection button. The display modes of the device icons corresponding to the smartphone, bedroom TV, TV, and laptop PC are displayed as presentable devices, and are ready to accept operations. The outlines of these device icons are shown with solid lines. This display indicates that the smartphone, bedroom TV, TV, and laptop PC can be selected as candidate presentation devices capable of displaying video. In contrast, the display modes of the device icons corresponding to the headset and additional speakers are displayed as not presentable devices, and are not ready to accept selection. The outlines of these device icons are shown with dashed lines. This display indicates that the headset and additional speakers cannot be selected as candidate presentation devices capable of displaying video. Therefore, when switching the characters indicating the type of presentation information (i.e., "video", etc.) on the presentation information selection button in response to pressing, the display processing unit 228 may switch the display mode of the presentation information selection button to a display mode corresponding to the type of presentation information.

[0076] FIG. 16 is a diagram illustrating a third display example of a display screen according to this embodiment. In the example of FIG. 16, a pointer pt and a trajectory tr are further displayed on the display screen illustrated in FIG. 15. In the example of FIG. 16, the pointer pt is used to designate a polygon as a selected designated shape in response to an operation. For example, when detecting a press on a position on the display screen, the display processing unit 228 positions the pointer pt at that position. The display processing unit 228 moves the display position of the pointer pt to a position designated in response to a drag operation. The display processing unit 228 displays a trajectory tr of positions designated in response to a drag operation on the display screen. For example, the display processing unit 228 can determine positions designated at predetermined time intervals from the start of a drag operation to the end of the drag operation as the positions of the vertices of a polygon, and generate data indicating a sequence of positions determined in that order as user range designation data. When the drag operation ends, the display processing unit 228 may stop displaying the pointer pt and the trajectory tr on the display screen. The display processing unit 228 transmits the generated user range designation data to the control device 100. The device management unit 124 of the control device 100 refers to the device management data and identifies the TV as the presentation device corresponding to the device icon whose display position is included in the selection range surrounded by the locus tr based on the user range designation data received from the input / output device 200.

[0077] The device management unit 124 transmits a control signal to a TV that is not currently displaying video in order to instruct it to display the video (presentation device control). When an acknowledgement to the control signal is input from the TV, the device management unit 124 associates the TV with operating status information indicating that information for the video is being presented and stores the information in the storage unit 122. The input of the acknowledgement confirms that the TV is displaying the video. The device management unit 124 transmits device management data including the updated operating status information to the input / output device 200. The display processing unit 228 of the input / output device 200 detects an update to the operating status information from the device management data received from the control device 100 and changes the display mode of the TV's device icon to one indicating that video is being presented. In the example shown in FIG. 16, the display mode of the TV's device icon is set to a presentation-in-progress display, and is displayed with a darker gradation than the device icons of the other presentation devices. The display processing unit 228 may determine the display mode for the presentation-in-progress display depending on the type of presentation information. Different display modes depending on the type of presentation information are preset in the display processing unit 228. The display mode used to distinguish the presented information may be any one of color tones, fill patterns, other patterns, etc., or a combination thereof. A fill pattern refers to a pattern applied inside the outline of a figure, such as a hatched pattern, a grid, or diagonal lines. The display processing unit 228 may use the same display mode for the currently presented display and the presented information selection button. For example, in the example of FIG. 16, the presented information selection button indicating "audio" and the TV device icon may be filled in with the same color. This allows a user who visually views the device icon to immediately recognize the type of information presented by the presentation device.

[0078] FIG. 17 is a diagram illustrating a fourth display example of a display screen according to this embodiment. The example in FIG. 17 illustrates a state in which audio is selected as the type of presentation information by pressing a presentation information selection button. In this state, the word “audio” appears on the presentation information selection button. The device icons corresponding to the smartphone, bedroom TV, headset, TV, laptop PC, additional speaker, and smart speaker are displayed as presentable and selectable. A pointer pt is also displayed on the display screen. In the example in FIG. 17, the pointer pt is also used as a range designation figure to indicate the position of its own tip. For example, when the display processing unit 228 detects a press on any position on the display screen, it places the pointer pt at that position. The pointer pt can be moved to a position indicated by a drag operation. The display processing unit 228 displays the pointer pt at a position indicated by a drag operation without displaying a trajectory. For example, when the tip of the pointer pt is included in the display area of ​​one of the device icons and a click operation is detected, the display processing unit 228 transmits user range designation data indicating the pressed position (selected range) where the press was detected to the control device 100. The pointer pt can also be considered as one aspect of a range designation graphic corresponding to a click operation. The device management unit 124 of the control device 100 refers to the device management data and identifies the additional speaker as the presentation device corresponding to the device icon whose display area includes the pressed position indicated by the user range designation data received from the input / output device 200.

[0079] The device management unit 124 transmits a control signal to an additional speaker that is not currently playing auxiliary audio, instructing it to play auxiliary audio (presentation device control). When an acknowledgement to the control signal is input from the additional speaker, the device management unit 124 associates operation status information indicating that information is being presented for the auxiliary audio with the additional speaker and stores it in the storage unit 122. However, if the additional speaker does not have the function of acquiring auxiliary audio, which is distribution information, the device management unit 124 refers to the device management data and identifies a TV as a device that can acquire auxiliary audio and can be connected to the additional speaker. The device management unit 124 transmits a control signal to the identified device, the TV, instructing it to acquire auxiliary audio and transmit it to the additional speaker (presentation device control). When an acknowledgement to the control signal is further input from the TV, the device management unit 124 stores operation status information indicating that information is being presented for the auxiliary audio in the storage unit 122 in association with the additional speaker.

[0080] The device management unit 124 transmits device management data including the updated operating status information to the input / output device 200. The display processing unit 228 of the input / output device 200 detects the update of the operating status information from the device management data received from the control device 100, and changes the display mode of the device icon of the additional speaker to indicate that auxiliary audio is being presented. In the example of Fig. 17, the display mode of the device icon of the additional speaker is also changed to a presentation in progress display, and is displayed together with the device icon of the TV in a darker gradation than the device icons of the other presented devices.

[0081] FIG. 18 is a diagram showing a fifth display example of a display screen according to this embodiment. The example of FIG. 18 shows a state in which auxiliary video has been selected as the type of presentation information by pressing the presentation information selection button. In this state, the words "auxiliary video" are displayed above the presentation information selection button. The display modes of the device icons corresponding to the smartphone, bedroom TV, TV, and laptop PC are displayed as presentable and selectable. The display modes of the device icons corresponding to the headset and additional speakers are displayed as not presentable and cannot be selected. The outlines of device icons that are not presentable are shown with dashed lines. The dark fill in the presentation display of the device icon for the additional speaker indicates that information is being presented for any type of distribution information.

[0082] FIG. 18 illustrates a rectangle as another example of a range designation shape created by a drag operation. The display processing unit 228 of the input / output device 200 defines a rectangle having a diagonal line connecting the start point and end point of a single drag operation as the range designation shape. The display processing unit 228 causes the display unit 234 to display a display screen in which the defined rectangle is superimposed. The display processing unit 228 generates user range designation data, which indicates the coordinates of the vertices of the defined rectangle in the order in which each side is connected. The display processing unit 228 transmits the generated user range designation data to the control device 100. The device management unit 124 of the control device 100 refers to the device management data and identifies a device icon corresponding to a smartphone as a device icon whose display position is included in the selection range enclosed by the rectangle based on the user range designation data received from the input / output device 200.

[0083] The device management unit 124 transmits a control signal to a smartphone that is not currently displaying auxiliary video, instructing it to play the auxiliary video (presentation device control). When an acknowledgment response to the control signal is input from the smartphone, the device management unit 124 stores operation status information indicating that information is being presented for the auxiliary video in association with the additional speaker in the storage unit 122. However, if the smartphone does not have the function to acquire the auxiliary video, which is distributed information, the device management unit 124 refers to the device management data and identifies a TV as a device that can acquire the auxiliary video and can be connected to the smartphone. The device management unit 124 transmits a control signal to the identified TV, instructing it to acquire the auxiliary video and transmit it to the smartphone (presentation device control). When an acknowledgment response to the control signal is input from the TV, the device management unit 124 stores operation status information indicating that information is being presented for the auxiliary video in association with the smartphone in the storage unit 122.

[0084] The device management unit 124 transmits device management data including the updated operating status information to the input / output device 200. The display processing unit 228 of the input / output device 200 detects the update of the operating status information from the device management data received from the control device 100, and changes the display mode of the smartphone's device icon to one indicating that auxiliary video is being presented. In the example of Fig. 18, the display mode of the smartphone's device icon is set to a presentation in progress display, and is displayed together with the device icon of the TV in a darker shade than the device icons of the other presented devices.

[0085] FIG. 19 is a diagram showing a sixth display example of a display screen according to this embodiment. The example in FIG. 19 shows a state in which auxiliary audio has been selected as the type of presentation information by pressing the presentation information selection button. In this state, the words "auxiliary audio" are displayed above the presentation information selection button. The display modes of the device icons corresponding to the smartphone, bedroom TV, headset, TV, additional speaker, and laptop PC are displayed as presentable and selectable. Among these, the display mode of the headset device icon is displayed as a presenting display, and is distinguished by being darkly filled in. This indicates that the headset is playing auxiliary audio.

[0086] Also, a pull-down menu pd is displayed adjacent to and above the location selection button (room selection) at the bottom right of the display screen. The pull-down menu pd is displayed in response to pressing the location selection button, and lists "living room" and "bedroom" as installation location options, with either option selected by using the pointer pt.

[0087] When the location selection button is selected based on the input operation information, the display processing unit 228 of the input / output device 200 refers to the device management data, identifies the installation locations indicated in the installation location information as options, and arranges a pull-down menu with the identified options as its upper limit adjacent to the location selection button. The display processing unit 228 makes the display position of the pointer pt movable to a position indicated by the operation information. When a position within the display area of ​​one of the options is indicated and a click operation is detected, the display processing unit 228 selects the installation location as the option. Thereafter, the display processing unit 228 refers to the device management data and identifies presentation devices installed at the selected installation location. The display processing unit 228 maintains the display mode of the device icon corresponding to the identified presentation device on the display screen as presentable, and changes the display mode of device icons corresponding to the other presentation devices to unpresentable. The display processing unit 228 transmits presentation information type information indicating the type of the selected presentation information and installation location information indicating the installation location to the control device 100. The device management unit 124 of the control device 100 receives the presentation information type information and installation location information from the input / output device 200. The device management unit 124 refers to the device management data and identifies a headset, a notebook PC, and a smart speaker as presentation devices that are capable of presenting presentation information of the type indicated in the presentation information type information and that are installed or available at the installation location indicated in the installation location information but that do not present any type of presentation information.

[0088] The device management unit 124 transmits control signals to the headset, the laptop PC, and the smart speaker to instruct them to play auxiliary audio (presentation device control). When an acknowledgement response to the control signal is input from the headset, the laptop PC, and the smartphone, the device management unit 124 stores in the storage unit 122 operation status information indicating that information is being presented for the auxiliary audio, in association with the headset, the laptop PC, and the smart speaker.

[0089] If any one or any combination of the headset, laptop PC, and smart speaker does not have the function of acquiring auxiliary audio, the device management unit 124 transmits a control signal to an auxiliary audio acquisition device (e.g., a TV) that is a presentation device that has the function of acquiring auxiliary audio, to instruct it to acquire the auxiliary audio and to transmit the auxiliary audio to an auxiliary audio presentation device (e.g., a headset and a smart speaker) that is a presentation device that does not have the function of acquiring auxiliary audio (presentation device control).When an acknowledgement response to the control signal is input from the auxiliary audio acquisition device and the auxiliary audio presentation device, the device management unit 124 stores operation status information indicating that information is presented for the auxiliary audio in the storage unit 122 in association with the auxiliary audio presentation device.

[0090] The device management unit 124 transmits device management data including the updated operating status information to the input / output device 200. The display processing unit 228 of the input / output device 200 detects the update of the operating status information from the device management data received from the control device 100, and changes the display mode of the device icons of the headset, laptop PC, and smartphone to indicate that auxiliary video is being presented. In the example shown in Fig. 19, the display mode of the device icon of the smartphone is set to a presentation in progress display, and it is displayed together with the device icon of the TV in a darker shade than the device icons of the other presented devices.

[0091] In this state, whether or not auxiliary audio is played can be controlled by selecting the display area of ​​the device icon on the headset, laptop, smartphone, TV in the living room, smart speaker that can be moved to the living room, or smartphone. On the other hand, selecting the display area of ​​an additional speaker not in the living room or a device on the bedroom TV is not accepted.

[0092] Fig. 20 is a diagram showing a seventh display example of the display screen according to this embodiment. The example of Fig. 20 shows a state in which auxiliary audio is selected as the type of presentation information by pressing the presentation information selection button. In this state, the position of the laptop PC device icon indicated by the pointer pt is selected as the selection range on the display screen shown in Fig. 19, and playback of auxiliary audio is stopped. The device icon of the laptop PC is no longer filled in as it is being presented, and the display returns to its original state.

[0093] The display processing unit 228 updates the display screen by moving the pointer pt to the position indicated by the operation information. When the display processing unit 228 detects a click operation from the operation information, it transmits user range designation data indicating the position to the control device 100. The device management unit 124 of the control device 100 refers to the device management data and identifies a presentation device corresponding to a device icon whose display area includes the position indicated by the user range designation data received from the input / output device 200. It transmits a control signal to the laptop PC as the presentation device presenting the auxiliary audio, instructing it to stop playing the auxiliary audio. When an acknowledgement response to the control signal is input from the laptop PC, the device management unit 124 stores operation status information indicating that no information is presented for the auxiliary audio in the storage unit 122 in association with the laptop PC.

[0094] The device management unit 124 transmits device management data including the updated operating status information to the input / output device 200. The display processing unit 228 of the input / output device 200 detects an update to the operating status information from the device management data received from the control device 100, and changes the display mode of the device icon of the notebook PC for which the update was detected to one indicating that no auxiliary video is being presented.

[0095] As illustrated in Figures 16-18, the device management unit 124 and the display processing unit 228 can present the selected type of presentation information, and may select one or more of the presentation devices available at the selected location based on the selection range in which the position of the device icon is initially indicated based on the operation information. The display processing unit 228 does not change the display mode of any device icon simply by moving the display position of the device icon by a drag operation. The device management unit 124 does not change the operating state of any presented device unless there is a change in the positional relationship (i.e., inclusion relationship, presence or absence of overlap) between the selection range and any device icon. In the example of FIG. 21, the position of the smart speaker device icon, together with the pointer pt, has been moved to the upper right side more than in the example of FIG. 20. Because the selection range indicating the position of the smart speaker device icon is not specified by the operation information, playback of the auxiliary audio has not started.

[0096] In this situation, when the display processing unit 228 detects a drag operation on any of the device icons based on operation information input from the operation unit 236, it places the device icon at the position indicated by the drag operation. The display processing unit 228 transmits display position information indicating the indicated position for the device icon to the control device 100. The device management unit 124 of the control device 100 updates the device management data by storing the display position information received from the input / output device 200 in the storage unit 122 in association with the presentation device corresponding to the device icon. During this process, the device management unit 124 does not control whether or not the presentation information needs to be presented.

[0097] FIG. 22 is a diagram showing a second example of device management data according to this embodiment. The example of FIG. 22 differs from the example of FIG. 12 in that it shows video display by a TV, auxiliary video display by a smartphone, audio playback by an additional speaker, and auxiliary audio playback by a headset, as well as in the display position of the smart speaker's device icon. The display of video by a TV, the display of auxiliary video by a smartphone, audio playback by an additional speaker, and auxiliary audio playback by a headset are instructed by operations on the display screens exemplified in FIG. 16, FIG. 18, FIG. 17, and FIG. 19, respectively. The display position of the smart speaker's device icon is instructed by operations on the display screen exemplified in FIG. 21.

[0098] The display processing unit 228 may represent the input / output relationship between a pair of connectable devices using the direction of an arrow placed on the pair. That is, a pair of a presentation device that provides presentation information and a presentation device that receives the presentation information may be represented by an arrow (single-sided arrow) starting from the device icon of the source of the presentation information and ending (leading end) at the device icon of the destination of the presentation information. An arrow indicating a pair of devices between which input and output of presentation information is performed may be displayed in a clearer manner than an arrow indicating a pair of devices between which input and output of presentation information is not performed. In the example of FIG. 23 , the display processing unit 228 places a solid single-sided arrow starting from the device icon of the TV and ending at the device icon of the additional speaker on the display screen. This indicates that audio acquired by the TV is output to the additional speaker. The display processing unit 228 also places a dashed arrow starting from the device icon of the TV and ending at the device icon of the smart speaker, a dashed arrow starting from the device icon of the TV and ending at the device icon of the headset, and a dashed arrow starting from the device icon of the smartphone and ending at the device icon of the headset on the display screen. This indicates that audio can be input and output from the TV to the smart speaker, from the TV to the headset, and from the smartphone to the headset, but that no input or output is actually taking place.

[0099] Therefore, the connectable device information included in the device management data may include input / output direction information indicating whether another presentation device to be connected will be the source or destination of the presentation information. In the example of FIG. 24, the symbols (I) and (O) attached to each connectable device indicate the source (input source) and destination (output destination), respectively. For example, a smartphone, an additional speaker, and a headset, which are connectable devices to a TV, are all destinations of the presentation information from the TV. A TV, which is a connectable device to an additional speaker, is a source of the presentation information for the additional speaker. The display processing unit 228 of the input / output device 200 can refer to the device management data and specify that a connectable device for a presentation device designated by operation information will be the destination of the presentation information when the connectable device for the presentation device designated by user range designation data is the source of the presentation information. The device management unit 124 of the control device 100 can also refer to the device management data and specify that a presentation device will be the destination of the presentation information when the connectable device for a presentation device designated by user range designation data is the source of the presentation information. In this case, the device management unit 124 may transmit a control signal to a connectable device that is a source of the presentation information, instructing the connectable device to acquire the presentation information and transmit the presentation information to the presentation device. Thus, the presentation device that cannot acquire the presentation information on its own is instructed to provide the presentation information.

[0100] Similarly, when a connectable device for a presentation device specified by the operation information is to be the recipient of the presentation information, the display processing unit 228 can refer to the device management data and determine that the presentation device will be the source of the presentation information. In this case, the display processing unit 228 can determine that the presentation device can acquire the presentation information on its own and therefore does not need to be connected to the connectable device. The device management unit 124 of the control device 100 can also refer to the device management data and determine that when a connectable device for a presentation device specified by the user range designation data is to be the recipient of the presentation information, the presentation device can acquire the presentation information on its own and therefore does not need to be connected to the connectable device.

[0101] Next, an example of device selection processing in the device management system 1 according to this embodiment will be described. Fig. 25 is a sequence diagram showing a first example of device selection processing according to this embodiment. Fig. 25 illustrates an example in which, after device detection in the management area, a video is selected as presentable information, a path indicated by a drag operation is designated as a range designation figure, and a TV is selected as a presentation device for presenting the video (see Fig. 16).

[0102] (Step S202) The router 400 detects an active presentation device connected to the router 400 in its management area. The device management unit 124 of the control device 100 receives, from the router 400, detected device information indicating the detected presentation device. (Step S204) The device management unit 124 stores detection information indicating the detection of a newly detected presentation device in the storage unit 122 of its own device in association with the presentation device. The device management unit 124 transmits the detection information to the input / output device 200. The storage unit 222 of the input / output device 200 stores the detection information received from the control device 100 in association with the presentation device, thereby updating the device management data.

[0103] (Step S206) The display processing unit 228 of the input / output device 200 reads the updated device management data from the storage unit 222. (Step S208) The display processing unit 228 identifies the detected presentation devices based on the device management data, arranges device icons in different positions for each presentation device, and further arranges presentation information selection buttons and location selection buttons to generate a display screen. (Step S210) Display processing unit 228 outputs display data indicating the generated display screen to display unit 234. On display unit 234, a display screen based on the display data input from display processing unit 228 is displayed.

[0104] (Step S212) The display processing unit 228 detects that a presentation information selection button on the display screen has been pressed based on operation information input from the operation unit 236, and selects video as the type of presentable information. (Step S214) The display processing unit 228 refers to the device management data and identifies a presentation device capable of displaying video of the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device as presentable, and sets the display mode of device icons corresponding to other presentation devices as not presentable. The display screen transitions to a device selection screen on which a presentation device capable of displaying video as presentation information can be selected. (Step S216) The display processing unit 228 outputs to the display unit 234 display data indicating the transitioned display screen.

[0105] (Step S218) The display processing unit 228 identifies the trajectory of the drag operation as a range designation graphic based on the operation information input from the operation unit 236 (range designation operation). (Step S220) The display processing unit 228 updates the display screen by reflecting the designated trajectory as a range designation graphic. (Step S222) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen. (Step S224) The display processing unit 228 transmits to the control device 100 user range designation data indicating the range designation graphic. (Step S226) The device management unit 124 of the control device 100 selects, as a device to be controlled, a presented device corresponding to a device icon included in the selection range of the range designation figure indicated by the user range designation data received from the input / output device 200.

[0106] (Step S228) The device management unit 124 transmits a control signal instructing the selected presentation device to display the video. (Step S230) When an acknowledgement is received in response to the control signal, the device management unit 124 updates the device management data by storing, in the storage unit 122, operation status information indicating that information is being presented for the video in association with the selected presentation device. (Step S232) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input / output device 200. (Step S234) The display processing unit 228 of the input / output device 200 reflects the updated operating status on the display screen by changing the display mode of the device icon of the selected presentation device based on the updated operating status information in the device management data received from the control device 100 to a display mode corresponding to the operating status information (display of video) of the presentation device. (Step S236) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen.

[0107] Fig. 26 is a sequence diagram showing a second example of the device selection process according to this embodiment. Fig. 26 shows an example in which audio is selected as presentable information, a pointer that can be dragged to specify a position is designated as a range designation figure, and an additional speaker is selected as a presentation device for presenting the audio (see Fig. 17). However, this is premised on the fact that the additional speaker can acquire the distributed audio voluntarily.

[0108] (Step S242) The display processing unit 228 detects that a presentation information selection button on the display screen has been pressed based on operation information input from the operation unit 236, and selects audio as the type of presentable information. (Step S244) The display processing unit 228 refers to the device management data and identifies a presentation device that can play audio as the selected type. The display processing unit 228 displays the device icon corresponding to the identified presentation device as presentable, and displays device icons corresponding to other presentation devices as not presentable. The display screen transitions to a device selection screen on which a presentation device that can play audio as presentation information can be selected. (Step S246) The display processing unit 228 outputs to the display unit 234 display data indicating the transitioned display screen.

[0109] (Step S248) The display processing unit 228 determines the position of the pointer by a drag operation based on operation information input from the operation unit 236 (range designation operation). (Step S250) The display processing unit 228 updates the display screen by reflecting the pointer at the specified position as a range designation figure. (Step S252) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen. (Step S254) The display processing unit 228 transmits to the control device 100 user range designation data indicating the position of the pointer designated by a click operation while the pointer is displayed. (Step S256) The device management unit 124 of the control device 100 selects, as a device to be controlled, a presented device corresponding to a device icon whose display range includes the position indicated by the user range designation data received from the input / output device 200.

[0110] (Step S258) The device management unit 124 transmits a control signal instructing audio playback to the selected presentation device. (Step S260) When an acknowledgement is input in response to the control signal, the device management unit 124 updates the device management data by storing, in the storage unit 122, operation status information indicating that information is being presented in response to the voice, in association with the selected presentation device. (Step S262) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input / output device 200. (Step S264) The display processing unit 228 of the input / output device 200 reflects the updated operating status on the display screen by changing the display mode of the device icon of the selected presented device based on the updated operating status information in the device management data received from the control device 100 to a display mode corresponding to the operating status information (audio playback) of the presented device. (Step S266) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen.

[0111] Fig. 27 is a sequence diagram showing a third example of the device selection process according to this embodiment. Fig. 27 shows an example in which an auxiliary video is selected as presentable information, a rectangle indicated by a drag operation is designated as an area specification figure, and a smartphone is selected as a presentation device for presenting the auxiliary video (see Fig. 18). However, this is premised on the assumption that the smartphone can acquire the distributed auxiliary video voluntarily.

[0112] (Step S272) The display processing unit 228 detects that the presentation information selection button on the display screen has been pressed based on the operation information input from the operation unit 236, and selects the auxiliary video as the type of presentable information. (Step S274) The display processing unit 228 refers to the device management data and identifies a presentation device capable of displaying auxiliary video of the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device as presentable, and sets the display mode of device icons corresponding to other presentation devices as not presentable. The display screen transitions to a device selection screen on which a presentation device capable of displaying auxiliary video as presentation information can be selected. (Step S276) The display processing unit 228 outputs to the display unit 234 display data indicating the transitioned display screen.

[0113] (Step S278) The display processing unit 228 identifies the rectangle formed by the drag operation as the range designation figure based on the operation information input from the operation unit 236 (range designation operation). (Step S280) The display processing unit 228 updates the display screen by reflecting the specified rectangle as a range specification figure. (Step S282) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen. (Step S284) The display processing unit 228 transmits to the control device 100 user range designation data indicating the range designation graphic. (Step S286) The device management unit 124 of the control device 100 selects, as a device to be controlled, a presented device corresponding to a device icon included in the selection range of the range designation figure indicated by the user range designation data received from the input / output device 200.

[0114] (Step S288) The device management unit 124 transmits a control signal instructing the selected presentation device to display the auxiliary video. (Step S290) When an acknowledgement is input in response to the control signal, the device management unit 124 updates the device management data by storing operation status information indicating that information is being presented in the auxiliary video in the storage unit 122 in association with the selected presentation device. (Step S292) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input / output device 200. (Step S294) The display processing unit 228 of the input / output device 200 reflects the updated operating status on the display screen by changing the display mode of the device icon of the selected presentation device based on the updated operating status information in the device management data received from the control device 100 to a display mode corresponding to the operating status information (display of auxiliary video) of the presentation device. (Step S296) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen.

[0115] Fig. 28 is a sequence diagram showing a fourth example of the device selection process according to this embodiment. Fig. 28 shows an example in which auxiliary audio is selected as presentable information, a pull-down menu is displayed in response to pressing of a location selection button, the living room is selected as installation location information, and a headset, a laptop PC, and a smart speaker are selected as presentation devices capable of playing the auxiliary audio in the living room (see Fig. 19).

[0116] (Step S302) The display processing unit 228 detects that a presentation information selection button on the display screen has been pressed based on operation information input from the operation unit 236, and selects auxiliary audio as a type of presentable information. (Step S304) The display processing unit 228 refers to the device management data and identifies a presentation device that can display auxiliary audio as the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device as presentable, and sets the display mode of device icons corresponding to other presentation devices as not presentable. The display screen transitions to a device selection screen on which a presentation device that can play auxiliary audio as presentation information can be selected. (Step S306) The display processing unit 228 outputs to the display unit 234 display data indicating the transitioned display screen.

[0117] (Step S308) The display processing unit 228 detects (selects) that a location selection button on the display screen has been pressed based on operation information input from the operation unit 236. (Step S310) The display processing unit 228 references the device management data and extracts the living room and bedroom as installation locations of presentation devices capable of playing auxiliary audio. The display processing unit 228 updates the display screen by placing a pull-down menu containing the extracted living room and bedroom as options as a room list adjacent to the location selection button. The display processing unit 228 outputs display data showing the updated display screen to the display unit 234. (Step S312) The display processing unit 228 displays a pointer at a position indicated by the operation information, and when that position indicates the display area of ​​one of the options and detects a press based on the operation information, selects the installation location related to that option (room selection operation). (Step S314) The display processing unit 228 transmits to the control device 100 installation location information indicating the selected installation location and presentation information type information indicating auxiliary audio as the selected type of presentation information.

[0118] (Step S316) The device management unit 124 of the control device 100 refers to the device management data and identifies the headset, the notebook PC, and the smart speaker as presentation devices that are capable of playing auxiliary audio as the type indicated in the presentation information type information received from the input / output device 200 and that are available in the living room as the installation location indicated in the installation location information and that are not presenting any type of presentation information at that time. (Step S318) The device management unit 124 activates the identified presentation devices, ie, the headset, the laptop PC, and the smart speaker, by transmitting control signals for instructing them to play the auxiliary audio. (Step S320) When an acknowledgement response to the control signal is input from the headset, the notebook PC, and the smartphone, the device management unit 124 updates the device management data by storing in the storage unit 122 operation status information indicating that information is presented for the auxiliary audio, respectively, in association with the headset, the notebook PC, and the smart speaker.

[0119] (Step S322) The device management unit 124 transmits the updated device management data to the input / output device 200. (Step S324) The display processing unit 228 of the input / output device 200 reflects the updated operating status on the display screen by changing the display mode of the device icon of the selected presented device based on the updated operating status information in the device management data received from the control device 100 to a display mode corresponding to the operating status information of the presented device (display of auxiliary audio). (Step S326) The display processing unit 228 outputs display data showing the updated display screen to the display unit 234.

[0120] Fig. 29 is a sequence diagram showing an example of device deselection processing according to this embodiment. Fig. 29 shows an example in which, when a headset, a laptop PC, and a smart speaker are selected as presentation devices for playing auxiliary audio, playback of auxiliary audio is stopped on the laptop PC (see Fig. 20).

[0121] (Step S338) The display processing unit 228 of the input / output device 200 detects the drag operation as a range specification operation from the operation information input from the operation unit 236. (Step S340) The display processing unit 228 updates the display screen by placing the pointer at the position indicated by the operation information (reflecting the range specification operation). (Step S342) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen. (Step S344) When the display processing unit 228 detects a click operation from the operation information input from the operation unit 236, it identifies the position indicated by the pointer at that time and transmits user range designation data indicating the identified position to the control device 100.

[0122] (Step S346) The device management unit 124 of the control device 100 refers to the device management data and identifies a laptop PC as the presentation device corresponding to the device icon whose display area includes the position specified by the user range designation data received from the input / output device 200. The device management unit 124 transmits a control signal to the identified laptop PC, instructing it to stop playing the auxiliary audio. (Step S350) When an acknowledgement is input in response to the control signal, the device management unit 124 updates the device management data by storing in the storage unit 122 operation status information indicating whether or not there is information for the auxiliary audio, in association with the identified presentation device, the notebook PC. (Step S352) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input / output device 200. (Step S354) The display processing unit 228 of the input / output device 200 reflects the updated operating status on the display screen by changing the display mode of the device icon of the selected presented device based on the updated operating status information in the device management data received from the control device 100 to a display mode corresponding to the operating status information of the presented device (no auxiliary audio playback). (Step S356) The display processing unit 228 outputs to the display unit 234 display data showing the updated display screen.

[0123] As described above, in the device management system 1 according to this embodiment, the display processing unit 228 displays a range designation figure on the display screen within a selection range designated in response to an operation. The device management unit 124 selects the presented devices corresponding to the device icons included in the selection range as the devices whose operating states are to be controlled. According to this configuration, it is possible to intuitively select a device whose operational state is to be controlled by using a selection range defined by an operation using a range designation figure.

[0124] <Modification> Next, modified examples of each of the above embodiments will be described. In each embodiment, the input / output device 200 may be configured as a single device management device, including a device management unit as a component that realizes functions equivalent to the device management unit 124 of the control device 100. In this case, the control device 100 may be omitted. Transmission and reception of various data between the above input / output device and the control device is performed internally by the device management device without passing through the communication unit 226. In addition, the data stored in the storage unit 122 of the control device 100 in the above is stored in the storage unit 222 of the input / output device 200. The input / output device 200 configured as a single device management device constitutes the device management system 1 by itself.

[0125] Even when the input / output device 200 includes a device management unit, the control device 100 may remain. In this case, the input / output device 200 transmits the updated device management data to the control device 100 every time the device management data is updated or every predetermined period (for example, several hours to several days). The device management unit 124 of the control device 100 updates the device management data stored in the storage unit 122 to the device management data received from the input / output device 200.

[0126] The device management unit of the input / output device 200 may not perform device control on the presentation device, but may instead have the device management unit 124 of the control device 100 perform device control. In this case, the device management unit of the input / output device 200 transmits, for example, a control signal indicating the type of presentation information and whether or not presentation of the presentation information is required to the presentation device that is to be controlled for whether or not presentation of the presentation information is required, via the device management unit 124. When an acknowledgement response to the control signal is input, the device management unit 124 updates the device management data as described above, stores it in the storage unit 122, and transmits it to the input / output device 200.

[0127] In each of the above embodiments and variations, the transmission target in the transmission of device management data between the device management unit 124 and the display processing unit 228 does not necessarily have to be the entire device management data, but may include differences from the device management data immediately before the transmission. For example, if the display positions of device icons for some presented devices are updated while other information is not updated, the transmission target may include display position information indicating the display positions of the device icons for those presented devices.

[0128] In the above embodiments, a circle is used as a device icon, which is a display element representing an individual presentation device, but this is not limiting. These display elements may be screen components (UI (User Interface) components) with different display modes, such as shape, size, color (including shading), and pattern. For example, a display element for a certain presentation device may be a button having a typical appearance shape. A display element representing a user may also be a screen component with a different display mode from the user icon described in the first embodiment. For example, a button representing a human face may be used as a display element representing a user. Furthermore, a shape with a different display mode from that described in the first embodiment may be used as a range designation shape. For example, the shape of the range designation shape may be a rectangle or a trapezoid.

[0129] In the second embodiment, a solid line and a dashed line are used to indicate the presentable and non-presentable elements, respectively. However, the presentable element is not limited to this example as long as the presentable element is more clearly expressed than the non-presentable element. For example, the non-presentable element may have a solid line or may be expressed with another line type as long as the contrast between the background and the inside of the outline of the presentable element is higher. In the above description, the presentation display is expressed by whether or not the display element is filled in, but the presenting display is not limited to this example as long as it is expressed more clearly than the display mode corresponding to the state in which the presentation device is not presenting the presentation information. For example, the presentation display may be expressed by a shade of fill, a blinking outline, or the like.

[0130] Some of the components of each of the above embodiments may be omitted or may be combined in other embodiments. For example, in the second embodiment, the display processing unit 228 may not need to identify the type of presentable information based on the operation information. In that case, the device management data may not need to be managed to include operation status information indicating the operation status for each type of presentation information. The display processing unit 228 may not need to identify the installation location of the presentation device based on the operation information. In that case, the device management data may not need to include installation location information for each presentation device. Furthermore, the control device 100 may not need to be connected to the router 400 via the Internet, as long as it can connect to each device within its management area.

[0131] Furthermore, in the first embodiment, the display processing unit 228 may identify the type of presentable information based on the operation information, and express the presentable display of the display elements of the presentation device that can present the identified type of information in a display mode corresponding to the type (for example, a color tone, a fill pattern, or other patterns). In this case, the device management data may be configured to include operation status information that indicates the operation status for each type of presentation information. The display processing unit 228 may identify the installation location of the presentation device based on the operation information, and express the display mode of the presentation device that can present the type of information at the identified installation location as the presentable display. In this case, the device management data is managed including installation location information for each presentation device. In the first embodiment, the control device 100 may be connected to each device within its management area via the router 400 and the Internet. Furthermore, the screen components used to select the installation location and the installation position in accordance with the operation information can be any components that allow one of multiple options to be selected in accordance with the operation information. For example, in addition to buttons and pull-down menus, slider bars, radio buttons, etc. may also be used.

[0132] In the above-described embodiments and modifications, the control device 100 may be configured as a server device connected to each device in each management area via the router 400 and the Internet. In this case, the control device 100 manages devices independently for each management area.

[0133] As described above, the device management system 1 according to the above embodiment includes a display processing unit (e.g., display processing unit 228) that displays a display screen on a display unit (e.g., display unit 234) in which display elements (e.g., device icons) representing individual devices (e.g., presentation devices) are arranged in different positions, and that changes, in response to an operation, at least one of the positions of the display elements and a range designation graphic (e.g., a circle, a locus, a rectangle, or a pointer) that indicates a selection range and that forms part of the display screen. The device management system 1 also includes a device management unit (e.g., device management unit 124) that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device. The display processing unit determines the display mode of the display elements of the target device to a display mode that corresponds to the operating state of the target device. With this configuration, the operating state of the target device can be controlled by changing the positional relationship between the display element representing the target device and the range specification shape, and the display element is displayed in a display mode corresponding to that operating state. The user can easily select the target device whose operating state he / she wants to control, and can understand the operating state of the device while looking at the display screen.

[0134] When a type of information is specified in response to an operation (for example, by pressing a presentation information selection button), the display processing unit refers to device management data indicating the type of information that can be presented for each device (type of presentable information), and identifies a device that can present the specified type of information as a presentable device. The display processing unit may display display elements of the presentable device in a display mode (for example, a presentable display) that is different from display elements of a device separate from the presentable device. According to this configuration, by specifying the type of information that can be presented, devices that can present the specified type of information are selected. Therefore, by specifying the type of information, it is possible to narrow down the selectable devices and make it easy to distinguish them from other devices.

[0135] When an installation location is specified in response to an operation (for example, by pressing a location information selection button), the display processing unit may refer to device management data indicating the installation location of each device and identify devices that can be used in the specified location as available devices. The display processing unit may display display elements of available devices in a display mode that differs from display elements of devices separate from available devices. With this configuration, by specifying the installation location, devices that can be used at the specified installation location are selected. Therefore, by specifying the installation location, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[0136] The display processing unit may refer to device management data indicating other devices that can be connected for each device, and may display a graphic (e.g., an arrow) indicating the possibility of connection with other devices for each device between the display elements corresponding to the one device and the other devices. This configuration allows the user to easily understand that the devices corresponding to the display elements shown on both ends of the graphic indicating the connectability are connectable to each other.

[0137] The embodiments of the present invention have been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope of the gist of the present invention.

[0138] Each component of the control device 100, the input / output device 200, the presentation device 300, and modified devices thereof may be configured with, but is not limited to, a dedicated component or an integrated circuit. The control device 100, the input / output device 200, the presentation device 300, and modified devices thereof may be configured as a computer system including a processor such as a CPU (Central Processing Unit) and a computer-readable storage medium such as a ROM or RAM. The processor may read a program stored in advance in a computer-readable storage medium, execute the read program, and cooperate with the computer-readable storage medium or other hardware resources to perform some or all of the functions of the above components. In this application, "executing a program" means executing the processing instructed by each instruction written in the program. The term "computer system" refers to the computer system built into the control device 100, the input / output device 200, the presentation device 300, and variations thereof, including hardware such as an operating system (OS) and peripheral devices. The term "computer-readable recording medium" refers to portable media such as optical magnetic disks, memory chips, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client. The program may be designed to implement some of the functions described above, or may be capable of implementing the functions described above in combination with a program already stored in the computer system. [Explanation of symbols]

[0139] 1...Device management system, 100...Control device, 122...Storage unit, 124...Device management unit, 126...Communication unit, 200...Input / output device, 222...Storage unit, 226...Communication unit, 228...Display processing unit, 234...Display unit, 236...Operation unit, 300...Presentation device, 326...Communication unit, 332...Presentation information processing unit, 334...Information presentation unit

Claims

1. displaying a display screen on the display unit in which display elements representing individual devices are arranged at different positions; a display processing unit that changes, in response to an operation, at least one of the position of the display element and a range designation figure that indicates a selection range and that forms part of the display screen; a device management unit that selects a device corresponding to a display element whose positional relationship with the range specification figure has changed as a target device and controls the operating state of the target device. the display processing unit determines a display mode of a display element of the target device to a display mode corresponding to an operating state of the target device; When the type of information is indicated according to the operation, Refer to device management data that indicates the types of information that can be presented for each device; identifying a device capable of presenting the indicated type of information as a presentable device; Displaying display elements of the presentable device in a display manner different from that of display elements of a device separate from the presentable device Device management system.

2. displaying a display screen on the display unit in which display elements representing individual devices are arranged at different positions; a display processing unit that changes, in response to an operation, at least one of the position of the display element and a range designation figure that indicates a selection range and that forms part of the display screen; a device management unit that selects a device corresponding to a display element whose positional relationship with the range specification figure has changed as a target device and controls the operating state of the target device. the display processing unit determines a display mode of a display element of the target device to a display mode corresponding to an operating state of the target device; For each device, referencing device management data that indicates other devices to which it can connect; For each of the devices, a graphic indicating the possibility of connection with the other devices is displayed between the display elements corresponding to the one device and the other devices. Device management system.

3. On the display screen, a user display element that is a display element indicating a user is disposed at a position different from a device display element that is a display element indicating the device, the range designation figure is displayed in an area including the user display element; The device management unit selects, as the target device, a device corresponding to a device display element entering the selection range or a device display element exiting the selection range.

3. The device management system according to claim 1.

4. the display processing unit displays the range designation figure in the selected range designated in response to an operation; The device management unit selects a device corresponding to a display element included in the selection range as a target device. The device management system according to any one of claims 1 to 3.

5. The display processing unit When the installation location is indicated according to the operation, Refer to device management data that shows the location of each device, Identifying devices available at the designated location as available devices; Displaying the display elements of the available devices in a different display manner from the display elements of devices separate from the available devices The device management system according to any one of claims 1 to 4.

6. displaying a display screen on the display unit in which display elements representing individual devices are arranged at different positions; a display processing unit that changes, in response to an operation, at least one of the position of the display element and a range designation figure that indicates a selection range and that forms part of the display screen; a device management unit that selects a device corresponding to a display element whose positional relationship with the range specification figure has changed as a target device and controls the operating state of the target device. the display processing unit determines a display mode of a display element of the target device to a display mode corresponding to an operating state of the target device; When the type of information is indicated according to the operation, Refer to device management data that indicates the types of information that can be presented for each device; identifying a device capable of presenting the indicated type of information as a presentable device; Displaying display elements of the presentable device in a display manner different from that of display elements of a device separate from the presentable device Device management equipment.

7. displaying a display screen on the display unit in which display elements representing individual devices are arranged at different positions; a display processing unit that changes, in response to an operation, at least one of a position of the display element and a range designation figure that indicates a selection range and that forms part of the display screen; The display processing unit selecting a device corresponding to a display element whose positional relationship with the range specification figure has changed as a target device; determining a display mode of the display element of the target device to a display mode corresponding to an operating state of the target device; When the type of information is indicated according to the operation, Refer to device management data that indicates the types of information that can be presented for each device; identifying a device capable of presenting the indicated type of information as a presentable device; Displaying display elements of the presentable device in a display manner different from that of display elements of a device separate from the presentable device Input / output devices.

8. A program for causing a computer to function as the device management apparatus according to claim 6 or the input / output device according to claim 7.

9. a first step in which a display processing unit displays a display screen on the display unit, in which display elements representing individual devices are arranged at different positions; a second step in which the display processing unit changes at least one of the position of the display element and a range designation figure indicating a selection range that is part of the display screen in response to an operation; a third step in which a device management unit selects, as a target device, a device corresponding to a display element whose positional relationship with the range specification figure has changed, and controls an operating state of the target device; a fourth step in which the display processing unit determines a display mode of the display element of the target device to a display mode corresponding to an operating state of the target device; A device management method for performing The fourth step includes: When the type of information is indicated according to the operation, Refer to device management data that indicates the types of information that can be presented for each device; identifying a device capable of presenting the indicated type of information as a presentable device; a step of displaying a display element of the presentable device in a different display manner from a display element of a device separate from the presentable device; Device management methods.

10. a first step in which a display processing unit displays a display screen on the display unit, in which display elements representing individual devices are arranged at different positions; a second step in which the display processing unit changes at least one of the position of the display element and a range designation figure indicating a selection range that is part of the display screen in response to an operation; a third step in which a device management unit selects, as a target device, a device corresponding to a display element whose positional relationship with the range specification figure has changed, and controls an operating state of the target device; a fourth step in which the display processing unit determines a display mode of the display element of the target device to a display mode corresponding to an operating state of the target device; A device management method for performing The fourth step includes: For each device, referencing device management data that indicates other devices to which it can connect; and displaying, for each of the devices, a graphic indicating the possibility of connection with the other devices between display elements corresponding to the one device and the other devices. Device management methods.

Images