Sound amount controller, apparatus, control system, and program

Abstract

To provide: a sound amount controller capable of performing control according to a user's intention to a plurality of apparatuses while having operability of transmitting a command to only an operation target apparatus; and the apparatuses.SOLUTION: An apparatus comprises an apparatus state management section, an apparatus information provision section, and a control section. The apparatus state management section manages apparatus information including information on how sound originated by the apparatus is heard by a user. The apparatus information provision section transmits the apparatus information to an outside sound amount controller. The control section receives control information transmitted from the sound amount controller, the control information influencing how the sound originated by the apparatus is heard by the user, and controls at least any of a sound amount of voice to be output by the apparatus and an operation section that operates while originating sound in the apparatus on the basis of the received control information.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a volume control device, a device, a control system, and a program. [Background technology]

[0002] In conventional technology, in order to adjust the volume emitted by a target device (such as a television receiver), a mechanism has been used in which, for example, a user operates a remote control device, and the target device, upon receiving a control signal from the remote control device, increases or decreases the volume.

[0003] In recent years, IoT (Internet of Things) devices for homes have become widespread. For example, in a single room, media devices such as televisions and smart speakers are mixed with sound-emitting IoT devices such as vacuum cleaners, air purifiers, and electric toothbrushes. The intermingling of sounds from these devices can sometimes make it difficult for users to hear the sound of the device they want to listen to. In such cases, simply adjusting the volume of the device they want to listen to is not enough to create an appropriate volume environment for the user. In other words, the overall volume level in the user's environment may not be appropriate. Furthermore, there is an upper limit to the allowable volume for all devices in a user's living environment. It is extremely difficult for users to take all of this information into account and control the volume and operation of each device.

[0004] Patent Document 1 describes an IoT system with a voice control function and an information processing method thereof. The technology described in Patent Document 1 manages the communication status and interactivity between any two IoT devices among multiple IoT devices present in the system. Then, recognized voice content is wirelessly transmitted in a one-to-many broadcast format, and a message is sent to one specified device. Then, by executing a specific command, the selected IoT device is driven in a broadcast format or the returned information is obtained, thereby improving the convenience of operation.

[0005] Patent Document 2 describes the configuration of a terminal device for controlling IoT devices. The technology described in Patent Document 2 operates IoT devices in conjunction with content provided by broadcasting or communication. Specifically, the terminal device selects an IoT device according to the content itself or metadata described in each scene of the content, and controls the operation of the selected IoT device. In this technology, the control details for the IoT device are described within the content. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2017-192091 [Patent Document 2] Patent Publication No. 2021-135621 Summary of the Invention [Problem to be solved by the invention]

[0007] Users want to be able to control the volume with simple operations using a remote control device, etc. When a user operates one device using a remote control device, it is desirable to be able to infer the user's intention and control devices other than the intended operation. In addition, it is desirable to avoid the effort of having to pre-write information for controlling each IoT device for each piece of content.

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide a volume control device, device, control system, and program that allows a user to easily send commands to only the device to be operated using a user input device (such as a remote control device), while enabling the user to control multiple devices including the device to be operated in accordance with the user's intentions. [Means for solving the problem]

[0009] [1] In order to solve the above problem, a volume control device according to one aspect of the present invention includes a user control receiving unit that acquires command information for a device to be operated from a user input device; a device information acquiring unit that acquires device information from each of a plurality of devices including the device to be operated, the device information including information on how the sound emitted by the device sounds to the user; a user intention interpretation unit that performs processing to interpret the user's intention regarding the volume of the sound emitted by the device based on the command information acquired by the user control receiving unit and the device information for each of the devices acquired by the device information acquiring unit; a device control determination unit that determines control content for at least some of the plurality of devices based on the user's intention obtained as a result of interpretation by the user intention interpretation unit; and a device control unit that controls each of the devices to be controlled based on the control content for each device determined by the device control determination unit. Here, the device to be operated is, for example, a television receiver that receives a broadcast signal and presents to the user video and audio based on the broadcast signal.

[0010] [2] Also, one aspect of the present invention is that, in the above-mentioned volume control device, the device information includes at least one of the following: location information of the device; if the device has an audio output unit, the volume level of the audio output by the audio output unit; information representing the operating status of the device; if the device has a means of transportation, the direction of movement of the device; if there is a setting item related to the sound emitted by the device, the setting value for the setting item; and if the device is capable of setting the output volume, information on the volume setting value previously set by the user. Here, the operating status of the device is represented by information indicating whether the device is operating or stopped, information about the device's operating mode, the strength of the device's operation (weak / medium / strong, etc.), and information about the output value of the device's operation.

[0011] [3] In another aspect of the present invention, the volume control device further includes a user information management unit that stores user information including at least one of the user's location information, information about the user's preferred volume, information about the priority of each device when the user listens to sound, and information about the user's preferred content, and the user intention interpretation unit performs processing to interpret the user's intention regarding the volume of sound emitted by the device based on the user information obtained from the user information management unit.

[0012] [4] Furthermore, a device according to one aspect of the present invention includes a device status management unit that manages device information including information about how the sound emitted by the device sounds to the user, a device information providing unit that transmits the device information to an external volume control device, and a control unit that receives control information transmitted from the volume control device that affects how the sound emitted by the device sounds to the user, and controls at least one of the volume of the sound output by the device and an operating unit that operates while emitting sound in the device, based on the received control information.

[0013] [5] Furthermore, one aspect of the present invention is that, in the above-mentioned device, the device information includes at least any of the following: location information of the device; if the device has an audio output unit, the volume level of the audio output by the audio output unit; information representing the operating status of the device; if the device has a means of transportation, the direction of movement of the device; if there is a setting item related to the sound emitted by the device, the setting value for the setting item; and if the device is capable of setting the output volume, information on the volume setting value previously set by the user.

[0014] [6] Furthermore, one aspect of the present invention is that the above-mentioned device further comprises a user control receiving unit that receives command information from a user input device to the device itself, and a user control providing unit that transmits the command information received by the user control receiving unit to the volume control device, wherein the control unit does not immediately control at least one of the volume of the sound output by the device itself and the operating unit that operates in the device while emitting sound based on the command information received by the user control receiving unit, but controls at least one of the volume of the sound output by the device itself and the operating unit that operates in the device while emitting sound based on the control information transmitted by the volume control device based on the command information transmitted to the volume control device. The device described in [6] receives commands directly from a user input device and is also called a "controlled device." An example of a controlled device is a television set that receives a broadcast signal and presents video and audio based on the broadcast signal to the user.

[0015] [7] Furthermore, a control system according to one aspect of the present invention is a control system comprising a user input device, an operation target device that receives command information sent to the device by the user input device, an apparatus that is not the operation target of the command information sent by the user input device, and a volume control device that controls the operation target device and the apparatus, wherein the volume control device included in the control system is a volume control device described in any of [1] to [3] above, the operation target device included in the control system is a device described in [6] above, and the device included in the control system is a device described in [4] or [5] above. An example of the device to be operated is a television set that receives a broadcast signal and presents to the user video and audio based on the broadcast signal.

[0016] [8] Also, one aspect of the present invention is a program that causes a computer to function as a volume control device, comprising: a user control receiving unit that acquires command information for a target device from a user input device; a device information acquiring unit that acquires device information from each of a plurality of devices including the target device, the device information including information on how the sound emitted by the device sounds to the user; a user intention interpretation unit that performs processing to interpret the user's intention regarding the volume of the sound emitted by the device based on the command information acquired by the user control receiving unit and the device information for each of the devices acquired by the device information acquiring unit; a device control determination unit that determines control content for at least some of the plurality of devices based on the user's intention obtained as a result of interpretation by the user intention interpretation unit; and a device control unit that controls each of the devices to be controlled based on the control content for each device determined by the device control determination unit.

[0017] [9] Another aspect of the present invention is a program that causes a computer to function as a device that includes a device status management unit that manages device information including information about how a sound emitted by the device sounds to a user, a device information providing unit that transmits the device information to an external volume control device, and a control unit that receives control information transmitted from the volume control device that affects how a sound emitted by the device sounds to a user, and controls at least one of the volume of the sound output by the device and an operating unit that operates while emitting sound in the device, based on the received control information. Alternatively, one aspect of the present invention is a program that causes a computer to function as a device status management unit that manages device information including information regarding how sounds emitted by the device sound to the user, a device information providing unit that transmits the device information to an external volume control device, and a control unit that receives control information transmitted from the volume control device that affects how sounds emitted by the device sound to the user, and controls at least one of the volume of the sound output by the device and an operating unit that operates while emitting sound in the device, based on the received control information. [Effects of the Invention]

[0018] According to the present invention, the volume control device interprets the user's intention based on command information for the device to be operated, which is acquired from the user input device, and device information including information on how the sound emitted by the device sounds to the user. Based on the user's intention as a result of the interpretation, the volume control device can determine the control content not only for the device to be operated, but also for other devices. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram showing a schematic configuration of a control system according to an embodiment of the present invention. [Figure 2] 2 is a block diagram showing a schematic functional configuration of the volume control device according to the embodiment; FIG. [Figure 3] FIG. 2 is a block diagram showing a schematic functional configuration of the user input device according to the embodiment. [Figure 4] 2 is a block diagram showing a schematic functional configuration of an operation target device (for example, a television receiver) according to the embodiment. FIG. [Figure 5] FIG. 2 is a block diagram showing a schematic functional configuration of an IoT device (other than an operation target device) according to the embodiment. [Figure 6] FIG. 10 is a sequence diagram (1 / 2) showing a first example of a processing sequence in the control system according to the embodiment. [Figure 7] FIG. 2 is a sequence diagram (2 / 2) showing a first example of a processing sequence in the control system according to the embodiment. [Figure 8] 3 is a schematic diagram showing an example of the configuration of device information (information on external devices) held by a device information management unit of the volume control device according to the embodiment. FIG. [Figure 9] FIG. 10 is a sequence diagram (1 / 2) showing a second example of a processing sequence in the control system according to the embodiment. [Figure 10] FIG. 10 is a sequence diagram (2 / 2) showing a second example of the processing sequence in the control system according to the embodiment. [Figure 11] FIG. 10 is a sequence diagram showing a third example of a processing sequence in the control system according to the embodiment. [Figure 12] FIG. 2 is a block diagram showing an example of the internal configuration of a computer included in each device constituting the control system of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Next, an embodiment of the present invention will be described with reference to the drawings.

[0021] In this embodiment, when a user uses a user input device to perform a volume control operation on one of the target devices, the volume control device interprets the location of the user's operation according to the operating status of each device. The volume control device automatically controls the volume and operation of multiple devices (the target device and other devices) based on the interpretation result. The volume control device not only controls the volume output by the devices, but also the operation, position, and direction of movement of the devices. This creates an environment appropriate for the user. In other words, a volume environment that matches the user's intentions is created. In other words, unlike the technology described in Patent Document 2, in this embodiment, there is no need to describe information for controlling IoT devices in the content.

[0022] FIG. 1 is a block diagram showing a schematic functional configuration of a control system according to this embodiment. As shown in the figure, the control system 100 includes a volume control device 1, a user input device 3, an operation target device 5, and an IoT device group 7G. The IoT device group 7G is a collection of IoT devices 7. The functions of these devices can be realized, for example, using a computer and a program. Each device also has a storage means as needed. The storage means is, for example, a program variable or memory allocated by program execution. Non-volatile storage means such as a magnetic hard disk drive or solid-state drive (SSD) may also be used as needed. At least some of the functions of each device may be realized as a dedicated electronic circuit rather than a program.

[0023] The devices included in the control system 100 can communicate with each other. For example, the volume control device 1 can exchange information with the user input device 3, the operation target device 5, and the IoT device 7, for example, by wireless communication. Furthermore, information can be transmitted from the user input device 3 to the operation target device 5 by infrared signals, radio waves, optical signals, sound, or other means.

[0024] The volume control device 1 is a device provided for controlling the volume of sounds (including both sounds produced by playing content and noise, static, etc.) emitted by each of the operation target devices 5 and the IoT devices 7. The volume control device 1 controls the operation target devices 5 and the IoT devices 7 based on device information, which is information relating to how sounds emitted by the operation target devices 5 and the IoT devices 7 sound to the user, and the definition of a command sent to the operation target devices 5 by the user input device 3. The volume control device 1 may further control the operation target devices 5 and the IoT devices 7 based on user information, which is information indicating the user's location, user preferences, etc.

[0025] The user input device 3 is a device through which a user inputs commands to a device. The user input device 3 outputs a control signal to the device based on the user's operation. Specific examples of the user input device 3 include a television remote control device and a smart speaker. A television remote control device outputs a control signal by an infrared signal or the like based on the user's operation of a push button or the like. A smart speaker outputs a control signal by communication based on the user's spoken voice.

[0026] The user input device 3 may use infrared signals, communication via Bluetooth (registered trademark), communication via BLE (Bluetooth Low Energy), millimeter wave communication, or communication via IP (Internet Protocol) packets via various physical means as signals for operating the operation target device 5. The user input device 3 may also use audio signals to transmit control information to the operation target device 5. The user input device 3 may also transmit control information to the operation target device 5 via a touch operation interface using a touch panel or the like. The user input device 3 transmits commands to the operation target device 5 by using control signals such as infrared rays. The user input device 3 may also pass command information for the operation target device 5 to the volume control device 1.

[0027] The commands transmitted by the user input device 3 are commands related to the operation of the target device 5, such as a television receiver. These commands are commands for turning the power of the target device 5 on / off, increasing / decreasing the volume of the audio output from the target device 5, and selecting content (broadcast service, etc.) to be played by the target device 5.

[0028] The user input device 3 may be housed in a common housing together with the operation target device 5.

[0029] The operation target device 5 is a device that is the target of operation by the user input device 3. A typical example of the operation target device 5 is a television receiver. A television receiver is a device that receives broadcast signals and presents video and audio extracted from the broadcast signals to the user. The operation target device 5 includes wireless communication means. This allows the operation target device 5 to communicate with the volume control device 1 using, for example, IP (Internet Protocol) packets.

[0030] The IoT device group 7G is a collection of IoT devices 7, which are explained below. IoT stands for "Internet of Things."

[0031] The IoT device 7 is a device that has a function of communicating with the volume control device 1. Specifically, the IoT device 7 may include, for example, a television set, a speaker (an IoT speaker, a smart speaker, etc.), an IoT air purifier, an IoT vacuum cleaner (including an autonomously moving vacuum cleaner), a vibration device that generates vibrations, an air conditioner, a ventilation fan, an electric fan, a water heater for supplying hot water to a bathroom, a kitchen, etc., an electric toothbrush, an electric bed, a refrigerator, a washing machine, a microwave oven, a rice cooker, a mixer, a power management system, an automobile, and other devices. However, the IoT device 7 is not limited to the above-listed examples. The IoT device 7 is a device that emits sound. The IoT device 7 may emit sound by being equipped with a speaker or the like, or may emit sound by a function other than a speaker or the like, or may be equipped with both.

[0032] The mechanism by which the IoT device 7 produces sound using a function other than a speaker may be, for example, as follows. The IoT device 7 may produce sound by moving a moving part using an electric motor, an internal combustion engine, or the like. The IoT device 7 may also produce sound by vibrations caused by the movement of a fluid (liquid or gas) inside the IoT device 7. The IoT device 7 may also produce sound by burning a substance such as fuel.

[0033] Here, a method for determining the location of a user and each device or equipment in the control system 100 will be described. As will be described later, the control system 100 may control the volume based on the location of the user or each device or equipment. To this end, the control system 100 has means for measuring the location of the user and each device or equipment. As an example, the location (coordinate values) of the user and each device or equipment may be determined using the functions of a GPS device worn by the user or the functions of a GPS device provided in each device or equipment. GPS stands for "Global Positioning System." Alternatively, instead of a GPS device, the user may have a short-range wireless communication means, or each device may have a short-range wireless communication means, and the locations (coordinate values) of the user and each device or equipment may be determined by receiving beacon signals transmitted from devices provided in a room, etc. Alternatively, the locations (coordinate values) of the user and each device or equipment may be determined by geometrically analyzing images captured by an imaging means (camera) provided in the room, etc.

[0034] Furthermore, the control system 100 may determine not only the coordinate values ​​of the user and each device / appliance but also the room in which the user and each device / appliance are located. This is because, when controlling the volume in the control system 100, sound is blocked by walls, doors, etc., so information identifying the room in which the user and each device / appliance are located is useful. To identify the room in which the user and each device / appliance are located, the control system 100 allows each device / appliance (at least a portion of the device / appliance) to refer to floor plan information. The floor plan information is information that can associate information on three-dimensional coordinates representing a specific position (point) with information identifying the room. For example, the floor plan information may be information that can represent the boundary surfaces (walls, floor, ceiling, etc.) of a room using three-dimensional coordinates. Note that, if it is not necessary to determine the vertical position, two-dimensional coordinates may be used instead of three-dimensional coordinates.

[0035] In the control system 100, control may be performed using the volume of sound heard at the user's location. The sound in this case is a sound output by the target device 5 or a sound emitted by the IoT device 7. To measure the volume at the user's location, for example, one of the following two methods can be used. The first method is a method of calculating the volume at the user's location based on the power of the sound emitted by the target device 5 or the IoT device 7 and the attenuation of the sound in space, etc. The second method is a method of measuring the volume using a microphone installed near the user's location. Here, the microphone may be installed in a terminal device used by the user or in an IoT device near the user.

[0036] 2 is a block diagram showing a schematic functional configuration of a volume control device according to this embodiment. As shown in the figure, the volume control device 1 includes a user control receiving unit 11, a user intention interpretation unit 12, a device control determination unit 13, a device control unit 14, a user information management unit 15, a device information acquisition unit 16, and a device information management unit 17.

[0037] The user control receiving unit 11 receives commands (control signals) from the user input device 3 or commands passed from the user control providing unit 52 of the operation target device 5. In other words, the user control receiving unit 11 obtains information on commands sent from the user input device 3 to the operation target device 5. The user control receiving unit 11 passes the contents of the received commands to the user intention interpretation unit 12.

[0038] The user intention interpretation unit 12 performs processing to interpret the user's intention in the command regarding the volume of the sound emitted by the device, based on the command information acquired by the user control receiving unit 11 and the device information for each device acquired by the device information acquisition unit 16. The user intention interpretation unit 12 may also perform processing to interpret the user's intention in the command regarding the volume of the sound emitted by the device, based on user information acquired from the user information management unit 15. That is, in this embodiment, the user intention interpretation unit 12 interprets the intention of the command (user-based control signal) passed from the user control receiving unit 11, based on the user information obtained from the user information management unit 15 and the device information obtained from the device information management unit 17. The user intention interpretation unit 12 passes the interpretation result to the device control determination unit 13.

[0039] The specific processing executed by the user intention interpretation unit 12 is processing to determine output information (intention as an interpretation result) based on the pattern of input information (command, device information, user information, etc.). An example of the processing method by the user intention interpretation unit 12 will be described later.

[0040] The device control determiner 13 determines the control content based on the interpretation (interpretation of the command from the user) passed from the user intention interpreter 12 and the device information passed from the device information manager 17. In other words, the device control determiner 13 determines the control content for at least some of the multiple devices (the operation target devices 5 and the IoT devices 7) based on the user's intention obtained as a result of the interpretation by the user intention interpreter 12. The device control determiner 13 communicates the determined control content to the device controller 14.

[0041] The device control unit 14 outputs control signals to the operation target device 5 and the IoT device 7 based on the control content determined by the device control determination unit 13. That is, the device control unit 14 controls each device to be controlled based on the control content determined by the device control determination unit 13 for each device (operation target device 5 and IoT device 7).

[0042] The user information management unit 15 acquires, stores, and manages user information. The user information includes information about the user's media viewing. The user information includes at least one of the following: location information of the user operating the user input device 3, information about the user's preferred volume, information about the priority of each device when the user listens to sound, and information about the user's preferred content. The user information management unit 15 can acquire the user information from the user input device 3 by requesting the user information from the user input device 3. The user information management unit 15 may also store a history of information that the user has input or set in the past as user information.

[0043] The device information acquisition unit 16 acquires device information about each device from the device information provision unit 55 of the operation target device 5 or the device information provision unit 75 of the IoT device 7. This device information includes information about how sounds emitted by the device (operation target device 5 or IoT device 7) sound to the user. The device information acquisition unit 16 passes the acquired device information to the device information management unit 17.

[0044] The device information acquired by device information acquisition unit 16 includes at least any of the following: location information of each device; if the device has an audio output unit, the volume level of the audio output by the audio output unit; information indicating the operating status of the device; if the device has a means of transportation, the direction of movement of the device; if there is a setting item related to the sound emitted by the device, the setting value for the setting item; and if the device allows setting of the output volume, information on the volume setting value previously set by the user. Here, the operating status of the device is represented by information indicating whether the device is operating or stopped, information about the operating mode of the device, the strength of the device's operation (weak / medium / strong, etc.), and information on the output value of the device's operation.

[0045] The device information management unit 17 holds and manages the device information of each device (the operation target device 5 and the IoT device 7) acquired by the device information acquisition unit 16. When the device information management unit 17 receives the device information from the device information acquisition unit 16, it transmits the device information to the user intention interpretation unit 12.

[0046] 3 is a block diagram showing a schematic functional configuration of the user input device according to this embodiment. As shown in the figure, the user input device 3 includes a user control transmission unit 31 and a user information transmission unit 32.

[0047] The user control transmission unit 31 transmits a command signal to the operation target device 5. The command may be, for example, an instruction to turn the volume up / down or an instruction to turn the power on / off. The user control transmission unit 31 may pass the command signal not only to the operation target device 5 but also to the volume control device 1. The user control transmission unit 31 transmits the command to the operation target device 5 using means such as an infrared signal, Bluetooth (Bluetooth, registered trademark), BLE (Bluetooth Low Energy), an audio signal, a touch signal (a signal generated by contact with a touch panel or the like of the operation target device 5), millimeter wave communication, or an IP packet.

[0048] The user information transmitting section 32 transmits the user information to the user information managing section 15 of the volume control device 1. The user information is as already explained.

[0049] 4 is a block diagram showing a schematic functional configuration of a target device 5 according to this embodiment. As shown in the figure, the target device 5 includes a user control receiving unit 51, a user control providing unit 52, a control unit 53, a device status management unit 54, a device information providing unit 55, an audio output unit 56, and an operation unit 57.

[0050] The user control receiving unit 51 receives commands (control signals) from the user input device 3. The user control receiving unit 51 passes the received commands to the user control providing unit 52. The user control receiving unit 51 also passes the received commands to the control unit 53. The user control receiving unit 51 is equipped with a means for receiving signals from the user input device 3 (for example, a wireless signal receiver, etc.).

[0051] The user control providing unit 52 receives the above command from the user control receiving unit 51. The user control providing unit 52 transmits information indicating the content of the received command to the user control receiving unit 11 of the volume control device 1. This allows the content of the command sent from the user input device 3 to be transmitted not only to the device 5 to be operated, but also to affect other devices (such as the IoT device 7).

[0052] The control unit 53 controls the operation and audio output of its own device (the operation target device 5). When the control unit 53 in this embodiment receives a command (a command transmitted from the user input device 3) from the user control receiving unit 51, it instructs the device status management unit 54 to pass information about the device (the operation target device 5) to the device information providing unit 55. When the control unit 53 receives a control signal from the device control unit 14 of the volume control device 1, it controls at least one of the audio output unit 56 and the operation unit 57 in accordance with the control signal. The control signal received from the device control unit 14 of the volume control device 1 includes information for control that affects how the sound emitted by its own device sounds to the user.

[0053] In addition, the control unit 53 does not immediately control at least one of the volume of the audio output by its own device (audio output unit 56) and the operating unit 57 that operates while emitting sound in its own device based on the command information received by the user control receiving unit 51, but controls at least one of the volume of the audio output by its own device (audio output unit 56) and the operating unit 57 that operates while emitting sound in its own device based on control information sent from the volume control device 1 based on the command information sent to the volume control device 1.In other words, the control unit 53 does not immediately control the audio output unit 56 or the operating unit 57 based on commands from the user input device 3, but performs control based on a control signal sent from the volume control device 1 based on the result of the volume control device 1 interpreting the command from the user input device 3.

[0054] Specifically, the control unit 53 controls the volume of the sound output by the sound output unit 56. That is, the control unit 53 controls the UP / DOWN of the volume in the sound output unit 56. The control unit 53 also controls the operation of the operation unit 57. The control unit 53 may also control the ON / OFF of the power of the operation target device 5 itself.

[0055] The device status management unit 54 holds and manages information about the device (called device information). When the device status management unit 54 receives an instruction from the control unit 53, it passes the device information to the device information providing unit 55 based on the instruction. The device information is information that includes information about how the sound emitted by the device sounds to the user. The device information includes information about the state of the device at that time. Specific examples of device information have already been explained.

[0056] The device information providing unit 55 transmits the device information passed from the device state management unit 54 to the device information acquisition unit 16 of the volume control device 1.

[0057] The audio output unit 56 outputs audio based on an electrical signal. The audio output unit 56 includes a speaker (including earphones, etc.) for outputting audio. The audio output unit 56 outputs sound by vibrating a diaphragm of the speaker based on the audio signal. The volume of the audio output by the audio output unit 56 is controlled by the control unit 53.

[0058] The operating unit 57 is a unit having a function of performing operations other than audio output. If the operation target device 5 is a television set, the operating unit 57 turns the television set on and off, selects a service to receive, requests content distributed over the Internet from a server device, and adjusts the brightness and color of the screen. If the operation target device 5 is a device with a moving function, the operating unit 57 moves the operation target device 5 toward a predetermined position, in a predetermined direction, and at a predetermined speed. The destination position, destination direction, and speed of the movement here are variable and may be controlled by the control unit 53. If the operation target device 5 has a moving part, the operating unit 57 moves the moving part using energy supplied by a motor, internal combustion engine, etc. Here, the type, direction, speed, strength, etc. of the movement of the moving part are variable and may be controlled by the control unit 53. If the device to be operated 5 moves a fluid (liquid or gas), the operating unit 57 moves the fluid using a fan or the like, or changes the flow rate by adjusting the speed of the fan or the opening of a valve or the like that can be opened or closed.

[0059] These operations by operating unit 57 can be controlled by control unit 53. In other words, the amount (intensity) of sound emitted by the operations caused by operating unit 57 can be controlled by control unit 53.

[0060] The volume of sound output by the audio output unit 56 and the volume of sound emitted by the operation of the operation unit 57 change depending on the positional relationship between the operation target device 5 and the user (the volume perceived by the user). In other words, the volume of sound perceived by the user changes depending on the distance between the operation target device 5 and the user and whether or not there is an obstruction, such as a wall, between the operation target device 5 and the user. In other words, if the operation target device 5 is a device that has a mobile function, the operation unit 57 can change the volume of sound perceived by the user, of the sound emitted from the audio output unit 56 or the operation unit 57, by moving the operation target device 5. In other words, the control unit 53 can perform control to change these.

[0061] FIG. 5 is a block diagram showing a schematic functional configuration of an IoT device according to this embodiment. As shown in the figure, the IoT device 7 includes a control unit 73, a device status management unit 74, a device information providing unit 75, an audio output unit 76, and an operation unit 77. In other words, the IoT device 7 has a configuration similar to that of the operation target device 5. However, the IoT device 7 does not need to have functions corresponding to the user control receiving unit 51 and the user control providing unit 52 in the operation target device 5. In other words, unlike the operation target device 5, the IoT device 7 does not receive commands (control signals) from the user input device 3. However, like the operation target device 5, the IoT device 7 provides its own device information to the volume control device 1. The device information has been described above. Also, like the operation target device 5, the IoT device 7 controls its operation (including volume up / down) based on commands (control signals) from the volume control device 1. The functions of each unit constituting the IoT device 7 are as follows.

[0062] The IoT device 7 may be configured without either the audio output unit 76 or the operation unit 77.

[0063] The control unit 73 controls the operation and audio output of its own device (IoT device 7) in the same way as the control unit 53 in the operation target device 5. Specifically, when the control unit 73 receives a control signal from the device control unit 14 of the volume control device 1, it controls the audio output unit 76 and the operation unit 77 in accordance with the control signal. That is, the control unit 73 controls the volume of the audio output by the audio output unit 76. That is, the control unit 73 controls the volume up / down of the audio output unit 76. The control unit 73 also controls the operation by the operation unit 77. The control unit 73 may also control the power on / off of the IoT device 7 itself.

[0064] The device status management unit 74 holds and manages information (referred to as device information) of the device (its own device) in the same way as the device status management unit 54 in the operation target device 5. The device status management unit 74 passes the device information to the device information providing unit 75 based on an instruction from the control unit 73.

[0065] The device information may include an audio output state, an audio output level, an operating condition, an operating mode, position information (coordinate values, etc.), a moving direction, a moving speed, and the like.

[0066] The device information providing unit 75 transmits the device information passed from the device state management unit 74 to the device information acquisition unit 16 of the volume control device 1, similar to the device information providing unit 55 in the operation target device 5.

[0067] The audio output unit 76 outputs audio based on an electrical signal (audio signal), similar to the audio output unit 56 in the operation target device 5. The audio output unit 76 is provided with a speaker (including earphones, etc.) for outputting audio. The volume of the audio output by the audio output unit 76 is controlled by the control unit 73. That is, the control unit 73 controls the volume of the audio output by the audio output unit 76 to increase or decrease.

[0068] The operation unit 77 is a unit having a function of performing operations other than audio output, similar to the operation unit 57 in the operation target device 5. That is, the operation unit 77 turns the power of the IoT device 7 on and off, and if the IoT device 7 is a device that receives content, selects the type of content. If the IoT device 7 is a device with a moving function, the operation unit 77 moves the IoT device 7 toward a predetermined position, in a predetermined direction, and at a predetermined speed. If the IoT device 7 has moving parts, the operation unit 77 moves the moving parts using energy supplied by a motor, an internal combustion engine, or the like. If the IoT device moves a fluid (liquid or gas), the operation unit 77 moves the fluid using a fan or the like, or changes the flow rate by adjusting the speed of the fan or the opening of an openable / closable valve or the like.

[0069] The operations of these operating units 57 can be controlled by the control unit 73. In other words, the volume of the sound emitted by the operation of the operating units 77 can be controlled by the control unit 73.

[0070] The IoT device 7 itself may be mobile. For example, if the IoT device 7 is an automobile, an electric kick scooter, an electric wheelchair, a drone, or the like, the operation unit 77 moves the IoT device 7 itself to change its position. Also, if the IoT device 7 is an autonomously mobile vacuum cleaner, for example, the operation unit 77 moves the IoT device 7 itself to change its position. The volume of sound output by the audio output unit 76 and the volume of sound emitted by the operation of the operation unit 77 change how the sound is heard by the user (the volume perceived by the user) depending on the positional relationship between the IoT device 7 and the user. In other words, the volume of sound perceived by the user changes depending on the distance between the IoT device 7 and the user and whether or not there is an obstruction, such as a wall, between the IoT device 7 and the user. In other words, if the IoT device 7 is a device with a mobile function, the operation unit 77 can change the volume of sound perceived by the user of the sound emitted from the audio output unit 76 or the operation unit 77 by moving the IoT device 7. In other words, the control unit 73 can perform control to change these.

[0071] Next, an example of the flow of processing in the control system 100 will be described with reference to a diagram showing communication between devices.

[0072] [Processing sequence (first example)] 6 and 7 are sequence diagrams showing a first example of a processing sequence in the control system 100. Both FIGS. 6 and 7 show a series of sequences. In this first example, it is assumed that the user input device 3 is a television remote control device, the operation target device 5 is a television receiver that receives a signal from the remote control device, the IoT device 7-1 is a smart speaker, and the IoT device 7-2 is an autonomous mobile vacuum cleaner. However, the specific types of devices may be different. In this example, as will be described later, the user information transmission unit 32 of the user input device 3 has a function of providing user information to the volume control device 1 in response to a request from the volume control device 1. The processing sequence of this first example is a process when the user input device 3 transmits a command (control signal) to turn up the volume to the operation target device 5. The first example will be described below with reference to this diagram.

[0073] First, in step S1 (FIG. 6), the user control transmitter 31 of the user input device 3 transmits a volume up command (control signal) to the user control receiver 51 of the target device 5. The user control receiver 51 receives this command. Note that the trigger for this volume up command is a button operation by a user who wants to hear the audio output of the television more clearly.

[0074] Next, in step S2, the user control providing unit 52 of the target device 5 transmits the signal of the volume UP command received in step S1 to the user control receiving unit 11 of the volume control device 1. The user control receiving unit 11 receives information about this command that has been provided. That is, the user control receiving unit 11 obtains information that the target device 5 has received the volume UP command.

[0075] In step S2, the user control receiving unit 11 may receive the volume up command directly from the user input device 3 instead of obtaining the volume up command information from the operation target device 5 side.

[0076] In either case, in the volume control device 1, the user control receiving unit 11 passes the information of the received command to turn up the volume to the user intention interpreting unit 12.

[0077] Next, in step S3, the device control unit 14 of the volume control device 1 requests device information from the operation target device 5 and the IoT devices 7-1 and 7-2.

[0078] Next, in step S4, the device information acquisition unit 16 of the volume control device 1 receives device information provided from the device information provision unit 55 of the operation target device 5 and the device information provision units 75 of the IoT devices 7-1 and 7-2 in response to the request in step S3. The device information acquisition unit 16 passes the acquired device information of each device to the device information management unit 17. The device information management unit 17 can pass the acquired device information to the user intention interpretation unit 12.

[0079] The device information management unit 17 holds and manages device information acquired from each device. The device information may include information such as the volume level of the device, the current operating mode of the device (e.g., "high" or "low"), the device's location, the direction the device is facing, the device's relative location relative to the user, past volume settings made by the user, the device's current audio playback status, the audio playback frequency, configurable items, and the device's total low output volume. Instead of the device information providing units 55 and 75 providing device information about their own devices, devices equipped with sensors may detect device information about other devices and provide the information to the device information acquisition unit 16. A specific example of the device information stored by the device information management unit 17 will be described later with reference to FIG. 8.

[0080] Next, in step S5, the user information management unit 15 of the volume control device 1 transmits a user information request to the user information transmission unit 32 of the user input device 3. The user information transmission unit 32 receives this request.

[0081] Next, in step S6, the user information transmission unit 32 of the user input device 3 transmits user information to the user information management unit 15 of the volume control device in response to the request received in step S5. The user information management unit 15 stores and manages the acquired user information. The user information management unit 15 passes the acquired user information to the user intention interpretation unit 12.

[0082] The user information management unit 15 may further acquire, as user information, a history of associations between past commands from the user and decisions made by the device control determination unit 13. The user information includes information about the user's media viewing. The user information includes, for example, information about the user's location, the user's preferred volume setting, the user's preferred audio output device, and the content the user plays.

[0083] Next, in step S7, the user intention interpretation unit 12 of the volume control device 1 interprets and estimates the intention of the user command transmitted from the user input device 3. Here, the user intention interpretation unit 12 executes the intention interpretation process based on the information of the user command (volume UP), the device information obtained from the device information management unit 17, and the user information obtained from the user information management unit 15.

[0084] Here, an example of command intent interpretation processing will be described based on assumed information. Assume that the device information of each device is as follows. That is, the device information of the operation target device 5 (television receiver) indicates the location information (coordinate values) of the television, that the television is currently receiving and playing a broadcast, and that the volume output by the television is 50 dB (decibels). Furthermore, from the location information of the television and the building layout information held by the volume control device 1, the user intent interpretation unit 12 can determine that the television is located in the living room. Furthermore, the device information of the IoT device 7-1 (smart speaker) indicates the location information of the smart speaker and that the smart speaker is currently not outputting. From the location information of the smart speaker and the building layout information held by the volume control device 1, the user intent interpretation unit 12 can determine that the smart speaker is located in the living room. The device information of IoT device 7-2 (autonomous mobile vacuum cleaner) indicates the location information of the vacuum cleaner, that the vacuum cleaner's current operating mode is "high" (the operating mode is either "high" or "low"), and that the volume of the sound emitted by the vacuum cleaner (such as the motor sound and running sound in "high" operating mode) is 50 dB. From the location information of the vacuum cleaner and the building layout information held by the volume control device 1, the user intention interpretation unit 12 can determine that the vacuum cleaner is located in the living room. In addition, the user information includes the user's location information. From the user's location information and the building layout information held by the volume control device 1, the user intention interpretation unit 12 can determine that the user is located in the living room.

[0085] That is, the user intention interpretation unit 12 recognizes that the smart speaker (IoT device 7-1) is present in the same room as the user and the television (operation target device 5) but is not outputting sound. The user intention interpretation unit 12 also recognizes that the autonomous mobile vacuum cleaner (IoT device 7-2) is present in the same room as the user and the television (operation target device 5) and is operating in the "high" operating mode. The user intention interpretation unit 12 previously stores and can utilize knowledge (or rules) that sounds are less likely to be blocked in the same room. The user intention interpretation unit 12 also recognizes that the difference between the output volume of the television (50 dB), which is considered to be the target sound, and the volume of the sound emitted by the vacuum cleaner (50 dB), which is considered to be noise, is less than a predetermined threshold (e.g., 6 dB). In this situation, the user intention interpretation unit 12 interprets the intention of the user's command using a method described separately. That is, the user intention interpretation unit 12 can assume that the sound emitted by the autonomous mobile vacuum cleaner (IoT device 7-2) is interfering (being a noise) with the sound emitted by the television receiver (operation target device 5). Based on this assumption, the user intention interpretation unit 12 interprets the intention of the user's command to "Volume up" as "the user wants to make the television easier to hear." The user intention interpretation unit 12 passes the interpretation result to the device control determination unit 13.

[0086] Next, in step S8, the device control determination unit 13 of the volume control device 1 determines the content of control for each device based on the user intention passed from the user intention interpretation unit 12 (as described above, "make it easier for the user to hear the television"). In this example, the device control determination unit 13 searches for pre-stored knowledge information to obtain a means for ensuring a difference of 3 dB or more between the output volume of the television and the volume of the sound emitted by the vacuum cleaner. As a result, the device control determination unit 13 determines to lower the volume of the sound emitted by the vacuum cleaner and to move the vacuum cleaner from the room where the user and the television are located to another room. More specifically, the device control determination unit 13 changes the operation mode of the vacuum cleaner to "low" and determines to move the vacuum cleaner to another room. The device control determination unit 13 passes the determined items to the device control unit 14. The device control unit 14 generates a control signal to be sent to the vacuum cleaner to implement the determined items passed from the device control determination unit 13.

[0087] At the same time, the device control determination unit 13 determines to issue a command to increase the volume to the operation target device 5. Specifically, it determines to increase the volume of the sound output by the operation target device 5 to 56 dB. This increase in volume is also based on the user intention (to make it easier for the user to hear the TV) estimated by the user intention interpretation unit 12.

[0088] Next, in step S9, the device control unit 14 of the volume control device 1 transmits a control command to the IoT device 7-2 (autonomous mobile vacuum cleaner). The control unit 73 of the IoT device 7-2 receives this instruction information from the volume control device 1. The control command transmitted to the IoT device 7-2 in this step is control information indicating that the operation mode of the vacuum cleaner should be changed to "low" and that the vacuum cleaner should be moved from the living room to another room. Based on the received control command, the control unit 73 controls the operation of its own device from the next step onwards.

[0089] Next, in step S10, the control unit 73 of the IoT device 7-2 (autonomous mobile vacuum cleaner) controls the operation unit 77 so that the device itself moves. The operation unit 77 includes a motor for moving the device itself. The movement instruction from the volume control device 1 is based on the floor plan information described above and includes coordinate information of the destination (another room). Alternatively, the IoT device 7-2 itself may hold floor plan information and calculate coordinate information of the location of the destination (specified room). Through the control in this step, the IoT device 7-2 moves from the original living room to another room. The original living room and the other room are separated by a wall or the like, and moving between rooms reduces the impact of the sound emitted by the IoT device 7-2 on the user's ears.

[0090] Next, in step S11, the control unit 73 of the IoT device 7-2 (autonomous mobile vacuum cleaner) controls the operation unit 77 to change the operation mode of the device to "weak." The operation unit 77 includes a motor for the vacuum cleaner to suck in air, etc. Through the operation in this step, the operation mode of the IoT device 7-2 is switched to "weak," and the volume of the sound (motor sound, etc.) emitted by the IoT device 7-2 is reduced. The device status management unit 74 of the IoT device 7-2 updates the device information of the device. That is, the device information of the IoT device 7-2 includes information on the position (coordinates) of the device, and the information on the current operation mode of the device is updated to "weak."

[0091] Next, in step S12, the device information providing unit 75 of the IoT device 7-2 (autonomous mobile vacuum cleaner) notifies the volume control device 1 of the status after receiving the command in step S9. Specifically, the device information providing unit 75 notifies the volume control device 1 of at least the updated location information (coordinate values) of the IoT device 7-2 and the operating mode (low) of the IoT device 7-2. The device information acquiring unit 16 of the volume control device 1 receives the updated device information from the IoT device 7-2. The device information managing unit 17 of the volume control device 1 updates the device information of the IoT device 7-2 that it manages to the latest state.

[0092] Next, in step S13, the device control unit 14 of the volume control device 1 transmits a volume UP control command to the operation target device 5 (television receiver). This command is also based on the decision of the above-mentioned device control decision unit 13. The control unit 53 of the operation target device 5 receives this command from the volume control device 1. The control unit 53 controls the audio output unit 56 to increase the volume of the output audio from the television receiver.

[0093] Through the series of processes up to step S13, the IoT device 7-2 moves to another room, the operation mode of the IoT device 7-2 is changed to "weak" (the volume of the sound it emits becomes lower), and the audio output of the IoT device 7-1 remains stopped. Also, the volume of the audio output of the operation target device 5 is increased. In other words, control is automatically performed in accordance with the user's intention, which is the interpretation result.

[0094] 7, in step S14, the user control transmitter 31 of the user input device 3 transmits a volume up command (control signal) to the user control receiver 51 of the target device 5 (television receiver). The user control receiver 51 receives this command. Note that the trigger for this volume up command is a button operation by the user on the user input device 3 (television remote control device). In other words, even though the control up to step S13 has been executed, in this example the user still performs a volume up command. The target device 5 operates in the same manner as when it received the command in step S1.

[0095] Next, in step S15, the user control providing unit 52 of the target device 5 transmits information about the command (volume UP) received in step S14 to the volume control device 1. The user control receiving unit 11 of the volume control device 1 receives this information in the same way as in step S2. The user control receiving unit 11 passes the received information about the command (volume UP) to the user intention interpretation unit 12.

[0096] The volume control device 1 has already received the device information of each device in the processes of steps S4 and S12. That is, the device information management unit 17 of the volume control device 1 already holds the latest device information. The volume control device 1 has also received the user information in the process of step S6. That is, the user information management unit 15 of the volume control device 1 already holds the latest user information. Therefore, this time, the volume control device 1 can omit the steps for acquiring the device information and user information.

[0097] Next, in step S16, the user intention interpretation unit 12 of the volume control device 1 interprets and estimates the user's intention regarding the command acquired by the volume control device 1 in step S15, based on the information already obtained. The user intention interpretation unit 12 passes the user's intention, which is the result of the interpretation, to the device control determination unit 13.

[0098] Specifically, the user intention interpretation unit 12 interprets the user's intention as follows: The user intention interpretation unit 12 has acquired the latest device information for the IoT device 7-2 (autonomous mobile vacuum cleaner) and understands that this vacuum cleaner (IoT device 7-2) is operating in a room other than the living room where the user is located, in the "low" operating mode. Based on previously acquired knowledge information, such as the relative positions of the vacuum cleaner and the user and the fact that the vacuum cleaner is operating in the "low" operating mode, the user intention interpretation unit 12 understands that the volume of the sound emitted by the vacuum cleaner at the user's location is 40 dB. The user intention interpretation unit 12 also understands the volume of the television (operation target device 5). Based on both of these volumes, the user intention interpretation unit 12 interprets the user's intention as "The sound emitted by the vacuum cleaner is still too loud, so I would like to make the sound output by the television easier to hear."

[0099] Next, in step S17, the device control determination unit 13 of the volume control device 1 determines the control content based on the interpretation result of the user intention interpretation unit 12 in step S16. Note that the device control determination unit 13 already understands that a volume UP instruction has been issued to the television (operation target device 5) in step S13, and that further increasing the volume of the television's output sound will cause the volume of the entire environment to exceed a desired upper limit. The upper limit of the volume of the entire environment is determined based on the knowledge that the user is located inside a home (information previously held by the volume control device 1) and the specified upper limit of the volume for the home. Furthermore, if the volume of the output sound of the television (operation target device 5) has already reached the settable upper limit, the volume cannot be increased further. The device control determination unit 13 determines control that is consistent with the user's intention within these constraints. Specifically, the device control determination unit 13 determines the following control to make the sound output by the television easier for the user to hear. That is, the control content decided here is to maintain the current volume of the output sound of the television (operation target device 5) (as changed in step S13) and to stop the operation of the vacuum cleaner (IoT device 7-2). That is, the device control decider 13 decides to send a control command (an instruction to stop operation) to the vacuum cleaner (IoT device 7-2) and not to give any instruction to the television (operation target device 5) or the smart speaker (IoT device 7-1) (to keep them as they are). The device control decider 13 passes information on this decision content to the device control unit 14.

[0100] In addition, the reference below states that the ease of hearing the desired volume is determined by the difference between the volume of the noise and the volume of the desired speech (both units are dB (decibels)). [References] "UNI-PEX Sound Story: Speaker Volume," Uni-Pex Corporation, downloaded on December 25, 2021

[0101] Specifically, when the volume of the sound (noise) of a vacuum cleaner is used as a reference, if the volume of the output sound of a television is about +1 dB or more and less than +3 dB, the user will only recognize that "something can be heard." If the volume of the output sound of a television is about +3 dB, the user will be able to hear it to the extent that "the content of the broadcast can be understood." If the volume of the output sound of a television is +6 dB or more, the user will recognize that the content of the television audio "can be heard clearly." The user intention interpretation unit 12 and the device control determination unit 13 store in advance knowledge of how users recognize things based on such volume differences, and perform processing based on this knowledge information as well.

[0102] Next, in step S18, the device control unit 14 of the volume control device 1 transmits a control command to the IoT device 7-2 (autonomous mobile vacuum cleaner) in accordance with the control content determined in step S17. Specifically, as described above, this control command is to stop the operation of the vacuum cleaner (IoT device 7-2). The control unit 73 of the IoT device 7-2 receives this control command.

[0103] As described above, the device control unit 14 does not transmit any control command to the operation target device 5 (television) or the IoT device 7-1 (smart speaker) at this timing.

[0104] Next, in step S19, the control unit 73 of the IoT device 7-2 controls the operation unit 77 of its own device based on the control command received in step S18. Specifically, the control unit 73 stops the operation of its own device (vacuum cleaner). That is, as a result, the vacuum cleaner (IoT device 7-2) stops the suction motor. The vacuum cleaner (IoT device 7-2) also stops moving its own device using the movement motor. As a result, the vacuum cleaner (IoT device 7-2) stops making sounds. The device status management unit 74 of the IoT device 7-2 updates the device information of its own device. That is, the device status management unit 74 updates the device information so that the operation status is "stopped" and the movement status is "stopped." The device status management unit 74 also fixes the location information (coordinate values) of its own device in the device information to the position at the time when the movement was stopped.

[0105] Next, in step S20, the device information providing unit 75 of the IoT device 7-2 notifies the volume control device 1 of the latest status. Specifically, the device information providing unit 75 transmits device information indicating that at least the operating state is "stopped" and the moving state is "stopped" to the volume control device 1. The device information providing unit 75 may also transmit the latest position information (coordinate values) of the device itself to the volume control device 1.

[0106] As described above, according to the processing sequence of this first example, in the control system 100, the command "Volume Up" from the user input device 3 is used as a trigger, and the user intention interpretation unit 12 interprets the user's intention in the command according to each situation, and based on the interpretation result, it is possible to control the operation of not only the target device 5 but also the IoT device 7.

[0107] 8 is a schematic diagram showing an example of the configuration of device information held by the device information management unit 17 of the volume control device 1. The device information shown here is collected by the device information acquisition unit 16 from the operation target device 5 and the IoT device group 7G, and is managed in a database or the like held by the device information management unit 17. As shown in the figure, the device information held by the device information management unit 17 can be managed, for example, as data in a table format. One row of data in this table holds information corresponding to one device (the operation target device 5 or the IoT device 7). Note that the device information shown here is an example in which only the latest information of each device is held. The device information management unit 17 may be configured to hold not only the latest device information but also the history of device information for each device.

[0108] The device information in the illustrated configuration has data items such as device ID, operation target, type, volume level, operating status, position (coordinate values), room / section, direction as seen from the user, and distance from the user. The device ID is identification information for uniquely identifying each device within the control system 100. The operation target is information indicating whether the device is an operation target device. A device whose operation target value is "TRUE" is an operation target device. A device whose operation target value is "FALSE" is not an operation target device. In the example sequences shown in FIGS. 6 and 7, the operation target device value for the operation target device 5 (television set) is "TRUE." On the other hand, the operation target device values ​​for the IoT device 7-1 (smart speaker) and IoT device 7-2 (autonomous mobile vacuum cleaner) are "FALSE." The type is a name (general noun) that indicates the type of device. The volume level is the volume of the sound emitted by the device at a specific point in time. The volume may represent only the volume of the audio output from a speaker or the like, or may represent the volume of the audio output from a speaker or the like as well as the audio generated by the operation of a functioning unit (such as noise or static). The operating status represents the operating status of a device. Values ​​of the operating status include "playing," "stopped," and "operating." The operating status value may also include information about the device's operating mode (e.g., strong or weak). The position represents the coordinate value of the location where the device is located at a specific time. In the illustrated example, the position is represented as a three-dimensional coordinate. If the coordinate value of the height direction (z-axis) of the device is not required, the position may be represented as a two-dimensional coordinate. The room / section represents information about the room or section where the device is located at a specific time. The room / section includes information identifying the room or section, such as "living room." The room / section may be determined based on the location and floor plan information. The direction from the user's perspective and the distance from the user represent information about the relative positional relationship between the device and the user. The direction from the user's perspective represents information about the direction of the device relative to the user's position when viewed in a planar view. For example, true north is set to 0° (degrees), and directions can be expressed clockwise using numbers greater than or equal to 0° and less than 360°.The distance from the user indicates the distance of the device from the user (unit: centimeters (cm), for example) based on the user's position.

[0109] In the data example of FIG. 8, the first line (with a device ID of 100000) is data representing the status of the operation target device 5 (television receiver) in the explanations of FIGS. 6 and 7. The second line (with a device ID of 100001) is data representing the status of the IoT device 7-1 (smart speaker). The third line (with a device ID of 100002) is data representing the status of the IoT device 7-2 (autonomous mobile vacuum cleaner).

[0110] [Processing sequence (second example)] 9 and 10 are sequence diagrams showing a second example of a processing sequence in the control system 100. Both FIGS. 9 and 10 show a series of sequences. In the second example, it is assumed that the user input device 3 is a television remote control device, the operation target device 5 is a television set that receives signals from the remote control device, and the IoT device 7-1 is a smart speaker. However, the specific types of devices may be different. The processing sequence in this second example is processing when the user input device 3 sends a volume down command (control signal) to the operation target device 5. Below, the second example will be described with reference to this diagram.

[0111] First, in step S31 (FIG. 9), the user control transmitter 31 of the user input device 3 transmits a volume down command (control signal) to the user control receiver 51 of the target device 5. The user control receiver 51 receives this command. Note that the trigger for this volume down command is a button operation by the user who thinks that the volume of the audio output from the television is too loud.

[0112] Next, in step S32, the user control providing unit 52 of the target device 5 transmits information representing the volume down command received in step S31 to the user control receiving unit 11 of the volume control device 1. The user control receiving unit 11 receives the provided information of this command. That is, the user control receiving unit 11 obtains information that the target device 5 has received a volume down command. Note that in step S32, instead of obtaining the volume down command information from the target device 5, the user control receiving unit 11 may receive this volume down command directly from the user input device 3. In either case, the user control receiving unit 11 passes the received volume down command information to the user intention interpretation unit 12.

[0113] Next, in step S33, the device control unit 14 of the volume control device 1 requests device information from each of the operation target device 5 and the IoT device 7-1.

[0114] Next, in step S34, the device information acquisition unit 16 of the volume control device 1 receives device information provided from the device information provision unit 55 of the operation target device 5 and the device information provision unit 75 of the IoT device 7-1 in response to the device information request sent in step S33. The device information acquisition unit 16 passes the acquired device information of each device to the device information management unit 17. The device information management unit 17 can pass the acquired device information to the user intention interpretation unit 12. The device information managed by the device information management unit 17 is the same information as that described in the first example.

[0115] Next, in step S35, the user information management unit 15 of the volume control device 1 transmits a user information request to the user information transmission unit 32 of the user input device 3. The user information transmission unit 32 receives this request.

[0116] Next, in step S36, the user information transmission unit 32 of the user input device 3 transmits user information to the user information management unit 15 of the volume control device in response to the request received in step S35. The user information management unit 15 stores and manages the acquired user information. The user information management unit 15 passes the acquired user information to the user intention interpretation unit 12. The user information managed by the user information management unit 15 is as described in the first example. In other words, the user information includes information such as the user's location information, the user's preferred volume setting value, the user's preferred audio output device, and the type of content the user plays.

[0117] Next, in step S37, the user intention interpretation unit 12 of the volume control device 1 interprets and estimates the intention of the user command transmitted from the user input device 3. Here, the user intention interpretation unit 12 executes the intention interpretation process based on the information of the user command (volume DOWN), the device information obtained from the device information management unit 17, and the user information obtained from the user information management unit 15.

[0118] In this example, it is assumed that the user intention interpretation unit 12 interprets the intention as follows. Based on the device information acquired in step S34, the user intention interpretation unit 12 determines that audio is being output from both the operation target device 5 (television set) and the IoT device 7-1 (smart speaker). Similarly, the user intention interpretation unit 12 determines that the operation target device 5 (television set) and the IoT device 7-1 (smart speaker) are located in the same room (living room). Similarly, the user intention interpretation unit 12 determines that, at the user's location, the volume output from the operation target device 5 (television set), which is playing a broadcast program, is +6 dB higher than the volume output from the IoT device 7-1 (smart speaker), which is outputting audio (playing music, etc.). In this situation, since the user's command was to turn down the volume of the operation target device 5 (television set), the user intention interpretation unit 12 interprets the user's intention as wanting to make the sound from the IoT device 7-1 (smart speaker) easier to hear. The user intention interpretation unit 12 passes the interpretation result to the device control determination unit 13.

[0119] Next, in step S38, the device control determiner 13 of the volume control device 1 determines the content of control for each device based on the user intention passed from the user intention interpreter 12 (as described above, "make it easier for the user to hear the sound of the smart speaker"). Specifically, in this example, the device control determiner 13 determines to perform control such that the output volume of the IoT device 7-1 (smart speaker) is increased and the output volume of the operation target device 5 (television receiver) is decreased. The device control determiner 13 passes this determination to the device control unit 14. In order to realize the determination passed from the device control determiner 13, the device control unit 14 generates control signals to be transmitted to each of the IoT device 7-1 (smart speaker) and the operation target device 5 (television receiver).

[0120] Next, in step S39, the device control unit 14 of the volume control device 1 transmits a control command to the IoT device 7-1 (smart speaker). The control unit 73 of the IoT device 7-1 receives this instruction information from the volume control device 1. The control command transmitted to the IoT device 7-1 in this step is to control the volume up. Based on the received control command, the control unit 73 controls the operation of the IoT device 7-1 from the next step onwards.

[0121] Next, in step S40, the control unit 73 of the IoT device 7-1 (smart speaker) controls the audio output unit 76 of the IoT device 7-1 to increase the output volume. As a result, the audio output unit 76 of the IoT device 7-1 (smart speaker) increases the output volume. Also, the device status management unit 74 of the IoT device 7-1 updates the device information of the IoT device 7-1. That is, the value of the output volume of the IoT device 7-1 is updated to the value after the volume is increased as the device information of the IoT device 7-1.

[0122] Next, in step S41, the device information providing unit 75 of the IoT device 7-1 (smart speaker) notifies the volume control device 1 of the status (device information) after the volume UP control in step S40. Specifically, the device information providing unit 75 notifies the volume control device 1 of at least the updated volume setting value of its own device. The device information obtaining unit 16 of the volume control device 1 receives the device information from the IoT device 7-1. The device information management unit 17 of the volume control device 1 updates the device information of the IoT device 7-1 that it manages to the latest state.

[0123] Next, in step S42, the device control unit 14 of the volume control device 1 transmits a volume DOWN control command to the operation target device 5 (television receiver). As described above, this command is also based on the decision made by the device control decision unit 13. The control unit 53 of the operation target device 5 receives this command from the volume control device 1.

[0124] Next, in step S43, the control unit 53 of the operation target device 5 controls the audio output unit 56 to lower the volume of the audio output from the television receiver.

[0125] Through the series of processes up to step S43, the volume output by the IoT device 7-1 (smart speaker) is increased and the volume output by the operation target device 5 (television receiver) is decreased. In other words, control is automatically performed in accordance with the user's intention as interpreted by the user intention interpretation unit 12.

[0126] 10, in step S44, the user control transmitter 31 of the user input device 3 transmits a volume down command (control signal) to the user control receiver 51 of the target device 5 (television receiver). The user control receiver 51 receives this command. The trigger for this volume up command is a button operation on the user input device 3 (television remote control device) by the user.

[0127] Next, in step S45, the user control providing unit 52 of the operation target device 5 transmits information about the command (volume DOWN) received in step S44 to the volume control device 1. The user control receiving unit 11 of the volume control device 1 receives this information. The user control receiving unit 11 passes the received information about the command (volume DOWN) to the user intention interpretation unit 12.

[0128] The volume control device 1 has already received the latest device information of each device. The volume control device 1 has also received user information. Therefore, the volume control device 1 can omit the procedure for acquiring device information and user information this time.

[0129] Next, in step S46, the user intention interpretation unit 12 of the volume control device 1 interprets and estimates the user's intention regarding the command acquired by the volume control device 1 in step S45 based on the information already obtained.

[0130] Specifically, the user intention interpretation unit 12 interprets the user's intention as follows: The user intention interpretation unit 12 recognizes that the user, the operation target device 5 (television receiver), and the IoT device 7-1 (smart speaker) are in the same room (living room). The user intention interpretation unit 12 also recognizes that the volume of the sound output by the operation target device 5 (television receiver) at the location where the user is currently located is 6 dB or more lower than the volume of the sound output by the IoT device 7-1 (smart speaker). Here, 6 dB is the threshold value for the difference in volume between the two sounds. In this situation, when the user issues a volume down command, the user intention interpretation unit 12 interprets the user's intention as "the user wants to lower the volume of the entire environment in the room (make it quieter)." The user intention interpretation unit 12 passes the user's intention, which is the result of the interpretation, to the device control determination unit 13.

[0131] Next, in step S47, the device control determiner 13 of the volume control device 1 determines the control content based on the interpretation result of the user intention interpreter 12 in step S46. That is, based on the interpretation result of the intention of "I want to lower the volume of the entire environment in the room," the device control determiner 13 determines to lower the volume output by the operation target device 5 (television receiver) and also to lower the volume output by the IoT device 7-1 (smart speaker). The device control determiner 13 passes information on this determination content to the device control unit 14. Based on the determination passed from the device control determiner 13, the device control unit 14 creates commands to control each device to be controlled (each of the operation target device 5 and the IoT device 7-1).

[0132] Next, in step S48, the device control unit 14 of the volume control device 1 transmits a control command to the IoT device 7-1 (smart speaker). The control unit 73 of the IoT device 7-1 receives the instruction information from the volume control device 1. The control command transmitted to the IoT device 7-1 in this step is to control the volume down.

[0133] Next, in step S49, the control unit 73 of the IoT device 7-1 (smart speaker) controls the audio output unit 76 of the IoT device 7-1 to lower the output volume. As a result, the audio output unit 76 of the IoT device 7-1 (smart speaker) lowers the output volume. In addition, the device status management unit 74 of the IoT device 7-1 updates the device information of the IoT device 7-1. That is, the value of the output volume of the IoT device 7-1 is updated to the value after the volume is lowered as the device information of the IoT device 7-1.

[0134] Next, in step S50, the device information providing unit 75 of the IoT device 7-1 (smart speaker) notifies the volume control device 1 of the status (device information) after the volume DOWN control in step S49. Specifically, the device information providing unit 75 notifies the volume control device 1 of the updated output volume value of its own device. The device information obtaining unit 16 of the volume control device 1 receives the device information. The device information management unit 17 of the volume control device 1 updates the device information of the IoT device 7-1 that it manages to the latest state.

[0135] Next, in step S51, the device control unit 14 of the volume control device 1 transmits a volume DOWN control command to the operation target device 5 (television receiver). This command is also based on the decision of the device control decision unit 13. The control unit 53 of the operation target device 5 receives this command from the volume control device 1.

[0136] Next, in step S52, the control unit 53 of the operation target device 5 (television receiver) controls the audio output unit 56 to lower the volume of the audio output from the television receiver.

[0137] As explained above, in the processing sequence of this second example, in the control system 100, the command "Volume Down" from the user input device 3 is used as a trigger, and the user intention interpretation unit 12 interprets the user's intention in the command according to each situation, and based on the interpretation result, it is possible to control the operation of not only the target device 5 but also the IoT device 7.

[0138] [Processing sequence (third example)] FIG. 11 is a sequence diagram showing a third example of a processing sequence in the control system 100. In the third example, it is assumed that the user input device 3 is a television remote control device, the operation target device 5 is a television set that receives a signal from the remote control device, and the IoT device 7-1 is a smart speaker. However, the specific types of devices may be different. The processing sequence of this third example is a process when the user input device 3 sends a power-on command (control signal) to the operation target device 5. Below, the third example will be described with reference to this diagram.

[0139] First, in step S61, the user control transmitting unit 31 of the user input device 3 transmits a power ON command (control signal) to the user control receiving unit 51 of the operation target device 5. The user control receiving unit 51 receives this command. Note that the trigger for this power ON command is the user pressing the power ON / OFF button.

[0140] Next, in step S62, the user control providing unit 52 of the target device 5 transmits the information of the power ON command received in step S61 to the user control receiving unit 11 of the volume control device 1. The user control receiving unit 11 receives the provided information of this command. Note that in step S62, instead of obtaining the information of the power ON command from the target device 5, the user control receiving unit 11 may receive this power ON command directly from the user input device 3. In either case, the user control receiving unit 11 passes the received information of the power ON command to the user intention interpretation unit 12.

[0141] Next, in step S63, the device control unit 14 of the volume control device 1 requests device information from each of the operation target device 5 and the IoT device 7-1.

[0142] Next, in step S64, the device information acquisition unit 16 of the volume control device 1 receives device information provided from the device information provision unit 55 of the operation target device 5 and the device information provision unit 75 of the IoT device 7-1 in response to the device information request sent in step S63. The device information acquisition unit 16 passes the acquired device information of each device to the device information management unit 17. The device information management unit 17 can pass the acquired device information to the user intention interpretation unit 12.

[0143] Next, in step S65, the user information management unit 15 of the volume control device 1 transmits a user information request to the user information transmission unit 32 of the user input device 3. The user information transmission unit 32 receives this request.

[0144] Next, in step S66, the user information transmission unit 32 of the user input device 3 transmits user information to the user information management unit 15 of the volume control device in response to the request received in step S65. The user information management unit 15 stores and manages the acquired user information. The user information management unit 15 passes the acquired user information to the user intention interpretation unit 12.

[0145] Next, in step S67, the user intention interpretation unit 12 of the volume control device 1 interprets and estimates the intention of the user command transmitted from the user input device 3. Here, the user intention interpretation unit 12 executes the intention interpretation process based on the user command information (power ON), device information obtained from the device information management unit 17, and user information obtained from the user information management unit 15.

[0146] In this example, it is assumed that the user intention interpretation unit 12 interprets the intention as follows. The user intention interpretation unit 12 originally knows that the operation target device 5 (television receiver) is present in the living room and that its power is OFF. The user intention interpretation unit 12 also knows that the IoT device 7-1 (smart speaker) is present in the living room and that the IoT device 7-1 is outputting sound at a predetermined volume. The user intention interpretation unit 12 also knows the volume of the IoT device 7-1 (smart speaker) at that time.

[0147] Based on its prior knowledge, the user intention interpretation unit 12 determines that, when the operation target device 5 and the IoT device 7-1 are present in the same room, the user would want the volume of the IoT device 7-1 (smart speaker) to be lowered when the power of the operation target device 5 is changed from OFF to ON. The user intention interpretation unit 12 also determines that, in that situation, it is desirable for the volume of the sound output from the operation target device 5 (television receiver) to be at least +6 dB higher than the volume of the sound output from the IoT device 7-1 (smart speaker). The user intention interpretation unit 12 regards these determination results as the user's intention. In other words, the user intention interpretation unit 12 interprets the user's intention as "to make the sound output from the television easier to hear when the television is turned on." The user intention interpretation unit 12 communicates this estimated user intention to the device control determination unit 13.

[0148] Next, in step S68, the device control determiner 13 of the volume control device 1 determines the device control content based on the above user intention ("make it easier to hear the sound output from the television when the television is turned on") passed from the user intention interpreter 12. Specifically, the device control determiner 13 determines to perform control such that the power of the operation target device 5 (television receiver) is turned on and the output volume of the IoT device 7-1 (smart speaker) is lowered. The device control determiner 13 passes the determined content to the device control unit 14. In order to realize the determined content passed from the device control determiner 13, the device control unit 14 generates control signals to be transmitted to each of the IoT device 7-1 (smart speaker) and the operation target device 5 (television receiver).

[0149] Next, in step S69, the device control unit 14 of the volume control device 1 transmits the control command generated in step S68 to each of the operation target device 5 (television receiver) and the IoT device 7-1 (smart speaker). The control unit 53 of the operation target device 5 and the control unit 73 of the IoT device 7-1 each receive instruction information from the volume control device 1. That is, the control unit 53 of the operation target device 5 receives a signal instructing it to power on. Also, the control unit 73 of the IoT device 7-1 receives a signal instructing it to turn down the volume.

[0150] Next, in step S70, the control unit 53 of the operation target device 5 turns on the power of its own device (television receiver) based on the command received in step S69. As a result, the operation target device 5 (television receiver) receives the predetermined broadcast service, displays the video on the screen, and outputs the audio from the audio output unit 56. Note that the volume of the sound output from the operation target device 5 (television receiver) at this time may be controlled by the control command in step S69. The operation target device 5 updates and stores the device information of its own device to the latest state.

[0151] Next, in step S71, the control unit 73 of the IoT device 7-1 performs control to lower the volume of the sound output from the sound output unit 76 of the IoT device 7-1 (smart speaker) based on the command received in step S69. The IoT device 7-1 updates and stores the device information of the IoT device 7-1 to the latest state.

[0152] Next, in step S72, the operation target device 5 (television receiver) and the IoT device 7-1 (smart speaker) each notify the volume control device 1 of the status of their own device. That is, device information indicating the latest status of each device is transmitted to the volume control device 1. The device information acquisition unit 16 of the volume control device 1 receives the status notification transmitted from each of these devices. The device information acquisition unit 16 passes the device information indicating the latest status of each device contained in the received status notification to the device information management unit 17. The device information management unit 17 stores the latest status information of each device.

[0153] As described above, according to the processing sequence of this third example, in the control system 100, the "power on" command from the user input device 3 is used as a trigger, and the user intention interpretation unit 12 interprets the user's intention in the command, and is able to not only turn on the power of the target device 5 but also control the operation of the IoT device 7.

[0154] Next, the processing of the user intention interpretation unit 12 and the processing of the device control determination unit 13 in the volume control device 1 of this embodiment will be further explained. Each of the user intention interpretation unit 12 and the device control determination unit 13 determines the information to be output (control signal, etc.) based on the information input thereto. The input / output relationships between the user intention interpretation unit 12 and the device control determination unit 13 in each of the first, second, and third examples of the processing described in the sequence diagrams can be summarized as shown in Tables 1 to 5 below.

[0155] Each of Tables 1 to 5 is a table having the following items: input information, intention (interpretation result), and control content. Of these, the input information is information that forms the basis of the process in which the user intention interpretation unit 12 interprets the user's intention. The intention (interpretation result) is information that is output as a result of the interpretation process by the user intention interpretation unit 12. The intention (interpretation result) is also information that serves as input for the device control determination unit 13 to determine device control. The device control determination unit 13 determines the control content based on the intention (interpretation result). Note that the device control determination unit 13 may refer to the input information in the tables when determining the control content.

[0156] Table 1 below summarizes the processing of the user intention interpretation unit 12 in step S7 of the first example and the contents of the processing of the device control determination unit 13 related thereto.

[0157] [Table 1]

[0158] Table 2 below summarizes the processing of the user intention interpretation unit 12 in step S16 of the first example and the contents of the processing of the device control determination unit 13 related thereto.

[0159] [Table 2]

[0160] Table 3 below summarizes the processing of the user intention interpretation unit 12 in step S37 of the second example and the contents of the processing of the device control determination unit 13 related thereto.

[0161] [Table 3]

[0162] Table 4 below summarizes the processing of the user intention interpretation unit 12 in step S46 of the second example and the contents of the processing of the device control determination unit 13 related thereto.

[0163] [Table 4]

[0164] Table 5 below summarizes the processing of the user intention interpretation unit 12 in step S67 of the third example and the contents of the processing of the device control determination unit 13 related thereto.

[0165] [Table 5]

[0166] The user intention interpretation unit 12 can perform the process of interpreting the user's intention by using any one of the first to third techniques described below, or by using a combination of two or more of them.

[0167] [First Method: User Intention Interpretation Unit 12] The user intention interpretation unit 12 stores in advance the correspondence relationships between input patterns and output patterns, such as those exemplified in Tables 1 to 5. When the user intention interpretation unit 12 receives a command (such as volume up) from the user input device 3, it refers to the correspondence table and searches for a pattern that matches the input information (command, device information, user information). When the user intention interpretation unit 12 finds a pattern that matches the input information, it outputs the interpretation result of that pattern as the user's intention. The correspondence relationships between input and output are identified in advance using appropriate means. This identification of the correspondence relationships may be performed manually, for example.

[0168] [Second Method: User Intent Interpretation Unit 12] The user intention interpretation unit 12 stores in advance the correspondence between input patterns and output patterns, such as those exemplified in Tables 1 to 5. The user intention interpretation unit 12 also stores in advance statistics of outputs (intention interpretations) that users find favorable when given input information is assumed. These statistics are obtained, for example, by obtaining feedback information from the user (information indicating whether the intention interpretation was favorable or not). In other words, the user intention interpretation unit 12 obtains statistics of outputs (intention interpretations) that users find favorable based on the user's operation history. When a pattern that matches the input information is found, the user intention interpretation unit 12 refers to the statistics, selects an output pattern that users tend to find favorable as the intention interpretation result, and outputs it.

[0169] [Third Method: User Intent Interpretation Unit 12] The user intention interpretation unit 12 performs machine learning in advance on the correspondence relationships between input patterns and output patterns as exemplified in Tables 1 to 5, using positive example data and negative example data as training data. The user intention interpretation unit 12 internally stores a model obtained as a result of the machine learning. When given input information, the model outputs a user intention that tends to be similar to the positive examples in the training data. In other words, the user intention interpretation unit 12 performs a process of estimating the user's intention using the machine-learned model.

[0170] The device control determination unit 13 stores and holds in advance in a storage means such as a table a correspondence relationship between the user's intention and the control content for the device (the operation target device 5 or the IoT device 7). Depending on the specific intention (user's intention) output by the user intention interpretation unit 12, the device control determination unit 13 refers to the table or the like and determines the control content that matches the intention.

[0171] In the first to third processing sequence examples (Figures 6, 7, 9, 10, and 11), we have described the cases where the IoT device 7 is a smart speaker and an autonomous mobile vacuum cleaner. In addition to the above examples, various other devices can be controlled as the IoT device 7. For example, if the IoT device 7 is an air purifier, vacuum cleaner, electric toothbrush, refrigerator, microwave, rice cooker, blender, or ventilation fan, the volume of the sound emitted by the device can be controlled by changing the operating state (operating mode, intensity, etc.) of the device. Furthermore, if the IoT device 7 is a water heater that supplies hot water to a bathroom, etc., the volume of the water heater's operating sound can be controlled by increasing or decreasing the speed of the water heater (to relatively speed up the rise in water temperature) or decreasing the speed (to relatively slow down the rise in water temperature). Furthermore, if the IoT device 7 is an electric bed, the volume of the sound emitted by the electric bed can be increased or decreased by increasing or decreasing the speed at which the electric motor changes the inclination of the bed. If the IoT device 7 is a control device for a power management system, the overall volume of the operating noise of devices operating in a specific power usage system can be increased or decreased by increasing or decreasing the overall power consumption in that system. If the IoT device 7 is a gasoline-powered automobile, the volume of the engine's operating noise can be increased or decreased by increasing or decreasing the engine's rotation speed per unit time (including the engine rotation speed when idling while parked). If the IoT device 7 is a device that moves by itself (such as a mobile robot or drone), the volume of the sound emitted by the IoT device 7 as perceived by the user can be increased or decreased by simply moving the IoT device 7 to a neighboring room or to a more distant location.

[0172] FIG. 12 is a block diagram showing an example of the internal configuration of a computer included in each device constituting the control system of this embodiment. At least some of the functions of the volume control device 1, the user input device 3, the operation target device 5, and the IoT device 7 can be realized using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM stands for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. Input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via an input / output port 903. A bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from / to RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906.

[0173] At least some of the functions of the volume control device 1, the user input device 3, the operation target device 5, and the IoT device 7 in the above-described embodiments can be implemented by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB flash drives, as well as storage devices such as hard disks built into computer systems. In other words, a "computer-readable recording medium" may be a non-transitory computer-readable recording medium. Furthermore, the term "computer-readable recording medium" may also include media that temporarily and dynamically store programs, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a certain period of time, such as volatile memory within the computer systems that serve as the server or client in such cases. Furthermore, the above program may be one that realizes part of the functions described above, or may be one that can realize the functions described above in combination with a program already recorded in the computer system.

[0174] Although a number of embodiments have been described above, the present invention can also be embodied in the following modified examples.

[0175] [Variation 1] In the first to third examples of the processing sequence described above, the user input device 3 was a television remote control device. The user input device 3 may be other devices. For example, the user input device 3 may be a smartphone, a tablet device, a wristwatch device, a smart speaker, or the like. In any case, the user input device 3 transmits a command signal to the operation target device 5 using an infrared signal, an IP packet, or the like.

[0176] [Variation 2] In the above embodiment, the volume control device 1 controls the volume of sounds emitted by the operation target device 5 and the IoT device 7. As a variation, instead of controlling the volume, the control device may control other objects. The controlled objects here may be, for example, the amount of light, power consumption (or the amount of power consumption), or the amount of other resources.

[0177] [Variation 3] In the above embodiment, the case where the operation target device 5 is a television receiver has been described. However, the operation target device 5 may be any other device. The operation target device 5 may be, for example, an audio playback device, a light-emitting device, a car, an airplane, a drone, an electric kick scooter, or any other device that can be operated.

[0178] [Variation 4] In the above embodiment, the user intention interpretation unit 12 performs processing to interpret the user's intention regarding a command based on information about the command from the user, device information, and user information. As a variation, the user intention interpretation unit 12 may perform processing to interpret the user's intention regarding a command based on information about the command from the user and device information, rather than based on user information.

[0179] The above has described in detail an embodiment of the present invention (including modified examples) with reference to the drawings, but the specific configuration is not limited to this embodiment, and also includes designs within the scope that do not deviate from the gist of the present invention. [Industrial Applicability]

[0180] The present invention can be used as a system for controlling multiple devices in cooperation with each other in various places, such as residential environments, offices, exhibition halls, event venues, public spaces, etc. However, the scope of use of the present invention is not limited to the examples given here. [Explanation of symbols]

[0181] 1 Volume control device 3 User Input Devices 5. Devices to be operated 7 IoT devices 7-1 IoT devices (smart speakers) 7-2 IoT device (autonomous mobile vacuum cleaner) 7G IoT equipment group 11 User Control Receiver 12 User intent interpretation section 13 Device control decision unit 14 Equipment control section 15 User Information Management Department 16 Device information acquisition section 17 Device information management department 31 User Controlled Transmitter 32 User information transmission unit 51 User Control Receiver 52 User Control Provisioning Unit 53 Control Unit 54 Equipment Status Management Unit 55 Device information provision department 56 Audio output section 57 Operating unit 73 Control Unit 74 Equipment Status Management Unit 75 Device information provision department 76 Audio output section 77 Operating unit 100 Control System 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. a device status management unit that manages device information including information about how sounds emitted by the device sound to a user; a device information providing unit that transmits the device information to an external volume control device; a control unit that receives control information transmitted from the volume control device and that affects how a user hears the sound emitted by the device itself, and controls at least one of the volume of the sound output by the device itself and an operating unit that operates while emitting sound in the device itself based on the received control information; Equipment comprising:

2. The device information includes at least one of the following: location information of the device; if the device has an audio output unit, the volume level of the sound output by the audio output unit; information indicating the operating state of the device; if the device has a means of transportation, the direction of movement of the device; if there is a setting item related to the sound emitted by the device, the setting value for the setting item; and if the device is capable of setting the output volume, information on the volume setting value previously set by the user.

10. The device of claim 1.

3. a user control receiving unit that receives information on commands from a user input device to the device itself; a user control providing unit that transmits information on the command received by the user control receiving unit to the volume control device; Furthermore, The control unit does not immediately control at least one of the volume of the sound output by the device itself and the operating unit that operates while emitting sound in the device itself based on the command information received by the user control receiving unit, but controls at least one of the volume of the sound output by the device itself and the operating unit that operates while emitting sound in the device itself based on the control information transmitted by the volume control device based on the command information transmitted to the volume control device.

3. The device according to claim 1 or 2.

4. a device status management unit that manages device information including information about how sounds emitted by the device sound to a user; a device information providing unit that transmits the device information to an external volume control device; a control unit that receives control information transmitted from the volume control device and that affects how a user hears the sound emitted by the device itself, and controls at least one of the volume of the sound output by the device itself and an operating unit that operates while emitting sound in the device itself based on the received control information; A program that makes a computer function as an apparatus equipped with the above.

Images