Information Processing Apparatus, Information Processing Method, and Program

Abstract

To enable a user using a mobile body to recognize in the mobile body an operation state of electrical device installed in a building.SOLUTION: In an information processing system including an information processing device mounted on a vehicle and a management device (such as a server device) connected via a building and a communication network, an information processing device 1 comprises: a user action recognition part 15 that recognizes an action of a user of a mobile body; an operation state acquisition part 17 that acquires an operation state of an electrical device installed in the building; and an information provision part 18 that displays the operation spate of the electrical device on a display part installed in the mobile body. When a specific action estimated by the user action recognition part 15 that the user starts traveling of the mobile body is detected, the operation state of the electrical device installed in the building by the operation state acquisition part 17 is acquired, and the operation state acquired is displayed by the information provision part 18.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, a method has been proposed for remotely controlling electrical devices such as household appliances by a user who uses a moving body such as a vehicle. For example, Patent Document 1 discloses a configuration in which information is transmitted from a navigation device of a vehicle driven by a user to a home-side device provided at the user's home, and the home-side device controls home appliances installed at home. The system of Patent Document 1 transmits an arrival prediction time by the navigation device of the vehicle when the user is driving the vehicle toward home, and the home-side device executes the pre-reserved control content in accordance with this arrival prediction time. Further, Patent Document 2 discloses a system for remotely controlling an air conditioner installed in a building. The system of Patent Document 2 outputs control data for controlling an air conditioner selected as an operation target device by a computer installed in a vehicle when the vehicle exists in a control area surrounding the building in which the air conditioner is installed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the configurations described in Patent Document 1 and Patent Document 2, control of the electrical equipment and air conditioning equipment in a building is executed. However, there has been a problem that it is not possible to handle the case where the user only needs to know the control content of the operating state of the electrical equipment. This is because it is not appropriate to make the user gaze at the screen that displays the operating state of the electrical equipment while the vehicle is in motion. Furthermore, it is not possible to handle the use case where the user determines the control content while viewing the operating state of the electrical equipment. The present invention has been made in view of such a background, and an object thereof is to enable a user who uses a moving body to know the operating state of electrical equipment installed in a building in the moving body.

Means for Solving the Problems

[0005] As a first aspect for achieving the above object, a user behavior recognition unit that recognizes the behavior of a user of a moving body, an operating state acquisition unit that acquires the operating state of electrical equipment installed in a building, and a display unit installed in the moving body, an information presentation unit that presents the operating state of the electrical equipment; A communication unit, When a specific behavior that is estimated by the user behavior recognition unit to cause the user to start running the moving body is detected, the operating state acquisition unit acquires the operating state of the electrical equipment, and the acquired operating state is displayed on the display unit by the information presentation unit. wherein the communication unit switches between and executes a first mode of connecting as a client to a wireless communication network including the electric device, and a second mode of enabling another device to be connected as a client to the communication unit, and when the operation state acquisition unit acquires the operation state of the electric device, the communication unit switches from the second mode to the first mode. An information processing apparatus is mentioned.

[0006] In the above information processing apparatus, the information presentation unit may be configured to display the operating states of a plurality of the electrical equipment side by side on the display unit for each of the electrical equipment.

[0007] In the above information processing apparatus , the An operation unit that presents by the information presentation unit and instructs switching of the operating state of the electrical equipment, a switching instruction reception unit that receives an instruction using the operation unit, and an instruction transmission unit that transmits an instruction to switch the operating state of the electrical equipment according to the instruction to the communication unit may be provided.

[0008] In the above information processing apparatus, the moving body is provided with an openable and closable door and a door lock mechanism for locking the door in the closed position, and the user behavior recognition unit may be configured to detect the specific behavior when detecting the unlocking of the door lock mechanism.

[0009] In the above information processing apparatus, a position information acquisition unit for acquiring the position of the moving body is provided, and when the specific behavior is detected by the user behavior recognition unit and the moving body is at a preset position, the information presentation unit may be configured to cause the display unit to display the operating state acquired by the user behavior recognition unit.

[0011] In the above information processing apparatus, the above-mentioned When the communication unit switches from the second mode to the first mode and the operating state acquisition unit acquires the operating state of the electrical equipment, after the operating state of the electrical equipment is acquired, the above-mentioned communication unit may be configured to switch from the first mode to the second mode.

[0012] In the above information processing apparatus, the communication unit is configured to be able to communicate with a portable device used by the user, and when the unlocking of the door lock mechanism is detected by the user behavior recognition unit, the operating state acquisition unit may be configured to cause the communication unit to communicate with the portable device to acquire the operating state of the electrical equipment. On In the information processing apparatus, the first mode may include a STA mode, and the second mode may include an AP mode.

[0013] As a second aspect for achieving the above object, when recognizing the behavior of a user of a moving body and detecting a specific behavior in which the user is estimated to start running the moving body, the operating state of electrical equipment installed in a building is acquired, and the operating state of the electrical equipment is displayed on a display unit installed in the moving body , the communication unit switches between and executes a first mode of connecting as a client to a wireless communication network including the electric device, and a second mode of enabling another device to be connected as a client to the communication unit, and when the operation state of the electric device is acquired, the communication unit switches from the second mode to the first mode. An information processing method can be mentioned.

[0014] As a third aspect for achieving the above object, a program executable by a computer that controls an information processing apparatus, wherein the computer recognizes the behavior of a user of a moving body, and when detecting a specific behavior estimated to cause the user to start running the moving body, acquires the operating state of an electric device installed in a building, and causes the operating state of the electric device to be displayed on a display unit installed in the moving body , the communication unit switches between and executes a first mode of connecting as a client to a wireless communication network including the electric device, and a second mode of enabling another device to be connected as a client to the communication unit, and when the operation state of the electric device is acquired, the communication unit switches from the second mode to the first mode. The program for this purpose may be mentioned.

Effect of the Invention

[0015] According to the above configuration, the user can efficiently use the electric device, and the convenience can be improved.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0017] [1. First Embodiment] [1-1. Overview of the Information Processing System] FIG. 1 is an explanatory diagram showing an overview of an information processing system including an information processing apparatus 1 and a management apparatus 200 according to the first embodiment. In the present embodiment, as an example of a moving body, a vehicle V which is a four-wheel automobile is shown.

[0018] Vehicle V in FIG. 1 is a moving body used by user U. Building 100 is a building managed by user U. Building 100 is, for example, the home or workplace of user U. The object managed by user U in building 100 may be a part of building 100. A plurality of electrical devices 102, 103, 104 are installed in building 100. As an example, electrical device 102 is an air conditioner, electrical device 103 is a smart speaker, and electrical device 104 is an electric kettle. A wireless access point 101 is installed in building 100. Wireless access point 101 forms a wireless communication network 110 using, for example, Wi-Fi (registered trademark). Electrical devices 102, 103, 104 installed in building 100 connect to wireless communication network 110 as clients.

[0019] Wireless access point 101 is connected to communication network 500 via a wired or wireless communication line. Communication network 500 may be any data communication network, including a wide area communication network including a public line network, a LAN (Local Area Network), etc.

[0020] A management device 200 is connected to communication network 500. Management device 200 is, for example, a server device and may be a cloud-type server. Management device 200 executes data communication with electrical devices 102, 103, 104 connected to wireless access point 101 via communication network 500. Further, management device 200 may execute data communication with various devices including portable device 150 described later via communication network 500.

[0021] Electrical devices 102, 103, 104 execute data communication with devices outside building 100 via wireless communication network 110. For example, electrical devices 102, 103, 104 communicate with management device 200 via wireless communication network 110.

[0022] For example, the electrical devices 102, 103, and 104 transmit information regarding their operating states to the management device 200 periodically or at the timing requested by the management device 200. The management device 200 has a management DB 210 that stores the information received from the electrical devices 102, 103, and 104.

[0023] FIG. 2 is a schematic diagram of the data stored in the management device 200. The management DB 210 illustrated in FIG. 2 stores operation information 220. The operation information 220 stores one record 221 for each electrical device. In the example of FIG. 1, three records 221 corresponding to each of the electrical devices 102, 103, and 104 are stored in the operation information 220.

[0024] The record 221 includes a device ID, a device type, a target function, and information indicating the state. The device ID is information that identifies each of the electrical devices 102, 103, and 104. The device type is information indicating the type of the electrical device. The record 221 may include the name assigned to the device instead of the device type. The target function is information indicating the function among the functions provided by the electrical device for which the operating state is to be recorded. The state is information indicating the operating state of the target function.

[0025] The record 221 is updated at a preset timing. For example, the record 221 is updated based on the information transmitted by the electrical devices 102, 103, and 104 at preset time intervals. Also, the electrical devices 102, 103, and 104 may transmit information to the management device 200 every time their operating state changes. The management device 200 updates the record 221 corresponding to the electrical device that transmitted the information every time it receives information from the electrical devices 102, 103, and 104.

[0026] Returning to FIG. 1, the wireless communication network 110 is not limited to Wi-Fi and may be formed by any wireless communication technology capable of performing data communication between the wireless access point 101 and the electrical devices 102, 103, and 104. Also, the wireless access point 101 may have a routing function, and the specific form of the wireless communication network 110 can be arbitrarily changed.

[0027] The vehicle V is equipped with the information processing device 1. The information processing device 1 may be a device fixedly installed in the vehicle V, or may be a device temporarily installed in the vehicle V. The information processing device 1 includes a touch panel 40 mounted on the vehicle V.

[0028] When the user U boards the vehicle V, the information processing system displays information regarding the operating states of the electrical devices 102, 103, and 104 on the touch panel 40 mounted on the vehicle V. In FIG. 1, the vehicle V is parked or stopped at a parking location Pa near the building 100, and when the user U starts moving from the parking location Pa by the vehicle V, information regarding the operating states of the electrical devices 102, 103, and 104 is displayed on the touch panel 40.

[0029] Also, a setting unit for remotely controlling the operating states of the electrical devices 102, 103, and 104 is displayed on the touch panel 40. When the user U operates the touch panel 40 and inputs to the setting unit, the functions of the electrical devices 102, 103, and 104 are controlled corresponding to the input.

[0030] The device indicated by reference numeral 150 in the figure is a mobile device or a portable computer called a handheld computer, such as a smartphone or a tablet computer. The portable device 150 of the present embodiment is a device used by the user U.

[0031] The information processing device 1 acquires the operation information of the electrical devices 102, 103, and 104 by wireless communication. The operation information is information indicating the operating states of the electrical devices 102, 103, and 104. The information processing device 1 may acquire all the operation information of the electrical devices 102, 103, and 104, or may acquire the operation information of some of the electrical devices. There are three modes in which the information processing device 1 acquires the operation information.

[0032] The first aspect is a mode in which the information processing apparatus 1 connects to the wireless access point 101 as a client and acquires operation information from the electric devices 102, 103, and 104 via the wireless communication network 110. In this case, the information processing apparatus 1 executes short-range wireless communication to connect to the wireless access point 101. In the first embodiment, the first aspect will be described.

[0033] The second aspect is a mode in which the information processing apparatus 1 communicates with the management apparatus 200 via the communication network 500 and acquires operation information from the management DB 210. In this case, the information processing apparatus 1 connects to the communication network 500 via a wireless communication line and acquires the record 221 stored in the management DB 210. In the second embodiment described later, the second aspect will be described.

[0034] The third aspect is that the information processing apparatus 1 connects to the portable device 150 by short-range wireless communication. The portable device 150 includes a short-range wireless communication function for communicating with the information processing apparatus 1 and a communication function for connecting to the communication network 500. The information processing apparatus 1 uses the communication function of the portable device 150 to connect to the communication network 500 and acquires the record 221 stored in the management DB 210. In the third embodiment described later, the third aspect will be described.

[0035] [1-2. Configuration of Information Processing Apparatus] FIG. 3 is a block diagram of the information processing apparatus 1 according to the first embodiment. The information processing apparatus 1 includes a CPU (Central Processing Unit) 10 and a memory 20 that stores programs and data. The memory 20 is a storage device composed of a semiconductor memory device or a magnetic recording medium. The memory 20 stores a control program 21 executed by the CPU 10 and various data processed by the CPU 10. The CPU 10 controls each part of the information processing apparatus 1 by executing the control program 21. The CPU 10 and the memory 20 may include an interface circuit (not shown). The memory 20 stores parking location information 22, communication management information 23, and operation information 24. The parking location information 22 is position information of a location registered as a location where the vehicle V parks. Specifically, the parking location information 22 is information indicating the position of the building 100 or an area within a predetermined distance from the building 100. For example, the parking lot Pa where the vehicle V parks while the user U is in the building 100 corresponds to the position indicated by the building 100. The communication management information 23 is information for the building 100 to connect to the wireless access point 101 by the function of the communication unit 45. The communication management information 23 includes, for example, the network name of the wireless communication network 110, the type of protocol used in the wireless communication network 110, the ID and access password for connecting to the wireless access point 101. The communication management information 23 may include the presence or absence of the wireless access point 101 to which the communication unit 45 can connect in the building 100 and the name of the wireless access point 101. The operation information 24 will be described later.

[0036] FIG. 3 shows, as functional parts configured by the CPU 10, a communication control part 11, an information presentation part 12, an input reception part 13, an audio processing part 14, a user behavior recognition part 15, a position information acquisition part 16, an operation state acquisition part 17, an information presentation part 18, and an information transmission part 19. Each of these functional parts is realized by the cooperation of software and hardware, for example, when the CPU 10 executes the control program 21. The functions of each of these parts will be described later.

[0037] The CPU 10 is connected to a touch panel 40, a speaker 41, a microphone 42, a GNSS (Global Navigation Satellite System) 43, a camera 44, and a communication unit 45.

[0038] The touch panel 40, the speaker 41, and the microphone 42 are installed, for example, on the dashboard of the vehicle V. The touch panel 40 has a configuration in which a liquid crystal display panel for displaying characters and images and a touch sensor for detecting contact with the liquid crystal display panel are superimposed. The touch panel 40 corresponds to an example of a display unit.

[0039] The GNSS 43 measures the current position of the vehicle V. When the vehicle V is equipped with a car navigation system (not shown), as the GNSS 43, a GPS (Global Positioning System) unit or the like provided in the car navigation system may be used.

[0040] The camera 44 is a digital camera that photographs the interior space of the vehicle V. The camera 44 is installed, for example, at the front of the interior of the vehicle V or on the ceiling of the interior of the vehicle V. The camera 44 may be an omnidirectional camera (360-degree camera) that photographs the front, rear, left, and right, or a wide-angle camera. The camera 44 corresponds to an example of a photographing unit.

[0041] The communication unit 45 includes a wireless communication device having an antenna, an RF circuit, an encoder, a decoder, etc. The communication unit 45 performs Wi-Fi and Bluetooth (registered trademark) communications. The communication unit 45 is connected to the wireless access point 101 via Wi-Fi and communicates with the electrical devices 102, 103, and 104 via the wireless communication network 110, which is a Wi-Fi network. The communication unit 45 corresponds to an example of a communication unit and a short-range wireless communication unit. Also, the communication unit 45 may be capable of performing wireless communication with the information processing device 1 via Wi-Fi or Bluetooth.

[0042] The communication unit 45 operates in two operation modes: the STA mode and the AP mode. These two operation modes are switched under the control of the CPU 10. The STA mode is an operation mode in which the communication unit 45 connects as a client to other devices. The STA mode can also be referred to as the client mode. The STA mode corresponds to an example of the first mode. In the STA mode, the communication unit 45 connects as a client to, for example, the wireless access point 101. The AP mode is an operation mode in which the communication unit 45 functions as an access point and connects other devices to the communication unit 45 as clients. The AP mode corresponds to an example of the second mode.

[0043] By operating the communication unit 45 in the AP mode, the information processing device 1 connects the portable device 150 to the information processing device 1 when, for example, the portable device 150 is present inside the vehicle V. In this case, the portable device 150 and the information processing device 1 execute wireless communication, and it becomes possible to display the screen of the application program executed by the portable device 150 on the touch panel 40. Also, when the user U operates the touch panel 40, the operation content is detected by the portable device 150, and it becomes possible for the portable device 150 to execute processing according to the operation. In the AP mode, for example, the portable device 150 executes a navigation application for guiding the route of the automobile and displays the screen of the navigation application on the touch panel 40. Thereby, even when the information processing device 1 does not have a route guidance function, the user U can use the car navigation function.

[0044] The information processing device 1 is connected to an ECU 50 (Electronic Control Unit). The ECU 50 is a control device that controls electronic devices mounted on the vehicle V. The information processing device 1 is connected to the electronic devices mounted on the vehicle V via the ECU 50. FIG. 3 shows an example in which the information processing device 1 is connected to a door switch 61, a door lock mechanism 62, a seat belt sensor 63, a seating sensor 64, a shift position sensor 65, an ignition sensor 67, and an ACC power supply 68 via the ECU 50. The vehicle V may be equipped with a plurality of ECUs 50. In this case, a plurality of ECUs 50 may be connected to the information processing device 1.

[0045] The door switch 61 is a switch-type sensor that detects whether the door provided in the vehicle V is open or not. The door switch 61 may be provided only on the door used by the person driving the vehicle V during getting on and off. The door lock mechanism 62 locks the door provided in the vehicle V in a closed state. When the vehicle V is provided with a plurality of doors, a plurality of door switches 61 and door lock mechanisms 62 may be provided corresponding to each door.

[0046] The seat belt sensor 63 detects the wearing state of the seat belt provided on the seat of the vehicle V. The seating sensor 64 detects whether a person is seated on the seat of the vehicle V. The seat belt sensor 63 and the seating sensor 64 may be provided on all seats provided in the vehicle V, or may be provided only on the driver's seat.

[0047] The shift position sensor 65 detects the shift position of the shift device provided in the vehicle V. The shift position is, for example, P (parking) used when parking, R (reverse) used when reversing, N (neutral), D (drive) used when driving, etc. The ignition sensor 67 detects the state of the ignition switch or the main switch of the vehicle V. The ACC power supply (accessory power supply) 68 is a power supply supplied from the power supply circuit provided in the vehicle V to electronic devices and auxiliary machines.

[0048] The communication control unit 11 controls the communication unit 45 to perform data communication. The communication control unit 11 causes the memory 20 to store the data received by the communication unit 45. The information presentation unit 12 outputs a control signal for display to the touch panel 40, and causes the touch panel 40 to display a screen including characters and images. The input reception unit 13 receives an input from a person boarding the vehicle V by detecting an operation on the touch panel 40. The audio processing unit 14 acquires digital audio data of the sound collected by the microphone 42. The audio processing unit 14 also drives the speaker 41 to output sound.

[0049] The user behavior recognition unit 15 recognizes the behavior of the person to be recognized. The person to be recognized is a person boarding the vehicle V or a person attempting to board the vehicle V by the user behavior recognition unit 15, and is, for example, the user U. The user behavior recognition unit 15 recognizes the behavior of the person to be recognized and detects that the person has performed a preset specific behavior.

[0050] Here, the specific behavior includes a behavior estimated to be when the user U starts moving by the vehicle V. The user behavior recognition unit 15 detects a specific behavior, for example, when (1) the user U has boarded the vehicle V, (2) the user U has started the vehicle V, or (3) the user U has performed an operation to start moving by the vehicle V.

[0051] As a specific example, when the user behavior recognition unit 15 detects that the door of the vehicle V has been opened, the door lock has been released, or the user U has taken a seat in the driver's seat, it detects that the user U has boarded the vehicle V. When the user behavior recognition unit 15 detects that the ignition switch has been turned on or the ACC power supply 68 has been turned on, it detects that the user U has started the vehicle V. When the user behavior recognition unit 15 detects that the seat belt in the driver's seat has been fastened or the shift position of the vehicle V has been switched from P to D or R, it detects that the user U has performed an operation to start moving. In addition, when the user behavior recognition unit 15 detects that the steering wheel of the vehicle V has been operated, the position information of the mobile terminal held by the user U has entered the predetermined range of the vehicle V, or the automatic driving function has been activated, it may also detect that the user U has performed an operation to start moving the vehicle V. Further, when the vehicle speed detected by the vehicle speed sensor of the vehicle V becomes greater than 0, the user behavior recognition unit 15 may detect that the user U has performed an operation to start moving the vehicle V.

[0052] The opening of the door of the vehicle V and the release of the door lock indicate the first operations performed by the user U to move the vehicle V. Therefore, preferably, when the vehicle V is stopped or parked and the user U is not in the passenger compartment of the vehicle V, the user behavior recognition unit 15 recognizes that a specific action has been performed when it detects that the door of the vehicle V has been opened or the door lock has been released.

[0053] The position information acquisition unit 16 acquires the position of the vehicle V measured by the GNSS 43. The operating state acquisition unit 17 acquires information indicating the operating states of the electrical devices 102, 103, and 104. In the first embodiment, the operating state acquisition unit 17 connects to the wireless communication network 110 through the communication unit 45, communicates with each of the electrical devices 102, 103, and 104 via the wireless communication network 110, and acquires information indicating the operating states. The operating state acquisition unit 17 generates operating information 24 based on the acquired information and stores it in the memory 20.

[0054] Based on the information acquired by the operation state acquisition unit 17, the information presentation unit 18 causes the touch panel 40 to display the operation states of the electric devices 102, 103, and 104. Specifically, for each of the electric devices 102, 103, and 104, the operation states are listed and displayed on the screen of the touch panel 40. The information presentation unit 18 causes the touch panel 40 to display a screen including a setting unit that is the target of the operation by the user U. The setting unit is a user interface for the user U to instruct to control the functions of the electric devices 102, 103, and 104. The information presentation unit 18 causes the touch panel 40 to display, for example, the operation states of the electric devices 102, 103, and 104 and the setting unit side by side.

[0055] The information transmission unit 19 transmits information to the electric devices 102, 103, and 104 through the communication unit 45. Specifically, when the user U instructs to control a function using the setting unit presented by the information presentation unit 18, the information transmission unit 19 transmits control data for controlling the function to the target electric device through the communication unit 45. This control data is transmitted from the communication unit 45 to the electric devices 102, 103, and 104 via the wireless communication network 110.

[0056] [1-3. Operations of the Information Processing Device] FIG. 4 is a flowchart showing the operations of the information processing device 1. FIG. 5 is a diagram showing an example of the screen displayed by the information processing device 1.

[0057] Among the operations of the information processing device 1, steps SA1 and SA2 are executed by the user action recognition unit 15, steps SA3, SA4, and SA5 are executed by the position information acquisition unit 16, SA6-SA9 and SA18 are executed by the communication control unit 11, SA10 and SA14 are executed by the operation state acquisition unit 17, steps SA11, SA15-SA17 are executed by the information presentation unit 18, and steps SA12 and SA13 are executed by the information transmission unit 19.

[0058] The operation shown in FIG. 4 is executed when the vehicle V is parked or stopped and the user U is not in the vehicle V.

[0059] The information processing apparatus 1 recognizes the behavior of the user U by detecting the states of various switches of the vehicle V etc. (step SA1), and determines whether it corresponds to a specific behavior (step SA2).

[0060] When it is determined that the behavior of the user U does not correspond to the specific behavior (step SA2; NO), the information processing apparatus 1 returns to step SA1. When it is determined that the behavior of the user U corresponds to the specific behavior (step SA2; YES), the information processing apparatus 1 acquires the position information of the vehicle V (step SA3). The information processing apparatus 1 collates the position information of the vehicle V with the position of the building 100 (step SA4). In step SA4, the information processing apparatus 1 collates the parking location information 22 with the position information acquired in step SA3.

[0061] The information processing apparatus 1 determines whether the vehicle V is located near the building 100 (step SA5). Here, when it is determined that the position of the vehicle V is not near the building 100 (step SA5; NO), the information processing apparatus 1 ends this process.

[0062] When it is determined that the position of the vehicle V is near the building 100 (step SA5; YES), the information processing apparatus 1 refers to the communication management information 23 and determines whether there is a wireless access point 101 to which the communication unit 45 can be connected in the building 100 (step SA6). When it is determined that there is no corresponding wireless access point 101 (step SA6; NO), the information processing apparatus 1 ends this process. When it is determined that there is a corresponding wireless access point 101 (step SA6; YES), the information processing apparatus 1 sets the operation mode of the communication unit 45 to the STA mode (step SA7). The default operation of the communication unit 45 is often the AP mode. In this case, the information processing apparatus 1 switches the operation mode of the communication unit 45 from the AP mode to the STA mode in step SA7.

[0063] The information processing apparatus 1 searches for the wireless access point 101 by the communication unit 45 (step SA8). The information processing apparatus 1 determines whether it is possible to connect to the wireless access point 101 (step SA9). In step SA9, for example, when the wireless access point 101 cannot be found or when the connection to the wireless access point 101 cannot be established, the information processing apparatus 1 makes a negative determination (step SA9; NO). In this case, the information processing apparatus 1 proceeds to step SA18 described later.

[0064] When it is possible to connect to the wireless access point 101 by the communication unit 45 (step SA9; YES), the information processing apparatus 1 connects to the wireless access point 101 and acquires information indicating the operating states of the electric devices 102, 103, and 104 (step SA10). In step SA10, the information processing apparatus 1 may sequentially execute communication with the electric device 102, communication with the electric device 103, and communication with the electric device 104 to inquire about the operating states. Also, when the wireless access point 101 has a function of managing the operating states of the electric devices 102, 103, and 104, the information processing apparatus 1 may communicate with the wireless access point 101 to acquire information indicating the operating states of the electric devices 102, 103, and 104. The information processing apparatus 1 generates operation information 24 based on the information acquired by the communication unit 45 and stores it in the memory 20.

[0065] The information processing apparatus 1 generates a user interface (UI) including a setting unit based on the operation information 24 and displays the user interface on the touch panel 40 (step SA11)

[0066] FIG. 5 is a diagram showing an example of a screen displayed by the information processing apparatus 1, which is a device operation screen 301. The device operation screen 301 is a user interface that presents information to the user U and accepts operations by the user U.

[0067] The device operation screen 301 includes a setting unit 310. In the setting unit 310, device operation buttons 311, 312, 313, 314, 315, and 316 are arranged. The device operation buttons 311, 312, 313, 314, 315, and 316 indicate the operating state of the functions of the electrical devices to be controlled. Furthermore, the device operation buttons 311, 312, 313, 314, 315, and 316 function as an operation unit for instructing the on / off switching of the functions to be controlled. The user U performs an operation of touching the device operation buttons 311, 312, 313, 314, 315, and 316, and this operation is detected by a touch sensor provided in the touch panel 40. The name of the function to be controlled is displayed on each of the device operation buttons 311, 312, 313, 314, 315, and 316.

[0068] A guide display 321 is displayed on the device operation screen 301. The guide display 321 includes characters and images for notifying the user U of the operation method of the setting unit 310.

[0069] For example, the device operation buttons 311 and 312 correspond to the functions of the electrical device 102. The electrical device 102 is an air conditioner, and the device operation button 311 controls the dehumidification function of the electrical device 102. The control target of the device operation button 312 is the timer function of the electrical device 102. In the example of FIG. 5, as the operating state of the electrical device 102, it is shown that the dehumidification function is off by the device operation button 311, and it is shown that the timer function is on by the device operation button 312.

[0070] When the information processing apparatus 1 receives a touch operation on the device operation button 311, it performs control to switch the on / off of the dehumidification function of the electrical device 102. That is, the user U can switch the on / off of the dehumidification function of the electrical device 102 by touching the device operation button 311. Similarly, the user U can switch the on / off of each function of the electrical devices 102, 103, and 104 displayed in the setting unit 310 by operating the device operation buttons 312, 313, 314, 315, and 316.

[0071] The device operation buttons 313 and 314 correspond to functions of the electric device 103. The electric device 103 is a smart speaker, and the device operation button 313 controls a pairing function of the electric device 103. The device operation button 314 controls a hands-free calling function of the electric device 103. In the example of Fig. 5, the operating state of the electric device 103 indicates that the pairing function and the hands-free calling function are off.

[0072] The device operation buttons 315 and 316 correspond to functions of the electric device 104. The electric device 104 is an electric kettle, and the device operation button 315 controls the keep-warm function of the electric device 104. The device operation button 316 controls the timer function of the electric device 104. In the example of Fig. 5, the device operation button 315 indicates that the keep-warm function is on, and the device operation button 316 indicates that the timer function is off, as the operating state of the electric device 104.

[0073] The information processing device 1 displays the device operation screen 301 when the specific behavior is detected. When the user U gets into the vehicle V, the device operation screen 301 is displayed on the touch panel 40, so the user U can learn the operation status of the electrical devices in the building 100 by looking at the device operation screen 301. For example, the device operation screen 301 in FIG. 5 notifies the user U that the warming function of the electric kettle is on. The user U can operate the device operation button 315 to switch off the warming function of the electric kettle. In this way, the user U can learn the operation status of the electrical devices 102, 103, and 104 before starting to move by the vehicle V, and can control the operation status of the electrical devices 102, 103, and 104. For example, if the user U gets into the vehicle V because he or she has forgotten to turn off the electrical devices or to operate the timer function, he or she can perform these operations inside the vehicle V.

[0074] Returning to FIG. 4, the information processing apparatus 1 determines whether there is an operation on the setting unit displayed in step SA11 (step SA12). If no operation is performed within a predetermined time (step SA12; NO), the information processing apparatus 1 proceeds to step SA17 described later.

[0075] If an operation on the setting unit is performed (step SA12; YES), the information processing apparatus 1 transmits control data to the electrical devices 102, 103, and 104 by the communication unit 45 according to the operation (step SA13). In step SA13, the information processing apparatus 1 transmits control data for controlling the function corresponding to the operation to the electrical device corresponding to the user U's operation on the setting unit among the electrical devices 102, 103, and 104.

[0076] The information processing apparatus 1 acquires information indicating the operating states of the electrical devices 102, 103, and 104 (step SA14), and updates the display of the user interface based on the acquired information (step SA15).

[0077] The information processing apparatus 1 determines whether a display end condition for ending the display of the user interface is satisfied (step SA16). The display end conditions include that the state without an operation on the touch panel 40 continues for a predetermined time or more, the ignition switch of the vehicle V is switched from off to on, the shift position of the vehicle V is switched to D, the vehicle speed of the vehicle V changes from 0 to a value greater than 0, and the like.

[0078] If the display end condition is not satisfied (step SA16; NO), the information processing apparatus 1 returns to step SA12. If the display end condition is satisfied (step SA16; YES), the information processing apparatus 1 ends the display of the user interface (step SA17). The information processing apparatus 1 returns the operation mode of the communication unit 45 to the state before switching in step SA7 (step SA18), and ends this process.

[0079] [2. Second Embodiment] FIG. 6 is a block diagram of the information processing apparatus 1 according to the second embodiment. In the second embodiment, the information processing apparatus 1 is mounted on a vehicle V including a TCU 51, and the information processing apparatus 1 is connected to the TCU 51. Except for the TCU 51, the configuration shown in FIG. 6 and the configuration of the information processing system including the information processing apparatus 1 are common to those of the first embodiment. The components common to those of the first embodiment are denoted by the same reference numerals, and the illustration and description thereof are omitted.

[0080] The TCU (Telematic Control Unit) 51 includes a wireless communication device having an antenna, an RF circuit, an encoder, a decoder, and the like. The wireless communication device included in the TCU 51 is, for example, a communication module compliant with the third generation, 3.9 generation, fourth generation, or fifth generation mobile communication standard established by 3GPP (Third Generation Partnership Project), and executes mobile data communication. The TCU 51 is connected to the communication network 500 through a mobile communication line and executes data communication with the management device 200.

[0081] The information processing apparatus 1 controls the TCU 51 by the function of the communication control unit 11 and executes data communication using the TCU 51. In this case, the communication management information 23 includes information for controlling the TCU 51 and access information for connecting to the management device 200 by the TCU 51. For example, the communication management information 23 includes the network address of the management device 200, the user ID for accessing the management device 200, the password, and the like.

[0082] FIG. 7 is a flowchart showing the operation of the information processing apparatus 1 according to the second embodiment. In the flowchart of FIG. 7, the operation steps common to those of FIG. 4 are denoted by the same step numbers, and the description thereof is omitted.

[0083] After step SA2, the information processing device 1 activates the TCU 51 and starts communication using the TCU 51 (step SA21), and connects to the management device 200 (step SA22). The information processing device 1 acquires the record 221 of the operation information 220 stored in the management DB 210 and stores it in the memory 20 as the operation information 24 (step SA23). Thereafter, the information processing device 1 generates a user interface having a setting unit based on the operation information 24 and displays it on the touch panel 40 (step SA11). The operations after step SA11 are the same as those in FIG. 4.

[0084] In the second embodiment, the information processing device 1 accesses the management device 200 using the TCU 51 provided in the vehicle V. Therefore, the information processing device 1 can be combined and used in a configuration in which the electrical devices 102, 103, and 104 do not have a function of directly communicating with the information processing device 1, or in a configuration in which the management device 200 manages the operating states of the electrical devices 102, 103, and 104. For example, when the user U has a contract for a cloud service in which the management device 200 manages the operating states of the electrical devices 102, 103, and 104, the information processing device 1 can be operated using this cloud service.

[0085] [3. Third Embodiment] FIG. 8 is a flowchart showing the operation of the information processing device 1 according to the third embodiment. The third embodiment shows another operation example of the information processing device 1. The configuration of the information processing device 1 and the information processing system according to the third embodiment may be either the configuration described in the first embodiment or the configuration described in the second embodiment. The configuration of the information processing device 1 and the information processing system is omitted in the third embodiment. Also, in the flowchart of FIG. 8, the operation steps common to FIG. 4 are denoted by the same step numbers and the description thereof is omitted.

[0086] In the third embodiment, the information processing apparatus 1 communicates with the portable device 150 through the communication unit 45. For example, the communication unit 45 executes Wi-Fi Direct or Bluetooth communication with the portable device 150 according to the control of the communication control unit 11. The connection form between the information processing apparatus 1 and the portable device 150 corresponds to so-called tethering. The information processing apparatus 1 communicates with the management apparatus 200 by connecting to the communication network 500 using the mobile communication function provided in the portable device 150.

[0087] In this case, the communication management information 23 includes information for connecting to the portable device 150 and access information for connecting to the management apparatus 200 by tethering. For example, the communication management information 23 includes an ID and a password for connecting to the portable device 150. Further, the communication management information 23 includes the network address of the management apparatus 200, a user ID, a password, etc. for accessing the management apparatus 200.

[0088] After step SA2, the information processing apparatus 1 connects to the portable device 150 through the communication unit 45 (step SA31). Subsequently, the information processing apparatus 1 connects to the management apparatus 200 in the same manner as the operation described in the second embodiment (step SA22). The information processing apparatus 1 acquires the record 221 of the operation information 220 stored in the management DB 210 and stores it in the memory 20 as the operation information 24 (step SA23). Thereafter, the information processing apparatus 1 generates a user interface having a setting unit based on the operation information 24 and displays it on the touch panel 40 (step SA11). The operations after step SA11 are the same as those in FIG. 4.

[0089] [4. Other Embodiments] The above embodiments show a specific example to which the present invention is applied, and do not limit the forms to which the invention is applied. For example, in the above embodiment, the operating states of the electrical devices 102, 103, and 104 were exemplified by a configuration that identifies whether the functions of the electrical devices 102, 103, and 104 are on or off. However, the operating state may be a more detailed function. For example, when the electrical device is an air conditioner, the set temperature of the air conditioner or the set time of the timer may be acquired as the operating state. Also, for example, a configuration may be such that the set temperature of the air conditioner or the set time of the timer can be set or changed using a user interface having a setting unit.

[0090] Also, for example, in the above embodiment, a four-wheel automobile was exemplified as an example of the vehicle V. However, the vehicle V can be various vehicles such as a two-wheeled vehicle or a cargo vehicle. Also, in the above embodiment, the information processing device 1 was described as displaying a user interface including a setting unit on the touch panel 40 and receiving a touch operation on the touch panel 40. This is an example. For example, after the information processing device 1 displays a user interface on the touch panel 40, it may receive an operation by voice using the microphone 42. Also, a configuration may be such that the operating state of the electrical device is presented by outputting voice from the speaker 41.

[0091] Also, the communication unit 45 is not limited to a configuration that executes Wi-Fi communication or Bluetooth communication, and may execute communication conforming to other short-range wireless communication standards. For example, a configuration that performs NFC (Near Field Communication) or Zigbee (registered trademark) communication may be used.

[0092] Also, the electrical devices 102, 103, and 104 whose operating states the information processing apparatus 1 acquires are not limited to being configured to perform Wi-Fi communication with the wireless access point 101. For example, in the building 100, a device management apparatus for managing the electrical devices 102, 103, and 104 may be installed in place of the wireless access point 101, and a configuration in which the device management apparatus and the electrical devices 102, 103, and 104 communicate with each other via Bluetooth or Zigbee may be adopted. In this case, the information processing apparatus 1 may acquire information indicating the operating states of the electrical devices 102, 103, and 104 by executing communication with the device management apparatus using the communication unit 45.

[0093] Also, the information processing apparatus 1 is not limited to being fixedly installed in the vehicle V. That is, if the touch panel 40, which is a device for presenting a user interface, is installed in the vehicle V, other configurations can be arbitrarily changed. For example, a part of the operations executed by the CPU 10 of the information processing apparatus 1 described in the above embodiment may be realized by a device similar to the portable device 150.

[0094] Note that FIG. 3 is a schematic diagram showing the functional configuration of the information processing apparatus 1 classified according to the main processing contents for easy understanding of the invention of the present application, and does not limit the configuration of the information processing apparatus 1. For example, the processing of the components included in the CPU 10 may be executed by one hardware unit or by a plurality of hardware units. Also, each of the processes shown in FIGS. 4, 7, and 8 may be executed by one program or by a plurality of programs.

[0095] Also, the control program 21 executed by the CPU 10 can also be realized by recording the control program 21 on a portable information recording medium. Examples of the information recording medium include magnetic recording media such as hard disks, optical recording media such as CDs, and semiconductor memory devices such as USB (Universal Serial Bus) memories and SSDs (Solid State Drives), but other recording media can also be used. The information processing apparatus 1 may read and execute the control program 21 from the information recording medium.

[0096] [5. Configuration Supported by the Above Embodiment] The above embodiment is a specific example of the following configuration.

[0097] (Item 1) An information processing apparatus comprising a user behavior recognition unit that recognizes the behavior of a user of a moving body, an operation state acquisition unit that acquires the operation state of electrical equipment installed in a building, and an information presentation unit that displays the operation state of the electrical equipment on a display unit installed on the moving body, wherein when a specific behavior is detected by the user behavior recognition unit and it is estimated that the user starts running the moving body, the operation state acquisition unit acquires the operation state of the electrical equipment, and the acquired operation state is displayed on the display unit by the information presentation unit. According to the information processing apparatus of Item 1, when the user starts running the moving body, the operation state of the electrical equipment installed in the building is displayed. Therefore, the operation state of the electrical equipment can be notified to the user before the user moves far away. Accordingly, the user can use the electrical equipment efficiently, and the convenience can be improved.

[0098] (Item 2) The information processing apparatus according to Item 1, wherein the information presentation unit displays the operation states of a plurality of the electrical equipment side by side on the display unit for each of the electrical equipment. According to the information processing apparatus of Item 2, the operation states of a plurality of electrical equipment can be displayed in an easy-to-see manner for the user.

[0099] (Item 3) A communication unit, an operation unit that is presented by the information presentation unit and instructs switching of the operating state of the electric device, a switching instruction reception unit that receives an instruction using the operation unit, and an instruction transmission unit that transmits an instruction to switch the operating state of the electric device to the communication unit according to the instruction. The information processing apparatus according to Item 1 or Item 2. According to the information processing apparatus of Item 3, a user can switch the operating state of the electric devices in a building.

[0100] (Item 4) The mobile body is provided with an openable and closable door and a door lock mechanism that locks the door in a closed position. The user behavior recognition unit detects the specific behavior when detecting unlocking of the door lock mechanism. The information processing apparatus according to Item 3. According to the information processing apparatus of Item 4, before the user moves the mobile body, the user can be notified of the operating state of the electric devices installed in the building.

[0101] (Item 5) A position information acquisition unit that acquires the position of the mobile body is provided. The information presentation unit causes the display unit to display the operating state acquired by the user behavior recognition unit when the specific behavior is detected by the user behavior recognition unit and the mobile body is at a preset position. The information processing apparatus according to Item 3 or Item 4. According to the information processing apparatus of Item 5, the operating state of the electric device can be notified to the user corresponding to the position of the vehicle used by the user. Therefore, when the usefulness of notifying the operating state of the electric devices in the building is high, such as in a place near the building, the operating state of the electric devices can be notified to the user, and unnecessary notifications to the user can be reduced.

[0102] (Item 6) The communication unit includes a short-range wireless communication unit that performs short-range wireless communication, and executes by switching between a first mode of connecting as a client to a short-range wireless communication network including the electrical device, and a second mode that enables other devices to be connected as clients to the short-range wireless communication unit. When the operation state acquisition unit acquires the operation state of the electrical device, the short-range wireless communication unit switches from the second mode to the first mode. The information processing apparatus according to any one of Items 3 to 5. According to the information processing apparatus of Item 6, it is possible to acquire the operation state of an electrical device by using a short-range wireless communication unit capable of connecting other devices as clients. For this reason, a device that performs short-range wireless communication with other devices can be used as an information processing apparatus that processes the operation state of an electrical device.

[0103] (Item 7) When the short-range wireless communication unit switches from the second mode to the first mode and the operation state acquisition unit acquires the operation state of the electrical device, the short-range wireless communication unit switches from the first mode to the second mode after the operation state of the electrical device is acquired. The information processing apparatus according to Item 6. According to the information processing apparatus of Item 7, by switching the operation mode of the short-range wireless communication unit after acquiring the operation state of the electrical device, it becomes possible to connect other devices as clients.

[0104] (Item 8) The communication unit is configured to be able to communicate with a portable device used by the user, and when the unlocking of the door lock mechanism is detected by the user behavior recognition unit, the operation state acquisition unit causes the communication unit to communicate with the portable device to acquire the operation state of the electrical device. The information processing apparatus according to Item 4. According to the information processing apparatus of Item 8, it is possible to acquire the operation state of an electrical device by communicating with a portable device. In addition, since the operation state of the electrical device is acquired and displayed when the unlocking of the door lock mechanism is detected, the user can be notified of the operation state of the electrical devices installed in the building before the user moves the mobile body.

[0105] (Item 9) An information processing method for recognizing the behavior of a user of a moving body, detecting a specific behavior that is estimated to cause the user to start running the moving body, obtaining the operating state of an electrical device installed in a building when the specific behavior is detected, and displaying the operating state of the electrical device on a display unit installed on the moving body. According to the information processing method of Item 9, when the user starts running the moving body, the operating state of the electrical device installed in the building is displayed. Therefore, the operating state of the electrical device can be notified to the user before the user moves far away. Accordingly, the user can use the electrical device efficiently, and the convenience can be improved.

[0106] (Item 10) A program executable by a computer that controls an information processing device, wherein the computer recognizes the behavior of a user of a moving body, and when detecting a specific behavior that is estimated to cause the user to start running the moving body, obtains the operating state of an electrical device installed in a building, and displays the operating state of the electrical device on a display unit installed on the moving body. According to the program of Item 10, when the user starts running the moving body, the operating state of the electrical device installed in the building is displayed. Therefore, the operating state of the electrical device can be notified to the user before the user moves far away. Accordingly, the user can use the electrical device efficiently, and the convenience can be improved.

Description of Reference Signs

[0107] 1… Information processing device, 10… CPU, 11… Communication control unit, 13… Input reception unit, 14… Audio processing unit, 15… User behavior recognition unit, 16… Location information acquisition unit, 17… Operating state acquisition unit, 18… Information presentation unit, 19… Information transmission unit, 20… Memory, 21… Control program (program), 22… Parking location information, 23… Communication management information, 24… Operating information, 40… Touch panel (display unit), 41… Speaker, 42… Microphone, 43… GNSS, 44… Camera (imaging unit), 45… Communication unit (communication unit), 100… Building, 101… Wireless access point, 102, 103, 104… Electrical equipment, 110… Wireless communication network, 150… Portable device, 200… Management device, 301… Device operation screen, 310… Setting unit, 500… Communication network, Pa… Parking location, U… User, V… Vehicle (mobile body).

Claims

1. A user behavior recognition unit that recognizes the behavior of a user of a moving body; An operating state acquisition unit that acquires the operating state of an electrical device installed in a building; An information presentation unit that displays the operating state of the electrical device on a display unit installed on the moving body; A communication unit, comprising: When a specific behavior is detected by the user behavior recognition unit, which is estimated to cause the user to start running the moving body, the operating state acquisition unit acquires the operating state of the electrical device, and the acquired operating state is displayed on the display unit by the information presentation unit; The communication unit switches between and executes a first mode in which it connects as a client to a wireless communication network including the electrical device, and a second mode in which another device can be connected to the communication unit as a client; An information processing apparatus, wherein when the operating state acquisition unit acquires the operating state of the electrical device, the communication unit switches from the second mode to the first mode.

2. The information processing apparatus according to claim 1, wherein the information presentation unit displays the operating states of a plurality of the electrical devices side by side on the display unit for each of the electrical devices.

3. An operation unit that presents the operating state of the electrical device and instructs switching of the operating state; A switching instruction reception unit that receives an instruction to use the operation unit; The information processing apparatus according to claim 1 or 2, further comprising: an instruction transmission unit that transmits an instruction to switch the operating state of the electrical device to the communication unit in response to the instruction.

4. The moving body is provided with an openable and closable door and a door lock mechanism that locks the door in a closed position; The user behavior recognition unit according to claim 3, wherein the specific behavior is detected when the unlocking of the door lock mechanism is detected.

5. Comprising a position information acquisition unit that acquires the position of the moving body; The information processing apparatus according to claim 3 or 4, wherein the information presentation unit displays the operating state acquired by the user behavior recognition unit on the display unit when the specific behavior is detected by the user behavior recognition unit and the moving body is at a preset position.

6. The information processing apparatus according to claim 1, wherein when the communication unit switches from the second mode to the first mode and the operating state acquisition unit acquires the operating state of the electrical device, the communication unit switches from the first mode to the second mode after the operating state of the electrical device is acquired.

7. The communication unit is configured to be able to communicate with a portable device used by the user. The operation state acquisition unit causes the communication unit to communicate with the portable device when the unlocking of the door lock mechanism is detected by the user behavior recognition unit, and acquires the operation state of the electrical equipment. The information processing apparatus according to claim 4.

8. The first mode includes the STA mode. The second mode includes the AP mode. The information processing apparatus according to any one of claims 1 to 7.

9. Recognize the behavior of the user of the moving body. When a specific behavior that is estimated that the user starts running the moving body is detected, acquire the operation state of the electrical equipment installed in the building. Display the operation state of the electrical equipment on a display unit installed in the moving body. The communication unit executes by switching between a first mode of connecting as a client to a wireless communication network including the electrical equipment and a second mode of enabling other devices to be connected to the communication unit as clients. An information processing method in which when acquiring the operation state of the electrical equipment, the communication unit switches from the second mode to the first mode.

10. A program executable by a computer that controls an information processing apparatus, by the computer, Recognize the behavior of the user of the moving body. When a specific behavior that is estimated that the user starts running the moving body is detected, acquire the operation state of the electrical equipment installed in the building. Display the operation state of the electrical equipment on a display unit installed in the moving body. The communication unit executes by switching between a first mode of connecting as a client to a wireless communication network including the electrical equipment and a second mode of enabling other devices to be connected to the communication unit as clients. When acquiring the operation state of the electrical equipment, the communication unit switches from the second mode to the first mode. A program for this purpose.

Images