Information processing device and system

Abstract

To acquire accurate information from a user's activity history.SOLUTION: An information processing device includes a communication part, and a control part for respectively acquiring an approaching history showing that a user approaches from a human detection sensor installed in a residence and an activity history of each user when carrying a terminal device from the terminal device by the communication part, and generating a user's activity history having the activity history by using the approaching history and the activity history having high correlation to the approaching history.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and system. [Background technology]

[0002] Various technologies have been proposed that utilize a user's behavior history for analyzing power consumption, managing the user's health, etc. In relation to such technologies, Patent Document 1 discloses a technology that detects the user's location using a human presence sensor installed in a home. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 2017-153850 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improving the accuracy of information obtained from user activity history.

[0005] The following describes an information processing device and the like that can acquire highly accurate information from a user's activity history. [Means for solving the problem]

[0006] The information processing device of the present disclosure has a communication unit, and a control unit that acquires, via the communication unit, a proximity history indicating the approach of a user from a human presence sensor installed in a residence, and an activity history for each user when carrying the terminal device from a terminal device, and generates a behavioral history of the user having the activity history using the proximity history and the activity history that has a high correlation with the proximity history.

[0007] The system disclosed herein is a system having a terminal device carried by a user, a human presence sensor installed in a residence, and an information processing device that communicates with the terminal device and the human presence sensor, wherein the information processing device acquires a proximity history from the human presence sensor indicating that the user has approached, and an activity history for each user when carrying the terminal device from the terminal device, and generates a behavioral history of the user having the activity history using the proximity history and the activity history that has a high correlation with the proximity history. [Effects of the Invention]

[0008] According to the information processing device and the like of the present disclosure, it is possible to acquire highly accurate information from the activity history of a user. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of the configuration of an information processing system. [Figure 2] FIG. 10 is a flowchart illustrating an example of the operation of the information processing device. [Figure 3] FIG. 10 is a diagram illustrating the correlation between the approach history and the activity history. [Figure 4] FIG. 10 is a diagram illustrating a display example of a behavior history. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes the embodiments.

[0011] 1 is a diagram illustrating an example of the configuration of a system including an information processing device according to an embodiment. The information processing system 1 includes one or more server devices 10, one or more terminal devices 12, and one or more human presence sensors 13, which are connected to each other via a network 11 so as to be able to communicate information with each other.

[0012] The server device 10 is, for example, a server computer that belongs to a cloud computing system or other computing system and functions as a server that implements various functions. The server device 10 may be configured of two or more server computers that are connected to each other so as to be able to communicate information and operate in cooperation with each other. The server device 10 corresponds to the "information processing device" in this embodiment.

[0013] The terminal device 12 is a terminal device carried by a user that detects vibrations, acceleration, etc. associated with the user's activities and has a communication function that can transmit the detection results via wired or wireless communication, such as a smartphone, smart watch, or wearable terminal.

[0014] The human presence sensor 13 is a sensor device that is installed in various locations within the user's residence and detects the approach of the user by, for example, reflecting infrared rays, sound waves, electromagnetic waves, etc., and is equipped with a communication function that can transmit the detection results via wired or wireless communication.

[0015] The network 11 is, for example, the Internet, but may also include an ad-hoc network, a local area network (LAN), a metropolitan area network (MAN), or other network, or any combination thereof.

[0016] In this embodiment, the information processing device, i.e., the server device 10, includes a communication unit 101 and a control unit 103 that acquires, via the communication unit 101, an approach history from a motion sensor 13 installed in the residence and an activity history from a terminal device 12 for each user while carrying the terminal device 12, and generates a behavior history for each user with an activity history using the approach history and the activity history that is highly correlated with the approach history. Here, the activity history is a history of steps detected from vibrations, acceleration, and the like associated with the user's movements. The behavior history also includes the user's movement trajectory and distance within the residence. When multiple users reside within a residence, the user corresponding to the detection result of the motion sensor 13 can be identified using the activity history that is highly correlated with the approach history. Therefore, by tracing the detection results of the motion sensor 13 for each position, a behavior history including each user's movement trajectory and step count can be generated. Such a behavior history is useful for understanding each user's exercise volume and managing their health. Furthermore, when analyzing the power consumption of each room in a residence, it becomes easier to identify which user is responsible for the power consumption. Furthermore, there is no need to use the user's location information, and there is no need to additionally provide a communication function between the human presence sensor 13 and the terminal device 12. Therefore, it is possible to acquire highly accurate information from the user's activity history with a simple configuration.

[0017] The configurations of the server device 10 and the terminal device 12 will now be described in detail.

[0018] The server device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the server device 10 is configured with two or more server computers, these components are appropriately arranged in the two or more computers.

[0019] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10 and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101 and communicates information with a terminal device 12 via the network 11.

[0020] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, that function as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained by the operation of the server device 10.

[0021] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0022] The input unit 105 includes one or more input interfaces. The input interfaces are, for example, physical keys, capacitance keys, a pointing device, a touch screen integrated with a display, or a microphone that accepts voice input. The input unit 105 accepts an operation to input information used in the operation of the server device 10 and sends the input information to the control unit 103.

[0023] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the server device 10.

[0024] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Alternatively, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Alternatively, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read by the server device 10 from the medium.

[0025] The terminal device 12 includes a communication unit 121, a storage unit 122, a control unit 123, and a detection unit .

[0026] The communication unit 121 includes a communication module compatible with wired or wireless LAN standards, a module compatible with mobile communication standards such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 121 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0027] The storage unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 122 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 122 stores information used in the operation of the control unit 123 and information obtained by the operation of the control unit 123.

[0028] The control unit 123 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), etc., or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 123 may have one or more dedicated circuits such as an FPGA, an ASIC, etc. The control unit 123 performs overall control of the operation of the terminal device 12 by operating according to a control / processing program or operating according to an operating procedure implemented as a circuit. The control unit 123 then transmits and receives various information to and from the server device 10, etc. via the communication unit 121, and performs the operation according to this embodiment.

[0029] The functions of the control unit 123 are realized by a processor included in the control unit 123 executing a control program. The control program is a program for causing the processor to function as the control unit 123. Alternatively, some or all of the functions of the control unit 123 may be realized by a dedicated circuit included in the control unit 123. Alternatively, the control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and read by the terminal device 12 from the medium.

[0030] The detection unit 124 includes one or more sensors. The sensors include, for example, an acceleration sensor, a gyro sensor, etc., and detect vibrations, acceleration, etc. that occur when the user walks while carrying the terminal device 12. The detection unit 124 sends the detection results to the control unit 123. Using these detection results, the control unit 123 detects the number of steps taken by the user.

[0031] 2 is a flowchart for explaining the operation procedure of the server device 10 in the information processing system 1. The procedure in FIG.

[0032] In step S20, the control unit 103 acquires an approach history and an activity history. The approach history is sent from each human presence sensor 13 to the server device 10 at an arbitrary period, for example, every few seconds to every few minutes. The approach history is accompanied by identification information of each human presence sensor 13. The control unit 103 acquires the approach history sent from each human presence sensor 13. The approach history is a discrete value indicating whether or not a user has approached or the degree of approach for each unit time, for example, every few milliseconds to every few seconds. Information on the detection time is attached to each discrete value. Furthermore, the activity history is sent from each terminal device 12 to the server device 10 at an arbitrary period, for example, every few seconds to every few minutes. The activity history is accompanied by identification information of each terminal device 12. The identification information of each terminal device 12 is linked to the identification information of each user and stored in advance in the storage unit 102. The control unit 103 acquires the activity history sent from each terminal device 12. The activity history is a discrete value indicating the number of steps taken by the user for each unit of time, for example, every few milliseconds to every few seconds. Each discrete value is accompanied by information about the time of detection.

[0033] In step S21, the control unit 103 derives the correlation between the approach history and the activity history. For each combination of the human presence sensor 13 and the terminal device 12, the control unit 103 derives the correlation between the approach history and the activity history obtained from each. For example, the control unit 103 multiplies the discrete values ​​of the approach history and the discrete values ​​of the activity history at the same time. If the multiplication result is greater than an arbitrary standard, that is, if the user's steps are detected when a certain human presence sensor 13 detects the approach of the user, the product of the discrete values ​​at the same time indicates a value equal to or greater than a certain value. If any of the discrete values ​​indicates a value that is not detected, for example, 0, the product of the discrete values ​​at the same time indicates a value less than a certain value, for example, 0.

[0034] FIG. 3 illustrates an example of the correlation between a user's proximity history and activity history. The horizontal axis represents the passage of time, and the vertical axis represents the product of discrete values. Graphs G1, G2, and G3 show the product of the proximity history from the motion sensors 13 installed in the living room, study, and bedroom, respectively, multiplied by the user's activity history. The magnitude of the product corresponds to the number of steps per unit time. The figure shows that the user's proximity history was obtained from the motion sensors 13 in the bedroom during time periods T1, T2, and T9, in the living room during time periods T2, and in the study during time periods T3, T5, and T7, and the product of the activity history was obtained. This indicates that the user was present in the living room, study, or bedroom during each time period. During time period T2, the motion sensor 13 obtained proximity history in both the living room and the bedroom, indicating that the user frequently traveled back and forth between the living room and the bedroom. It should be noted that during a time period where no proximity history was obtained from the motion sensor 13 but an activity history exceeding a certain threshold was obtained, it can be determined that the user was out.

[0035] In step S22, the control unit 103 generates a behavioral history for each user. The behavioral history is a movement path on a time axis. The control unit 103 generates the user's movement path over time by connecting in chronological order the locations of the human presence sensors 13 for which the product of the approach history and the activity history has been obtained. Such behavioral history information is generated for each user and associated with the identification information of each user and the identification information of the terminal device 12 of that user.

[0036] In step S23, the control unit 103 outputs the behavior history for each user. The control unit 103 sends the corresponding behavior history information to each user's terminal device 12. The behavior history information is, for example, text showing a movement route in chronological order, an image showing the behavior history, etc. Each terminal device 12 can receive the behavior history information and display it to the user.

[0037] FIG. 4 is an example of an illustrated behavior history. In FIG. 4, the positions of the motion sensors 13 and movement routes 41 are shown on a floor plan 40 of a residence. An image of the movement routes 41 is generated by estimation using an arbitrary algorithm based on the arrangement of the motion sensors 13. In addition, a display 43 of the movement time may be displayed at any time for each movement route 41. Furthermore, a display 42 of the number of steps taken for the movement, a movement distance derived from the number of steps and an arbitrarily set average stride length, etc. may be displayed. Furthermore, when it is determined that the user has gone out, a display 44 of the time of going out, the number of steps, and the movement distance may be displayed. The user can check such behavior history on each terminal device 12, and can use it to understand and manage their own exercise amount.

[0038] In a modified example, the control unit 103 of the server device 10 can identify the location of each user for each time period based on the behavioral history of multiple users living in the same residence, and associate the power consumption for each room for that time period with the user who was in that room during that time period. The power consumption is acquired from a power control system for the residence, etc. The server device 10 generates text information or an image indicating the association between the room, time period, power consumption, and user, and sends it to the terminal device 12, and the user can check the information output on the terminal device 12. This makes it possible to analyze which user is responsible for the power consumption in each residence, and use this information to help with power consumption.

[0039] In the above-described embodiment, the processing / control program that defines the operation of the control unit 123 of the terminal device 12 may be stored in the memory unit of the server device 10 or another server device and downloaded to each terminal device 12 via the network 11, or may be stored in a recording / storage medium readable by each terminal device 12 and read from the medium by each terminal device 12.

[0040] Although the embodiments have been described above based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined or divided into one. [Explanation of symbols]

[0041] 1. Information Processing Systems 10 Server device 11 Network 12 Terminal Equipment 101, 121 Communications Department 102, 122 Storage section 103, 123 Control section 124 Detection unit

Claims

1. The Communications Department and and a control unit that acquires, by the communication unit, an approach history indicating the approach of a user, including position information of each human presence sensor from the human presence sensors installed in the residence, and an activity history indicating the number of steps taken by each user when carrying the terminal device, and generates a behavior history of the user having the activity history, including a history of the user's position based on the position information of each human presence sensor, using the approach history and the activity history when a product of a discrete value of the approach history and a discrete value of the activity history at the same time in a time series is greater than a reference value, and sends information of the behavior history to the terminal device of the user. Information processing device.

2. In claim 1, The control unit outputs the behavior history. Information processing device

3. In claim 1, the control unit generates the behavior history without acquiring location information from the terminal device. Information processing device.

4. A system including a terminal device carried by a user, a human presence sensor installed in a residence, and an information processing device that communicates with the terminal device and the human presence sensor, the information processing device acquires, from the human presence sensors, an approach history indicating the approach of the user, including position information of each human presence sensor, and an activity history indicating the number of steps taken by each user when carrying the terminal device, and, using the approach history and the activity history when the product of a discrete value of the approach history and a discrete value of the activity history at the same time in a time series is greater than a reference, generates a behavior history of the user having the activity history, including a history of the user's position based on the position information of each human presence sensor, and sends information of the behavior history to the terminal device of the user; system.

Images