Information processing device and system

The system correlates approach and activity histories from residential sensors and user terminals to enhance accuracy in tracking user activities and power consumption, improving health and energy management without requiring additional communication functions.

JP2024162506AActive Publication Date: 2024-11-21TOYOTA JIDOSHA KK
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023078068
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-21
Estimated Expiration
2043-05-10

AI Technical Summary

Technical Problem

Existing technologies lack accuracy in obtaining information from user activity history.

Method used

An information processing device and system that utilizes a human sensor installed in a residence and a terminal device carried by a user to acquire and correlate approach and activity histories, generating highly accurate user activity histories without requiring additional communication functions between the sensors and devices.

Benefits of technology

Enables highly accurate tracking of user activities and power consumption analysis within a residence, identifying user movements and exercise levels without needing user location information, thus enhancing health management and power consumption monitoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162506000001_ABST
    Figure 2024162506000001_ABST
Patent Text Reader

Abstract

To acquire accurate information from a user's activity history.SOLUTION: An information processing device includes a communication part, and a control part for respectively acquiring an approaching history showing that a user approaches from a human detection sensor installed in a residence and an activity history of each user when carrying a terminal device from the terminal device by the communication part, and generating a user's activity history having the activity history by using the approaching history and the activity history having high correlation to the approaching history.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and system. [Background technology]

[0002] Various technologies have been proposed that utilize a user's behavior history for analyzing power consumption, managing the user's health, etc. In relation to such technologies, Patent Literature 1 discloses a technology that detects the user's location by a human presence sensor installed in a house. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 2017-153850 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room to improve the accuracy of information obtained from a user's activity history.

[0005] The following describes an information processing device and the like that are capable of acquiring highly accurate information from a user's activity history. [Means for solving the problem]

[0006] The information processing device of the present disclosure has a communication unit, and a control unit that acquires, via the communication unit, a proximity history indicating the approach of a user from a human presence sensor installed in the residence, and an activity history for each user when carrying the terminal device from a terminal device, and generates a behavioral history of the user having the activity history using the proximity history and the activity history that has a high correlation with the proximity history.

[0007] The system disclosed herein is a system having a terminal device carried by a user, a human presence sensor installed in the residence, and an information processing device that communicates with the terminal device and the human presence sensor, wherein the information processing device acquires a proximity history from the human presence sensor indicating the approach of the user, and an activity history for each user when carrying the terminal device from the terminal device, and generates a behavioral history of the user having the activity history using the proximity history and the activity history that has a high correlation with the proximity history. Effect of the Invention

[0008] According to the information processing device and the like of the present disclosure, it is possible to acquire highly accurate information from a user's activity history. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a configuration of an information processing system. [Diagram 2] FIG. 11 is a flowchart illustrating an example of the operation of the information processing device. [Diagram 3] FIG. 13 is a diagram illustrating the correlation between an access history and an activity history. [Figure 4] FIG. 13 is a diagram showing an example of a display of an action history. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The following describes the embodiments.

[0011] 1 is a diagram showing an example of the configuration of a system including an information processing device according to an embodiment. The information processing system 1 includes one or more server devices 10, one or more terminal devices 12, and one or more motion sensors 13, which are connected to each other via a network 11 so as to be able to communicate information with each other.

[0012] The server device 10 is, for example, a server computer that belongs to a cloud computing system or other computing system and functions as a server that implements various functions. The server device 10 may be composed of two or more server computers that are connected to each other so as to be able to communicate information and operate in cooperation with each other. The server device 10 corresponds to the "information processing device" in this embodiment.

[0013] The terminal device 12 is a terminal device carried by a user, which detects vibrations, acceleration, etc. associated with the user's activity and has a communication function that can transmit the detection results via wired or wireless communication, such as a smartphone, a smart watch, or a wearable terminal.

[0014] The human presence sensor 13 is a sensor device that is installed in various places in the user's residence and detects the approach of the user by reflecting, for example, infrared rays, sound waves, electromagnetic waves, etc., and has a communication function that can transmit the detection results via wired or wireless communication.

[0015] Network 11 may be, for example, the Internet, but may also include an ad-hoc network, a Local Area Network (LAN), a Metropolitan Area Network (MAN), or other network, or any combination thereof.

[0016] In this embodiment, the information processing device, i.e., the server device 10, includes a communication unit 101 and a control unit 103 that acquires, by the communication unit 101, an approach history from a human sensor 13 installed in a residence and an activity history for each user when carrying the terminal device 12 from the terminal device 12, and generates a behavior history of a user having an activity history using the approach history and the activity history that is highly correlated with the approach history. Here, the activity history is a history of the number of steps detected from vibrations, acceleration, etc. accompanying the user's movements. The behavior history also includes the movement trajectory of the user in the residence and its distance. When multiple users exist in a residence, it is possible to identify which user corresponds to the detection result of the human sensor 13 using the activity history that is highly correlated with the approach history, so that it is possible to generate a behavior history including the number of steps along with the movement trajectory for each user by tracing the detection result for each position of the human sensor 13. Such a behavior history is useful for grasping the amount of exercise of each user and managing health. In addition, when analyzing the power consumption of each room in a residence, it becomes easier to identify which user is responsible for the power consumption. In addition, there is no need to use the user's location information, and no need to additionally provide a communication function between the human presence sensor 13 and the terminal device 12. Therefore, it is possible to obtain highly accurate information from the user's activity history with a simple configuration.

[0017] The configurations of the server device 10 and the terminal device 12 will now be described in detail.

[0018] The server device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the server device 10 is configured with two or more server computers, these components are appropriately arranged in the two or more computers.

[0019] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the server device 10, and transmits information obtained by the operation of the server device 10. The server device 10 is connected to a network 11 by the communication unit 101, and communicates information with a terminal device 12 via the network 11.

[0020] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them, functioning as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable ROM (EEPROM). The storage unit 102 stores information used in the operation of the server device 10 and information obtained by the operation of the server device 10.

[0021] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the server device 10 while controlling each unit of the server device 10.

[0022] The input unit 105 includes one or more input interfaces. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen integrated with a display, or a microphone that accepts voice input. The input unit 105 accepts an operation to input information used in the operation of the server device 10, and sends the input information to the control unit 103.

[0023] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the server device 10.

[0024] The functions of the server device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing a computer to function as the server device 10. Also, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. Also, the control program may be stored in a non-transitory recording / storage medium readable by the server device 10, and read by the server device 10 from the medium.

[0025] The terminal device 12 includes a communication unit 121, a storage unit 122, a control unit 123, and a detection unit .

[0026] The communication unit 121 has a communication module compatible with a wired or wireless LAN standard, a module compatible with a mobile communication standard such as LTE, 4G, 5G, etc. The terminal device 12 is connected to the network 11 by the communication unit 121 via a nearby router device or a mobile communication base station, and performs information communication with the server device 10, etc. via the network 11.

[0027] The memory unit 122 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memories are, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The memory unit 122 functions as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 122 stores information used in the operation of the control unit 123 and information obtained by the operation of the control unit 123.

[0028] The control unit 123 has, for example, one or more general-purpose processors such as a CPU, an MPU (Micro Processing Unit), or one or more dedicated processors such as a GPU specialized for a specific process. Alternatively, the control unit 123 may have one or more dedicated circuits such as an FPGA, an ASIC, or the like. The control unit 123 generally controls the operation of the terminal device 12 by operating according to a control / processing program, or operating according to an operating procedure implemented as a circuit. The control unit 123 transmits and receives various information to and from the server device 10, etc. via the communication unit 121, and executes the operation according to this embodiment.

[0029] The functions of the control unit 123 are realized by a processor included in the control unit 123 executing a control program. The control program is a program for causing the processor to function as the control unit 123. A part or all of the functions of the control unit 123 may be realized by a dedicated circuit included in the control unit 123. The control program may be stored in a non-transitory recording / storage medium readable by the terminal device 12, and may be read by the terminal device 12 from the medium.

[0030] The detection unit 124 includes one or more sensors. The sensors include, for example, an acceleration sensor, a gyro sensor, and the like, and detect vibrations, acceleration, and the like that accompany walking when the terminal device 12 is carried by the user. The detection unit 124 sends the detection result to the control unit 123. Using such detection result, the control unit 123 detects the number of steps of the user.

[0031] 2 is a flow chart for explaining the operation procedure of the server device 10 in the information processing system 1. The procedure in FIG.

[0032] In step S20, the control unit 103 acquires the approach history and the activity history. The approach history is sent from each human sensor 13 to the server device 10 at an arbitrary period, for example, at a period of several seconds to several minutes. The approach history is attached with the identification information of each human sensor 13. The control unit 103 acquires the approach history sent from each human sensor 13. The approach history is a discrete value indicating the presence or absence of the approach of the user or the degree of the approach for each unit time, for example, at a period of several milliseconds to several seconds. Information on the detection time is attached to each discrete value. In addition, the activity history is sent from each terminal device 12 to the server device 10 at an arbitrary period, for example, at a period of several seconds to several minutes. The activity history is attached with the identification information of each terminal device 12. The identification information of each terminal device 12 is linked to the identification information of each user and stored in advance in the storage unit 102. The control unit 103 acquires the activity history sent from each terminal device 12. The activity history is a discrete value indicating the number of steps taken by the user for each unit of time, for example, every few milliseconds to every few seconds. Each discrete value is accompanied by information on the detection time.

[0033] In step S21, the control unit 103 derives a correlation between the approach history and the activity history. For each combination of the human presence sensor 13 and the terminal device 12, the control unit 103 derives a correlation between the approach history and the activity history obtained from each. For example, the control unit 103 multiplies the discrete values ​​of the approach history and the discrete values ​​of the activity history at the same time. If the multiplication result is greater than an arbitrary reference, that is, if the user's steps are detected when the human presence sensor 13 detects the approach of the user, the product of the discrete values ​​at the same time indicates a value equal to or greater than a certain value, and if any of the discrete values ​​indicates a value that is not detected, for example, 0, the product of the discrete values ​​at the same time indicates a value less than a certain value, for example, 0.

[0034] FIG. 3 is a diagram showing an example of the correlation between the approach history and the activity history of one user. The horizontal axis corresponds to the passage of time, and the vertical axis corresponds to the product of discrete values. Graphs G1, G2, and G3 show the product of the approach history by the human sensor 13 installed in the living room, study, and bedroom, respectively, multiplied by the activity history of the user. The magnitude of the product corresponds to the number of steps per unit time. It is shown that the product with the activity history was obtained by obtaining the approach history of the user by the human sensor 13 in the bedroom in time periods T1, T2, and T9, in the living room in time periods T2, T4, T6, and T8, and in the study in time periods T3, T5, and T7. That is, it is shown that the user was present in the living room, study, or bedroom in each time period. Since the approach history by the human sensor 13 was obtained in the living room and the bedroom in time period T2, it is shown that the user frequently went back and forth between the living room and the bedroom. It should be noted that in a time period in which the approach history by the human sensor 13 is not obtained but an activity history equal to or greater than an arbitrary standard is obtained, it can be determined that the user has gone out.

[0035] In step S22, the control unit 103 generates a behavior history for each user. The behavior history is a movement route on a time axis. The control unit 103 generates a movement route of the user over time by connecting in a chronological order the arrangement of the human presence sensors 13 from which the product of the approach history and the activity history is obtained. Such behavior history information is generated for each user and is associated with the identification information of each user and the identification information of the terminal device 12 of that user.

[0036] In step S23, the control unit 103 outputs the behavior history for each user. The control unit 103 sends information on the corresponding behavior history to the terminal device 12 of each user. The information on the behavior history is, for example, text for showing a moving route in chronological order, an image for illustrating the behavior history, etc. Each terminal device 12 can receive the information on the behavior history and display it to the user.

[0037] FIG. 4 is an example of an illustrated behavior history. In FIG. 4, the location of the human sensor 13 and a movement route 41 are shown on a floor plan 40 of a residence. The image of the movement route 41 is generated by inferring it by an arbitrary algorithm based on the arrangement of the human sensor 13. In addition, a display 43 of the movement time may be displayed at any time for each movement route 41. Furthermore, a display 42 of the number of steps taken for the movement, the movement distance derived from the number of steps and an average stride length set arbitrarily, etc. may be displayed. Furthermore, when it is determined that the user has gone out, a display 44 of the time of going out, the number of steps, and the movement distance may be displayed. Since the user can check such behavior history on each terminal device 12, it becomes possible to utilize it for grasping and managing the amount of exercise of the user.

[0038] In a modified example, the control unit 103 of the server device 10 can identify the location of each user for each time period based on the behavioral history of multiple users living in the same residence, and associate the power consumption for each time period in each room with the user who was in that room during that time period. The power consumption is acquired from a power control system of the residence, etc. The server device 10 generates text information or an image indicating the association of the room, time period, power consumption, and user, and sends it to the terminal device 12, and the user can confirm the information output by the terminal device 12. In this way, it is possible to analyze which user is responsible for the power consumption in each residence, and use this information to help with power consumption.

[0039] In the above-described embodiment, the processing / control program that specifies the operation of the control unit 123 of the terminal device 12 may be stored in a memory unit of the server device 10 or another server device and downloaded to each terminal device 12 via the network 11, or may be stored in a recording / storage medium readable by each terminal device 12 and read from the medium by each terminal device 12.

[0040] In the above, the embodiment has been described based on the drawings and examples, but it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined into one or divided into one. [Explanation of symbols]

[0041] 1. Information Processing Systems 10. Server device 11 Network 12 Terminal Equipment 101, 121 Communications Department 102, 122 Storage section 103, 123 Control section 124 Detection unit

Claims

1. The Communications Department and and a control unit that acquires, by the communication unit, an approach history indicating the approach of a user from a human presence sensor installed in the residence and an activity history for each user when carrying the terminal device from the terminal device, and generates a behavior history of the user having the activity history by using the approach history and the activity history that has a high correlation with the approach history. Information processing device.

2. In claim 1, The approach history includes position information of each human sensor, and the behavior history includes a history of the user's position based on the position information. Information processing device.

3. In claim 1, The control unit outputs the behavior history. Information processing device

4. In claim 1, The control unit generates the behavior history without acquiring location information from the terminal device. Information processing device.

5. A system including a terminal device carried by a user, a human presence sensor installed in a residence, and an information processing device that communicates with the terminal device and the human presence sensor, The information processing device acquires, from the human presence sensor, an approach history indicating that the user has approached, and from the terminal device, an activity history of each user when carrying the terminal device, and generates a behavior history of the user having the activity history using the approach history and the activity history having a high correlation with the approach history. system.

Citation Information

Patent Citations

  • Security system

    JP2006259953A

  • Action tracking system, action tracking method, and action tracking program

    JP2010108037A

  • Living body monitoring system and living body monitoring method

    JP2017153850A