Action acquisition device, action acquisition method, and program

Abstract

To solve the conventional problem in which an action of a user cannot be estimated by using position information, and the like, associated with time.SOLUTION: An action acquisition device 3 can estimate an action of a user using position information, and the like, associated with time, the action acquisition device including: a time acquisition unit 331 for acquiring a time; a position acquisition unit 332 for acquiring position information associated with the time; an action estimation unit 335 which uses two or more pieces of action source information each including the position information associated with the time, to acquire action information specifying an action of a user in a time slot specified by the time of each of the two or more pieces of action source information; and an action output unit 341 for outputting the action information in the time slot.SELECTED DRAWING: Figure 18

Description

Technical Field

[0001] The present invention relates to a behavior acquisition device or the like that acquires and outputs the behavior information and time zone of a user by using the position information of the user or the like.

Background Art

[0002] Conventionally, there has been a behavior determination system that determines the behavior of a resident, such as sleep, by using energy consumption data (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, the behavior of the user could not be appropriately estimated.

Means for Solving the Problems

[0005] The behavior acquisition device of the first invention includes a time acquisition unit that acquires time, a position acquisition unit that acquires position information associated with the time, and a behavior estimation unit that acquires behavior information that specifies the behavior of the user in the time zone specified by the time included in two or more pieces of behavior source information including the position information associated with the time, and a behavior output unit that outputs the behavior information in the time zone.

[0006] With such a configuration, the behavior of the user can be estimated by using the position information associated with the time.

[0007] Further, the action acquisition device of the second invention further includes an activity acquisition unit that acquires the user's activity data associated with time, or a vital acquisition unit that acquires one or more types of the user's vital data associated with time. The action estimation unit is an action acquisition device that acquires action information in a time zone using two or more pieces of action source information including the position information associated with time, the activity data, or one or more types of vital data.

[0008] With such a configuration, the user's actions can be estimated using the position information and the user's body data. Note that the body data is, for example, activity data or vital data.

[0009] Further, the action acquisition device of the third invention further includes an activity acquisition unit that acquires the user's activity data associated with time, and a vital acquisition unit that acquires one or more types of the user's vital data associated with time. The action estimation unit is an action acquisition device that acquires action information in a time zone using two or more pieces of action source information including the position information associated with time, the activity data, and one or more types of vital data.

[0010] With such a configuration, the user's actions can be estimated using the position information and the user's body data.

[0011] Further, the action acquisition device of the fourth invention, with respect to any one of the first to third inventions, includes an emotion estimation unit that acquires emotion information regarding the user's emotion in a time zone using action source information, and the action estimation unit is an action acquisition device that also uses the emotion information to acquire action information in a time zone.

[0012] With such a configuration, the user's actions can be estimated with higher accuracy using the emotion information as well.

[0013] Further, the action acquisition device of the fifth invention, with respect to any one of the first to third inventions, the action estimation unit is an action acquisition device that also uses one or more pieces of past action information including the previous action information to acquire action information in a time zone.

[0014] With such a configuration, by using past action information as well, the actions of the user can be estimated with higher accuracy.

[0015] In addition, the action acquisition device of the sixth invention of the present invention further includes, with respect to the first invention, an emotion estimation unit that acquires emotion information regarding the emotion of the user in a time zone by using action information or action source information, and an emotion output unit that outputs the emotion information.

[0016] With such a configuration, the emotion at the time of the user's action can be estimated.

[0017] In addition, the action acquisition device of the seventh invention of the present invention, with respect to the sixth invention, the emotion estimation unit uses the learning information of the learning management unit in which learning information based on two or more pieces of teacher data having one or two or more pieces of action source information or action information and emotion information is stored, and the action information acquired by the action estimation unit or the action source information that is the source of the acquired action information to acquire emotion information in a time zone.

[0018] With such a configuration, the emotion at the time of the user's action can be estimated with higher accuracy.

[0019] In addition, the action acquisition device of the eighth invention of the present invention, with respect to any one of the first to seventh inventions, the action estimation unit uses the learning information of the learning management unit in which learning information based on two or more pieces of teacher data having one or two or more pieces of action source information and action information is stored, and two or more pieces of action source information including position information and activity data associated with a time to acquire action information in a time zone.

[0020] With such a configuration, by using past records, the actions of the user can be estimated with higher accuracy.

[0021] In addition, the action acquisition device of the ninth invention of the present invention, with respect to the seventh invention, the action information included in at least one piece of teacher data among two or more pieces of teacher data is the action information input by the user.

[0022] With such a configuration, the user's behavior can be estimated more accurately using past records.

[0023] Also, the behavior acquisition device of the tenth invention is a behavior acquisition device in which, with respect to the ninth invention, the behavior information included in at least two or more pieces of teacher data out of two or more pieces of teacher data is behavior information for two or more users.

[0024] With such a configuration, the user's behavior can be estimated more accurately using the past records of two or more users.

[0025] Also, the behavior acquisition device of the eleventh invention further includes a location acquisition unit that refers to a map management unit storing a map having location information corresponding to the position information, and acquires the location information corresponding to the position information acquired by the position acquisition unit. The behavior output unit is a behavior acquisition device that outputs the location information acquired by the location acquisition unit when the behavior estimation unit cannot acquire the behavior information.

[0026] With such a configuration, when the user's behavior cannot be estimated, the location where the user was can be output.

[0027] Also, the behavior acquisition device of the twelfth invention includes, with respect to the first invention, a receiving unit that receives radio waves including a device identifier for identifying a communication device from three or more communication devices, an intensity acquisition unit that acquires time-series radio wave intensities for each of the three or more communication devices, and a type determination unit that determines whether each of the three or more communication devices is a fixed terminal fixed in place or a mobile terminal moving, using the time-series radio wave intensities acquired by the intensity acquisition unit. The position acquisition unit is a behavior acquisition device that acquires indoor position information using the radio wave intensities of the three or more communication devices determined by the type determination unit to be fixed terminals.

[0028] With such a configuration, the user's behavior can be estimated using indoor position information and the user's activity data.

[0029] In addition, the action acquisition device of the thirteenth invention further includes an accumulation unit that, for the first invention, associates the action information acquired by the action estimation unit with time zones and accumulates it in an action management unit in which the action information corresponding to each of two or more time zones is stored. The action information can be discriminated as to whether it is determined action information or not. The action output unit is an action acquisition device that outputs two or more pieces of action information in such a way that the determined action information and the undetermined action information can be visually distinguished.

[0030] With such a configuration, it is possible to easily confirm the estimated action information and the determined action information.

Effect of the Invention

[0031] According to the action acquisition device of the present invention, the action of the user can be estimated using the position information and the activity data of the user.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Modes for Carrying Out the Invention

[0033] Hereinafter, embodiments of the action acquisition device and the like will be described with reference to the drawings. In the embodiments, components denoted by the same reference numerals perform the same operations, and thus the description may be omitted again.

[0034] (Embodiment 1) In the present embodiment, a location information production device will be described. The location information production device is a device that acquires location information described later for a specific location.

[0035] In this specification, the fact that information X is associated with information Y means that information Y can be acquired from information X, or information X can be acquired from information Y, and the method of the association is not limited. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0036] In addition, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag for information Z, etc., as long as information Z can be accessed.

[0037] FIG. 1 is a conceptual diagram of an information system A including a location information production device 1 according to the present embodiment. The information system A includes a location information production device 1 and three or more communication devices B.

[0038] Each of the three or more communication devices B is a device that transmits radio waves to other devices such as the location information production device 1. The communication device B transmits a device identifier for identifying the communication device B to other devices. The communication device B is, for example, a Wi-fi router or a communication device using BLE (Bluetooth Low Energy), but it doesn't matter.

[0039] FIG. 2 is a block diagram of the location information production device 1 according to the present embodiment. The location information production device 1 includes a storage unit 11, a reception unit 12, a receiving unit 13, and a processing unit 14. The reception unit 12 includes a position reception unit 121. The processing unit 14 includes an intensity acquisition unit 141, a type determination unit 142, and an accumulation unit 143.

[0040] The reception unit 12 receives various instructions and information. The various instructions and information are, for example, the location information described later. The input means for the various instructions and information can be anything, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0041] The position reception unit 121 receives the position information of a specific location. The position reception unit 121 usually receives the position information of three or more specific locations. The position reception unit 121 receives, for example, the position information that is an input by the user. The position reception unit 121 reads the position information from, for example, the storage unit 11.

[0042] The specific location can be a specific location indoors, but it can also be a specific location outdoors. Here, the location information is information that identifies the location indoors or outdoors. The location information is, for example, three-dimensional coordinate values (x, y, z) indicating the relative position indoors or outdoors, but two-dimensional coordinate values (x, y) are also acceptable. Note that the origin of the coordinate values for identifying the relative position indoors or outdoors is not restricted. The specific outdoor location is preferably a location where GPS signals are difficult to reach, such as among high-rise building groups or in the middle of a forest, but it is not restricted. The location information can also be information (e.g., a character string) or an ID that a person can recognize a place by. Such location information can be, for example, labels such as "living room", "workroom", "meeting room", "east side of the library", "toy section of the department store", etc.

[0043] The position reception unit 121 may generate a unique ID. Such a unique ID is a label and can also be considered as location information.

[0044] The position reception unit 121 does not have to receive location information. In such a case, the position reception unit 121 is unnecessary.

[0045] The receiving unit 13 receives radio waves including device identifiers from three or more communication devices B at a specific location. The receiving unit 13 usually continuously receives radio waves including device identifiers from three or more communication devices B.

[0046] The device identifier is information that identifies the communication device B. The device identifier is, for example, the ID of the communication device B or the name of the communication device B. It is acceptable to consider the reception of radio waves as the reception of information.

[0047] The processing unit 14 performs various processes. The various processes are, for example, the processes performed by the intensity acquisition unit 141, the type determination unit 142, and the storage unit 143.

[0048] The intensity acquisition unit 141 acquires the intensity of the radio wave received from each communication device B with three or more communication devices B. The intensity acquisition unit 141 acquires the radio wave intensity in pairs with the device identifier of the communication device B. The intensity acquisition unit 141 acquires the time-series radio wave intensity. The time-series radio wave intensity is two or more radio wave intensities that are continuous in time. It goes without saying that being continuous in time may have a time interval.

[0049] The type determination unit 142 uses the time-series radio wave intensity acquired by the intensity acquisition unit 141 to determine whether each of the three or more communication devices B is a fixed terminal or a mobile terminal. A fixed terminal is a communication device whose installation position is fixed. A mobile terminal is a communication device whose installation position is not fixed and is moving.

[0050] For example, the type determination unit 142 acquires the degree of variation of two or three or more radio wave intensities that are continuous in time in a time series corresponding to one device identifier, and when the degree of variation is equal to or greater than a threshold value, it determines that the communication device B identified by the one device identifier is a mobile terminal. Further, for example, the type determination unit 142 acquires the degree of variation of two or three or more radio wave intensities that are continuous in time in a time series corresponding to one device identifier, and when the degree of variation is equal to or less than a threshold value, it determines that the communication device B identified by the one device identifier is a fixed terminal.

[0051] Note that the degree of variation is information indicating the degree of variation or change of the time-series radio wave intensity. The degree of variation is, for example, variance, standard deviation, or a number based on a difference (for example, a difference, a value obtained by adding the differences between two consecutive radio wave intensities among three or more radio wave intensities that are continuous in time).

[0052] For example, the type determination unit 142 is the number of radio wave intensities acquired in a predetermined time, and when the number of radio wave intensities that are continuous in time in a time series corresponding to one device identifier is equal to or less than a threshold value, it determines that the communication device B identified by the one device identifier is a mobile terminal.

[0053] The storage unit 143 forms and stores location information having the device identifier and the radio wave intensity of the communication device B determined by the type determination unit 142 to be a fixed terminal. It is preferable that the storage unit 143 forms and stores location information having the device identifier and the radio wave intensity for each of three or more communication devices B.

[0054] The storage unit 143 forms and stores location information having the device identifier, the radio wave intensity, and the location information of a specific location of the communication device B determined by the type determination unit 142 to be a fixed terminal. It is preferable that the storage unit 143 forms and stores location information having the device identifier, the radio wave intensity, and the location information of a specific location for each of three or more communication devices B. The storage unit 143 stores the location information in the storage unit 11, for example, but it may be stored in other devices. It is preferable that the location information has location information, but it may not have location information. The location information may be composed of only the device identifier and the radio wave intensity.

[0055] The radio wave intensity stored by the storage unit 143 is usually a representative value of the time-series radio wave intensity of the radio wave from the communication device B. The representative value is, for example, the median value, the average value, the maximum value, or the minimum value.

[0056] The storage unit 11 is preferably a non-volatile recording medium, but can also be realized with a volatile recording medium.

[0057] The process of storing information in the storage unit 11 is not limited. For example, information may be stored in the storage unit 11 via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11, or information input via an input device may be stored in the storage unit 11.

[0058] The reception unit 12 and the position reception unit 121 can be realized by a device driver of an input means such as a touch panel or a keyboard, control software of a menu screen, or the like.

[0059] The reception unit 13 is usually realized by wireless or wired communication means.

[0060] The processing unit 14, the intensity acquisition unit 141, the type determination unit 142, and the storage unit 143 can usually be realized from a processor, a memory, or the like. The processing procedures of the processing unit 14 and the like are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (a dedicated circuit). The processor may be a CPU, an MPU, a GPU, or the like, and its type is not limited.

[0061] Next, an operation example of the location information production device 1 will be described with reference to the flowchart of FIG. 3.

[0062] (Step S301) The position reception unit 121 determines whether or not it has received the position information of a specific location. If it has received the position information, it proceeds to step S302; if not, it returns to step S301.

[0063] (Step S302) The storage unit 143 acquires the position information received in step S301.

[0064] (Step S303) The processing unit 14 and the like perform a time-series intensity acquisition process. The time-series intensity acquisition process is a process of acquiring the time-series radio wave intensities of radio waves from three or more each communication device B. An example of the time-series intensity acquisition process will be described with reference to the flowchart of FIG. 4.

[0065] (Step S304) The processing unit 14 and the like perform a fixed information acquisition process. Then, it returns to step S301. The fixed information acquisition process is a process of acquiring the intensity of radio waves from a fixed terminal at the location specified by the position information received in step S301. An example of the fixed information acquisition process will be described with reference to the flowchart of FIG. 5.

[0066] In the flowchart of FIG. 3, it is preferable that the user holding the location information production device 1 moves to three or more each specific location, and the location information production device 1 receives the position information for each of the three or more specific locations and repeatedly performs the processes from S301 to S304.

[0067] Also, in the flowchart of FIG. 3, the process ends due to a power-off or an interrupt at the end of processing.

[0068] Next, an example of the time-series intensity acquisition process in step S303 will be described using the flowchart of FIG. 4.

[0069] (Step S401) The receiving unit 13 determines whether it has received radio waves from any communication device B. If radio waves are received, the process proceeds to step S402; if not, the process returns to step S401.

[0070] (Step S402) The storage unit 143 acquires the device identifier corresponding to the radio waves received in step S401.

[0071] (Step S403) The intensity acquisition unit 141 acquires the intensity of the radio waves received in step S401.

[0072] (Step S404) The storage unit 143 appends the radio wave intensity acquired in step S403 to a buffer (not shown) in association with the device identifier acquired in step S402.

[0073] (Step S405) The storage unit 143 determines whether it meets the storage conditions for location information. If it meets the storage conditions, it returns to the upper-level process; if not, it returns to step S401. The storage conditions are, for example, that a threshold time or more has elapsed since receiving the position information of a specific location, or that the radio wave intensities corresponding to three or more pairs of device identifiers have been stored.

[0074] Next, an example of the fixed information acquisition process in step S304 will be described using the flowchart of FIG. 5.

[0075] (Step S501) The type determination unit 142 assigns 1 to the counter i.

[0076] (Step S502) The type determination unit 142 determines whether the i-th device identifier exists in a buffer (not shown). If the i-th device identifier exists, the process proceeds to step S503. If the i-th device identifier does not exist, the process returns to the upper-level process.

[0077] (Step S503) The type determination unit 142 determines the type of the communication device B identified by the i-th device identifier. An example of such type determination processing will be described using the flowchart of FIG. 6.

[0078] (Step S504) If the determination result in step S503 is "fixed terminal", the process proceeds to step S505. If it is "mobile terminal", the process proceeds to step S508.

[0079] (Step S505) The storage unit 143 acquires two or more radio wave intensities corresponding to the i-th device identifier from a buffer (not shown).

[0080] (Step S506) The storage unit 143 acquires a representative value of the two or more radio wave intensities.

[0081] (Step S507) The storage unit 143 stores, in the storage unit 11 in association with the position information received in step S301, a pair of the i-th device identifier and the representative value of the radio wave intensity acquired in step S506.

[0082] (Step S508) The type determination unit 142 increments the counter i by 1. The process returns to step S502.

[0083] In the flowchart of FIG. 5, in step S506, the storage unit 143 may acquire the most recent radio wave intensity instead of the representative value of the two or more radio wave intensities.

[0084] Next, an example of the type determination processing in step S503 will be described using the flowchart of FIG. 6.

[0085] (Step S601) The type determination unit 142 acquires two or more radio wave intensities that correspond to the i-th device identifier in step S502 from a buffer (not shown).

[0086] (Step S602) The type determination unit 142 acquires the degree of variation of the two or more radio wave intensities acquired in step S601.

[0087] (Step S603) The type determination unit 142 determines whether the degree of variation acquired in step S602 is equal to or less than a threshold value or less than the threshold value. If the degree of variation is equal to or less than the threshold value or less than the threshold value, it proceeds to step S604. If it is equal to or greater than the threshold value or greater than the threshold value, it proceeds to step S605.

[0088] (Step S604) The type determination unit 142 sets the type of the communication device B as a "fixed terminal". Return to the upper-level process.

[0089] (Step S605) The type determination unit 142 sets the type of the communication device B as a "mobile terminal". Return to the upper-level process.

[0090] Hereinafter, a specific operation example of the location information production device 1 in the present embodiment will be described. Here, it is assumed that the accumulation condition is that a predetermined time (for example, 3 minutes) has elapsed since the position information of a specific location was received.

[0091] It is assumed that user A is, for example, in a certain indoor area (for example, user A's home or a department store that user A often visits). And it is assumed that user A has input the position information (x1, y1) to the location information production device 1.

[0092] Next, the position reception unit 121 of the location information production device 1 receives the position information (x1, y1) of the specific location. Next, the accumulation unit 143 acquires the received position information (x1, y1) in a buffer (not shown).

[0093] Then, the receiving unit 13 receives radio waves including device identifiers from each communication device B with 3 or more for a predetermined time (for example, 3 minutes). Then, the storage unit 143 acquires the device identifiers included in the received radio waves. Also, the intensity acquisition unit 141 acquires the intensity of the received radio waves. Next, the storage unit 143 appends the acquired radio wave intensity to a buffer (not shown) in association with the acquired device identifier. As a result, a time-series radio wave intensity management table shown in FIG. 7 is configured in the buffer (not shown). The time-series radio wave intensity management table shown in FIG. 7 is a table at a specific location indicated by the position information (x1, y1).

[0094] The time-series radio wave intensity management table is a table that manages the time-series radio wave intensities for each communication device B. The time-series radio wave intensity management table is a table that manages two or more records having "ID", "device identifier", and "time-series radio wave intensity". "ID" is information for identifying a record. "Time-series radio wave intensity" is radio wave intensity that is continuous in time. "R 11 」「R 12 」「R 21 」 etc. are radio wave intensities.

[0095] After the management table in FIG. 7 is configured, since a predetermined time (for example, 3 minutes) has elapsed after the position information (x1, y1) of the specific location is received by the position reception unit 121, the storage unit 143 determines that it meets the storage condition of the location information.

[0096] Next, the type determination unit 142 acquires the degree of variation in the time-series radio wave intensity of each record in FIG. 7 according to the operation of the flowchart in FIG. 6, and determines whether each communication device B is a fixed terminal or a mobile terminal. And let's assume that the type determination unit 142 determines that the communication device B identified by the device identifiers "device 1, device 3, device 4, device 6,..." is a fixed terminal, and the communication device B identified by the device identifiers "device 2, device 5,..." is a mobile terminal.

[0097] Next, the storage unit 143 forms radio field intensity information by pairing the device identifier of the communication device B, which is a fixed terminal, with the representative value of the radio field intensity. Then, the storage unit 143 stores a plurality of pieces of each radio field intensity information in association with the position information (x1, y1). By such processing, the record of "ID = 1" in the location information management table of FIG. 8 is configured. Note that the storage unit 143 may store only the device identifier and the radio field intensity. In such a case, each record in FIG. 8 does not have position information. Also, in such a case, the position reception unit 121 does not need to receive the position information of a specific location.

[0098] The location information management table is a table for managing location information. In the location information management table, there are records corresponding to the position information, and a plurality of records having "ID", "device identifier", and "radio field intensity information" are stored. The "radio field intensity information" has a "device identifier" and a "radio field intensity".

[0099] Through the above processing, the location information at the specific location 1 at the position indicated by the position information (x1, y1) is stored.

[0100] User A moves to the specific location 2 at the position indicated by the position information (x2, y2) with the location information production device 1 and performs the same operations as above. As a result, the location information production device 1 configures and stores the record of "ID = 2" in the location information management table of FIG. 8. Further, User A moves to each of one or more specific locations including the specific location 3 with the location information production device 1 and performs the same operations as above. As a result, the location information production device 1 configures and stores the record (not shown) of the transition of "ID = 3" in the location information management table of FIG. 8.

[0101] As described above, according to the present embodiment, location information for acquiring the position of the terminal device indoors can be obtained. That is, according to the present embodiment, three or more pieces of location information for acquiring the position of the terminal device indoors can be produced.

[0102] Note that the processing in this embodiment may be implemented by software. And this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software that realizes the location information production device 1 in this embodiment is a program as follows. That is, this program causes a computer to function as a position reception unit that receives the position information of a specific location, a reception unit that receives radio waves including a device identifier for identifying the communication device from each of three or more communication devices at the specific location, an intensity acquisition unit that acquires the time-series radio wave intensity for each of three or more communication devices, a type determination unit that determines whether each of the three or more communication devices is a fixed terminal that is a communication device fixed in place or a mobile terminal that is a moving communication device using the time-series radio wave intensity acquired by the intensity acquisition unit, and a storage unit that functions to configure and store location information having the device identifier, radio wave intensity, and the position information of the specific location of each of the three or more communication devices determined by the type determination unit to be a fixed terminal.

[0103] (Embodiment 2) In this embodiment, a terminal device that receives radio waves from each of three or more communication devices B, determines the type of each communication device B using the time-series radio wave intensity, and acquires and outputs a terminal position indicating the position of the terminal device indoors using only the radio wave intensity from the communication devices B of which the type is "fixed terminal" will be described.

[0104] Also, in this embodiment, a terminal device that determines whether the terminal device is moving or stopped, and uses the determination result to acquire and output the terminal position will be described.

[0105] FIG. 9 is a conceptual diagram of the information system C in this embodiment. The information system C includes one or two or more terminal devices 2 and three or more communication devices B.

[0106] The terminal device 2 is a terminal capable of acquiring position information indoors. The terminal device 2 is, for example, a smartphone, a tablet terminal, a smartwatch, a so-called personal computer, etc., and its type is not limited.

[0107] FIG. 10 is a block diagram of the terminal device 2 in the present embodiment. The terminal device 2 includes a storage unit 21, a reception unit 22, a processing unit 23, and an output unit 24. The storage unit 21 includes a location information storage unit 211. The processing unit 23 includes a strength acquisition unit 231, a type determination unit 232, a movement determination unit 233, and a position acquisition unit 234. The position acquisition unit 234 includes a strength acquisition means 2341, a location determination means 2342, and a position acquisition means 2343. The output unit 24 includes a position output unit 241.

[0108] Various types of information are stored in the storage unit 21. The various types of information are, for example, the location information described later.

[0109] Three or more pieces of location information are stored in the location information storage unit 211. It is preferable that the three or more pieces of location information in the location information storage unit 211 are the information accumulated by the location information production device 1.

[0110] Each of the three or more pieces of location information in the location information storage unit 211 has, for example, the location information of a specific location, a device identifier, and a radio wave intensity. It is preferable that three or more pieces of radio wave intensity information are associated with the three or more pieces of location information. The radio wave intensity information has a device identifier and a radio wave intensity. The three or more pieces of radio wave intensity information may form a radio wave intensity vector. The radio wave intensity vector is a vector using the radio wave intensity information, and has, for example, a structure of (radio wave intensity of device identifier 1, radio wave intensity of device identifier 2, radio wave intensity of device identifier 3, ··· radio wave intensity of device identifier n). A location information management table having the structure of FIG. 8, for example, is stored in the location information storage unit 211.

[0111] Note that the terminal device 2 does not necessarily have the location information storage unit 211. In such a case, the terminal device 2 refers to the location information storage unit 211 of an external device (not shown) and acquires the terminal position described later.

[0112] The receiving unit 22 receives radio waves including device identifiers for identifying the communication device B from each of three or more communication devices B. The receiving unit 22 usually performs the same functions as the above-described receiving unit 13. The receiving unit 22 usually receives radio waves including device identifiers from each of three or more communication devices B. The receiving unit 22 usually receives radio waves continuously.

[0113] The processing unit 23 performs various processes. The various processes are, for example, processes performed by the intensity acquisition unit 231, the type determination unit 232, the movement determination unit 233, and the position acquisition unit 234.

[0114] The intensity acquisition unit 231 acquires time-series radio wave intensities for each of three or more communication devices B. The intensity acquisition unit 231 acquires the radio wave intensity based on the radio waves received by the receiving unit 22. Note that the intensity acquisition unit 231 performs the same functions as the above-described intensity acquisition unit 141.

[0115] The type determination unit 232 uses the time-series radio wave intensities acquired by the intensity acquisition unit 231 to determine whether each of three or more communication devices B is a fixed terminal or a mobile terminal. The type determination unit 232 performs the same functions as the above-described type determination unit 142.

[0116] The movement determination unit 233 determines whether the terminal device 2 is in motion or stopped, and obtains a movement determination result that is the result of the determination. The movement determination result is, for example, "in motion" or "stopped".

[0117] The movement determination unit 233, for example, acquires sensor information of the terminal device 2 and uses the sensor information to obtain a movement determination result. The sensor information is, for example, acceleration by a gyroscope and time-series position information.

[0118] The movement determination unit 233, for example, obtains a movement determination result of "stopped" if the acceleration by the gyroscope is "0" or less than or equal to a threshold value. The movement determination unit 233, for example, obtains a movement determination result of "in motion" if the acceleration by the gyroscope is greater than or equal to the threshold value or greater than the threshold value.

[0119] The movement determination unit 233 determines, for example, using the time-series radio wave intensities of each communication device B of 3 or more obtained by the intensity acquisition unit 231, that it is in a stopped state when there is no change in the time-series radio wave intensities of 1 or 2 or more communication devices B, and acquires a movement determination result "stopped".

[0120] The position acquisition unit 234 acquires the radio wave intensities of 3 or more communication devices B determined by the type determination unit 232 to be fixed terminals, and uses the radio wave intensities of the fixed terminals to acquire the terminal position, which is the position information of the terminal device 2 indoors.

[0121] The position acquisition unit 234 acquires, for example, the radio wave intensities of 3 or more communication devices B determined by the type determination unit 232 to be fixed terminals, refers to 3 or more location information in the location information storage unit 211 using the 3 or more radio wave intensities, and acquires the terminal position by the fingerprint method.

[0122] It is preferable for the position acquisition unit 234 to acquire position information using the movement determination result. For example, it is preferable for the position acquisition unit 234 to acquire the terminal position only when the movement determination result is "stopped".

[0123] The position acquisition unit 234 may acquire the terminal position using the location information including the device identifier only when the device identifier corresponding to the radio wave received by the reception unit 22 is included in the location information of the location information storage unit 211. This is because the location information of the location information storage unit 211 is the location information of the fixed terminal.

[0124] The intensity acquisition means 2341 acquires the radio wave intensities of 3 or more communication devices determined by the type determination unit 232 to be fixed terminals in association with the device identifiers.

[0125] The location determination means 2342 determines 1 or 2 or more location information satisfying similar conditions with the radio wave intensities corresponding to each of the 3 or more device identifiers acquired by the intensity acquisition means 2341 from the location information of the location information storage unit 211.

[0126] The location determination means 2342 acquires, for example, a first radio wave intensity vector which is a vector having, as elements, the radio wave intensities paired with each of three or more device identifiers included in the location information. Further, the location determination means 2342 acquires, for example, a second radio wave intensity vector which is a vector having, as elements, the radio wave intensities corresponding to each of three or more device identifiers acquired by the intensity acquisition means 2341. The location determination means 2342 acquires, for example, the similarity between the two radio wave intensity vectors, and when the similarity is equal to or greater than a threshold value, acquires the location information corresponding to the first radio wave intensity vector.

[0127] The position acquisition means 2343 acquires the position information included in each of one or more location information determined by the location determination means 2342, and acquires the terminal position using the one or more pieces of position information.

[0128] The output unit 24 outputs various types of information. The various types of information are, for example, here, the terminal position and an indoor map.

[0129] Here, output includes concepts such as display on a display, projection using a projector, printing by a printer, sound output, transmission to an external device, storage in a recording medium, and delivery of a processing result to another processing device or another program.

[0130] The position output unit 241 outputs the terminal position acquired by the position acquisition unit 234. The position output unit 241, for example, displays a symbol indicating the position specified by the terminal position on an indoor map.

[0131] The storage unit 21 and the location information storage unit 211 are preferably non-volatile recording media, but can also be realized using volatile recording media.

[0132] The process of storing information in the storage unit 21 or the like is not limited. For example, information may be stored in the storage unit 21 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 21 or the like, or information input via an input device may be stored in the storage unit 21 or the like.

[0133] The receiving unit 22 is usually realized by wireless or wired communication means.

[0134] The processing unit 23, the intensity acquisition unit 231, the type determination unit 232, the movement determination unit 233, the position acquisition unit 234, the intensity acquisition means 2341, the location determination means 2342, and the position acquisition means 2343 can usually be realized from a processor, a memory, etc. The processing procedures of the processing unit 23, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (a dedicated circuit). Note that the processor is a CPU, an MPU, a GPU, etc., and its type does not matter.

[0135] The output unit 24 and the position output unit 241 can be realized, for example, by driver software for an output device such as a display or a speaker, or by the driver software for the output device and the output device, etc.

[0136] Next, a first operation example of the terminal device 2 will be described using the flowchart of FIG. 11.

[0137] (Step S1101) The movement determination unit 233 determines whether the terminal device 2 is moving or stopped. An example of such movement determination processing will be described using the flowchart of FIG. 12.

[0138] (Step S1102) If the determination result in Step S1101 is "stopped", the process proceeds to Step S1103; if it is "moving", the process returns to Step S1101.

[0139] (Step S1103) The intensity acquisition means 2341 performs time-series intensity acquisition processing. An example of the time-series intensity acquisition processing was described using the flowchart of FIG. 4.

[0140] (Step S1104) The type determination unit 232, the location determination means 2342, etc. perform fixed information acquisition processing. An example of the fixed information acquisition processing was described using the flowchart of FIG. 5.

[0141] (Step S1105) The position acquisition means 2343 performs a position estimation process for acquiring the terminal position. An example of the position estimation process will be described using the flowchart of FIG. 13.

[0142] (Step S1106) The position output unit 241 outputs the terminal position acquired in Step S1105. Return to Step S1101.

[0143] Note that in the flowchart of FIG. 11, the process ends due to a power-off or a process end interrupt.

[0144] Next, an example of the movement determination process in Step S1101 will be described using the flowchart of FIG. 12.

[0145] (Step S1201) The movement determination unit 233 acquires the sensor value (for example, acceleration) of the terminal device 2 and temporarily stores it in a buffer (not shown).

[0146] (Step S1202) The movement determination unit 233 determines whether to perform a movement determination using the sensor values in the buffer (not shown). If performing a movement determination, go to Step S1203; if not performing a movement determination, return to Step S1201. Note that the movement determination unit 233 may always perform a movement determination, or may perform a movement determination after accumulating a predetermined number or more of sensor values in the buffer, or after a predetermined time has elapsed since the acquisition of the sensor values.

[0147] (Step S1203) The movement determination unit 233 determines whether the terminal device 2 is in motion or stopped using one or more sensor values in the buffer (not shown). If it is stopped, go to Step S1204; if it is in motion, go to Step S1205.

[0148] (Step S1204) The movement determination unit 233 sets the movement determination result to "stopped". Go to Step S1206.

[0149] (Step S1205) The movement determination unit 233 sets the movement determination result to "moving".

[0150] (Step S1206) The movement determination unit 233 clears a buffer (not shown) and returns to the upper-level process.

[0151] Next, an example of the position estimation process in Step S1105 will be described using the flowchart of FIG. 13.

[0152] (Step S1301) The position acquisition means 2343 acquires radio field intensity information of three or more (pairs of device identifiers and radio field intensities) of the terminal device 2.

[0153] (Step S1302) The position acquisition means 2343 vectorizes the radio field intensity information of three or more to obtain a radio field intensity vector. The radio field intensity vector is, for example, (radio field intensity of device identifier 1, radio field intensity of device identifier 2, radio field intensity of device identifier 3,... radio field intensity of device identifier n).

[0154] (Step S1303) The position acquisition means 2343 substitutes 1 into the counter i.

[0155] (Step S1304) The position acquisition means 2343 determines whether the i-th location information exists in the location information storage unit 211. If the i-th location information exists, it proceeds to Step S1305; if not, it proceeds to Step S1309.

[0156] (Step S1305) The position acquisition means 2343 acquires the i-th radio field intensity vector included in the i-th location information from the location information storage unit 211.

[0157] (Step S1306) The position acquisition means 2343 acquires the similarity between the radio wave intensity vector acquired in step S1302 and the i-th radio wave intensity vector acquired in step S1305. Next, the position acquisition means 2343 determines whether the similarity satisfies the similarity condition (for example, the similarity is equal to or greater than a threshold value). If the similarity satisfies the similarity condition, it proceeds to step S1307; if not, it proceeds to step S1308.

[0158] (Step S1307) The position acquisition means 2343 acquires the position information and the similarity of the i-th point information and stores them in a buffer (not shown).

[0159] (Step S1308) The position acquisition means 2343 increments the counter i by 1. It returns to step S1304.

[0160] (Step S1309) The position acquisition means 2343 acquires the terminal position that specifies the position of the terminal device 2 using the set of position information and similarity of 3 or more stored in a buffer (not shown). It returns to the upper-level process.

[0161] Next, a second operation example of the terminal device 2 will be described using the flowchart of FIG. 14.

[0162] (Step S1401) The intensity acquisition means 2341 performs a time-series intensity acquisition process. An example of the time-series intensity acquisition process was described using the flowchart of FIG. 4.

[0163] (Step S1402) The point determination means 2342 performs a fixed information acquisition process. An example of the fixed information acquisition process was described using the flowchart of FIG. 5.

[0164] (Step S1403) The position acquisition means 2343 performs a position estimation process for acquiring the terminal position. An example of the position estimation process was described using the flowchart of FIG. 13.

[0165] (Step S1404) The position output unit 241 outputs the terminal position acquired in Step S1403. Return to Step S1401.

[0166] In addition, in the flowchart of FIG. 14, the process ends due to a power-off or a processing end interrupt.

[0167] Next, a second example of the type determination process in the fixed information acquisition process of Step S1402 in the flowchart of FIG. 14 will be described with reference to the flowchart of FIG. 15. Note that the first example of the type determination process was described with reference to the flowchart of FIG. 6.

[0168] (Step S1501) The type determination unit 232 acquires the device identifier of the communication device B for which the type is to be determined.

[0169] (Step S1502) The type determination unit 232 determines whether the device identifier acquired in Step S1501 exists in any of the location information in the location information storage unit 211. If it exists in any of the location information, go to Step S1503; if it does not exist, go to Step S1504.

[0170] (Step S1503) The type determination unit 232 sets the type as "fixed terminal". Return to the upper-level process.

[0171] (Step S1504) The type determination unit 232 sets the type as "mobile terminal". Return to the upper-level process.

[0172] Hereinafter, a specific operation example of the terminal device 2 in the present embodiment will be described.

[0173] Assume that User B holds his / her terminal device 2 and enters an indoor location identified by a location identifier (P). Then, the receiving unit 22 of the terminal device 2 transmits a location information request having the location identifier (P) to an external device (not shown) and receives a location information management table shown in FIG. 8 from the device. Then, the processing unit 23 temporarily stores the location information management table in the location information storage unit 211.

[0174] Then, the terminal device 2 operates as follows according to the processing from step S1103 to S1106 in FIG. 11 or the processing of the flowchart in FIG. 14.

[0175] That is, the intensity acquisition means 2341 performs the time-series intensity acquisition process described with reference to the flowchart in FIG. 4, acquires the radio wave intensities of three or more communication devices B at the location X where the terminal device 2 is present, and constructs a time-series radio wave intensity management table having the structure shown in FIG. 7.

[0176] Next, the intensity acquisition means 2341 performs the time-series intensity acquisition process, acquires the time-series radio wave intensities of each communication device B that can receive radio waves at the location X, and constructs a time-series radio wave intensity management table having the structure shown in FIG. 7.

[0177] Next, the type determination unit 142 refers to the time-series radio wave intensity management table and determines whether each communication device B is a "fixed terminal" or a "mobile terminal" by the type determination process described with reference to the flowchart in FIG. 4.

[0178] Next, the location determination means 2342 acquires the radio wave intensities of "Device 1", "Device 3", "Device 4", "Device 6",... determined to be fixed terminals. Then, the location determination means 2342 acquires a radio wave intensity vector "(radio wave intensity of Device 1, radio wave intensity of Device 3, radio wave intensity of Device 4, radio wave intensity of Device 6,...) = (P1, P3, P4, P6,...)". Here, the radio wave intensity of the communication device B acquired by the location determination means 2342 may be a representative value of two or more radio wave intensities or may be a single radio wave intensity such as the latest radio wave intensity of the communication device B.

[0179] Next, the position acquisition means 2343 performs the position estimation process described using the flowchart of FIG. 13, and calculates the similarity between the radio wave intensity vector at point X and the radio wave intensity vectors of each record in FIG. 8 (vectors composed of radio wave intensity information). Next, the position acquisition means 2343 determines radio wave intensity vectors that satisfy the similarity condition "similarity >= threshold value" (for example, the radio wave intensity vector of "ID=1" in FIG. 8 (S 11 , S 12 , S 13 , ···), the radio wave intensity vector of "ID=2" (S 21 , S 22 , S 23 , ···), ···). Next, the position acquisition means 2343 acquires a set of position information and similarity that forms a pair with the radio wave intensity vector that satisfies the similarity condition (for example, "(x1, y1), DS1", "(x2, y2), DS2", ···). Next, the position acquisition means 2343 acquires the position (x1 × DS1 / sum of similarities + x2 × DS2 / sum of similarities + ···, y1 × DS1 / sum of similarities + y2 × DS2 / sum of similarities + ···) of point X indoors. Note that the sum of similarities is "DS1 + DS2 + ···".

[0180] Next, the position output unit 241 outputs the map of the indoor or outdoor area stored in the storage unit 21, and arranges a symbol at the position indicated by the position information (terminal position) acquired by the position acquisition means 2343 on the map.

[0181] As described above, according to the present embodiment, the position of the terminal device 2 indoors or outdoors can be easily acquired.

[0182] Note that the processing in this embodiment may be implemented by software. And this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software that realizes the terminal device 2 in this embodiment is the following program. That is, this program causes a computer to function as a receiving unit that receives radio waves including device identifiers for identifying the communication devices from three or more communication devices, an intensity acquisition unit that acquires the radio wave intensity in time series for each of the three or more communication devices, a type determination unit that determines whether each of the three or more communication devices is a fixed terminal where the device is fixed or a mobile terminal that is moving, using the radio wave intensity in time series acquired by the intensity acquisition unit, a position acquisition unit that acquires the terminal position, which is the position information of the terminal device, using the radio wave intensity of the three or more communication devices determined by the type determination unit to be fixed terminals, and a position output unit that outputs the terminal position acquired by the position acquisition unit.

[0183] (Embodiment 3) In this embodiment, an action acquisition device that acquires the user's action information and time zone using the user's position information corresponding to the time and outputs the action information associated with the time zone will be described. Note that it is preferable that the action acquisition device further uses the user's activity data and vital data to acquire the user's action information and time zone. Also, the position information used when acquiring the action information may be the indoor position information or the like acquired by the terminal device 2 described in Embodiment 2.

[0184] Also, in this embodiment, an action acquisition device that acquires action information using past record information will be described. Note that the past record information may be information based on the input of one or two or more users.

[0185] Also, in this embodiment, a behavior acquisition device that acquires and outputs a user's emotional information will be described. Also, in this embodiment, it is preferable to acquire emotional information using past record information.

[0186] In this embodiment, the behavior acquisition device is a terminal. However, as described in Embodiment 4, the behavior acquisition device may also be a server. That is, the process of acquiring the behavior information or the emotional information described later may be performed by the user's terminal or the server.

[0187] Also, in this embodiment, when behavior information cannot be acquired, a behavior acquisition device that acquires and outputs location information using a map will be described.

[0188] Furthermore, in this embodiment, a behavior acquisition device that displays the estimated behavior information and the confirmed behavior information in a visually distinguishable manner will be described.

[0189] FIG. 16 is a conceptual diagram of the information system D in this embodiment. The information system D includes one or two or more behavior acquisition devices 3, a server device 4, and three or more communication devices B.

[0190] The behavior acquisition device 3 is a terminal. The behavior acquisition device 3 is a device that acquires and outputs behavior information. The behavior acquisition device 3 is, for example, a smartphone, a tablet terminal, a smartwatch, a so-called personal computer, etc., and its type is not limited.

[0191] The server device 4 is, for example, a set of behavior source information and behavior information of two or more users, and is a device that stores a set for each time zone. The server device 4 stores, for example, learning information described later, and is a device that provides the learning information to the behavior acquisition device 3. The server device 4 is, for example, a cloud server, an ASP server, but its type is not limited.

[0192] FIG. 17 is a block diagram of the information system D in this embodiment. FIG. 18 is a block diagram of the behavior acquisition device 3.

[0193] The behavior acquisition device 3 includes a storage unit 31, a reception unit 32, a processing unit 33, and an output unit 34. The storage unit 31 includes a learning management unit 311, a map management unit 312, and a behavior management unit 313. Note that the behavior management unit 313 may exist in an external device (not shown). The processing unit 33 includes an intensity acquisition unit 231, a type determination unit 232, a time acquisition unit 331, a position acquisition unit 332, an activity acquisition unit 333, a vital sign acquisition unit 334, a behavior estimation unit 335, an emotion estimation unit 336, a location acquisition unit 337, a storage unit 338, and a configuration unit 339. The output unit 34 includes a behavior output unit 341 and an emotion output unit 342.

[0194] The server device 4 includes a server storage unit 41, a server reception unit 42, a server processing unit 43, and a server transmission unit 44.

[0195] The behavior acquisition device 3 receives, for example, an output instruction, a confirmation instruction, and information input. The output instruction is an instruction to output output information described later. The output instruction usually has period information. The period information is information that specifies the period for outputting behavior information and the like. The confirmation instruction is an instruction to confirm the estimated behavior information or emotion information. The information input is an input of information for changing the behavior information or the emotion information when the estimated behavior information or emotion information is different. The information input is the updated behavior information or the updated emotion information.

[0196] Various types of information are stored in the storage unit 31 that constitutes the behavior acquisition device 3. The various types of information are, for example, learning information described later, a map described later, behavior information described later, location information, a calendar template, behavior information associated with two or more behavior conditions, and emotion information associated with two or more emotion conditions.

[0197] The calendar template is information indicating the calendar template to be output. The calendar template is, for example, an ICS file, a file described in HTML, or a file described in XML, and its data structure is not limited.

[0198] An action condition is a condition for acquiring action information. The action condition is a condition using two or more pieces of action source information. The action condition is associated with the action information. The action condition is, for example, "location information = office AND 8:00 <= time <= 19:00", and the action information associated with such an action condition is "work". The action condition is, for example, "location information = kitchen AND activity data = standing AND 7:00 <= time <= 8:00", and the action information associated with such an action condition is "cooking". The action condition is, for example, "location information = park AND activity data = standing AND 120 <= heart rate", and the action information associated with such an action condition is "running".

[0199] An emotion condition is a condition for acquiring emotion information. The emotion condition is a condition using two or more pieces of emotion source information. The emotion condition is associated with the emotion information. The emotion condition is, for example, "location information = office AND 8:00 <= time <= 19:00", and the emotion information associated with such an action condition is "positive". The emotion condition is, for example, "location information = kitchen AND activity data = standing AND 7:00 <= time <= 8:00", and the emotion information associated with such an action condition is "positive". The action condition is, for example, "location information = park AND activity data = standing AND 120 <= heart rate", and the emotion information associated with such an action condition is "negative".

[0200] Learning information is stored in the learning management unit 311. The learning information is information based on two or more pieces of teacher data. The learning information of the learning management unit 311 is, for example, action learning information and emotion learning information. Two or more pieces of learning information of the learning management unit 311 may each be associated with different user attribute value conditions. A user attribute value condition is a condition regarding one or two or more user attribute values.

[0201] A user attribute value is an attribute value of a user. The user attribute value is, for example, occupation, family composition, unmarried or married, gender, age, age group, morning type or night type, place of residence, etc., without limitation.

[0202] The behavior learning information is information based on two or more behavior teacher data. The behavior learning information is, for example, a behavior learning model or a behavior response table. The emotion learning information is information based on two or more emotion teacher data. The emotion learning information is, for example, an emotion learning model or an emotion response table.

[0203] The behavior teacher data has, for example, one or two or more behavior source information and behavior information. The emotion teacher data has, for example, one or two or more behavior source information, behavior information, and emotion information.

[0204] The emotion teacher data has, for example, one or two or more behavior source information or behavior information and emotion information. The emotion teacher data has, for example, one or two or more behavior source information, behavior information, and emotion information. In the emotion teacher data, one or more behavior source information, behavior information, or one or more behavior source information and behavior information are explanatory variables, and the emotion information is the target variable.

[0205] The behavior information is information that identifies the user's behavior. The behavior information is, for example, "work", "watching TV", "walking", "running", "going to the gym", "taking a bath", "sleeping".

[0206] The emotion information is information regarding the user's emotion. The emotion information is, for example, positive (e.g., "1") or negative (e.g., "0"). The emotion information is, for example, the degree of positivity or the degree of negativity. The emotion information is, for example, happy (e.g., "1"), angry (e.g., "2"), sad (e.g., "3"), happy (e.g., "4").

[0207] Action source information is the information that serves as the basis for acquiring action information. Action source information includes location information. It is preferable that the action source information includes activity data or one or more types of vital data. The action source information may include emotion information. The action source information may include one or more pieces of past action information. Among the past action information, usually, the immediately preceding action information is included. The action source information may include one or more pieces of the user's future schedule information. The schedule information is, for example, information stored in a calendar server (e.g., the server of "Google Calendar (registered trademark)") not shown in the figure. The action source information may include the elapsed time since arriving at the same location information. The action source information may include one or more user attribute values.

[0208] Location information is the information that identifies the location of the action acquisition device 3. The location information is, for example, (latitude, longitude), (latitude, longitude, altitude), three-dimensional relative position (x, y, z) indoors, or two-dimensional relative position (x, y) indoors, or place information. Place information is the information that expresses the meaning of a place. The place information is, for example, indoor place information or outdoor place information. Indoor place information is the information that identifies an indoor place. The indoor place information is, for example, "living room", "kitchen", "study", "office". Outdoor place information is the information that identifies an outdoor place. The outdoor place information is, for example, "ABC Station", "library", "izakaya", "Location A".

[0209] Body data is the information related to the user's body. The body data is, for example, activity data or vital data.

[0210] Activity data is the information that identifies the user's activity. The activity data is, for example, "standing", "grounded (e.g., sitting)".

[0211] Vital data is information that can be obtained from a user's living body. Vital data may also be referred to as biometric information. Vital data includes, for example, the heart rate, heart rate variability, blood pressure (systolic and / or diastolic), respiratory rate per unit time (e.g., per minute or per 30 seconds), and body temperature. Learning information is, for example, a learning model or a correspondence table. A learning model is information constructed by a learning process of machine learning using two or more pieces of teacher data and is information used for the prediction process of machine learning. A learning model may also be referred to as a learner, a classifier, a classification model, etc. The algorithm of machine learning is not limited, such as deep learning, random forest, decision tree, SVM, etc. Also, for machine learning, various machine learning functions such as the TensorFlow (registered trademark) library, the random forest module of the R language, TinySVM, and various existing libraries can be used. Also, when the learning information is a learning model, one or more pieces of source information of the teacher data are explanatory variables, and the action information is the target variable.

[0212] The learning model here is, for example, a behavior learning model or an emotion learning model. A behavior learning model is a learning model for obtaining behavior information and is information obtained by a learning process of machine learning using behavior teacher data. Behavior teacher data has one or more pieces of source information and behavior information.

[0213] An emotion learning model is a learning model for obtaining emotion information and is information obtained by a learning process of machine learning using emotion teacher data. Emotion teacher data has one or more pieces of source information and emotion information.

[0214] The correspondence table is an action correspondence table or an emotion correspondence table. The action correspondence table is a table for acquiring action information. The action correspondence table has two or more pieces of action correspondence information. The action correspondence information is information indicating the correspondence between one or more pieces of action source information and action information. Note that one or more pieces of action source information have, for example, a vector structure. Such a vector is called an action source vector. The action source vector is a vector having one or more pieces of each action source information as elements. The emotion correspondence table has two or more pieces of emotion correspondence information. The emotion correspondence information is information indicating the correspondence between one or more pieces of emotion source information and emotion information. One or more pieces of emotion source information have, for example, a vector structure. Such a vector is called an emotion source vector. The emotion source vector is a vector having one or more pieces of each emotion source information as elements.

[0215] The map management unit 312 stores a map. The map has location information corresponding to one or more pieces of each location information. The map is, for example, in the KIWI format, but its structure does not matter.

[0216] The action management unit 313 stores action information associated with two or more time zones. The action information here is, for example, information acquired by the action estimation unit 335. The action information is information acquired by the action estimation unit 335 and changed by the user.

[0217] It is preferable that it is possible to determine whether the action information here is confirmed action information. The action information is associated with a confirmation flag, for example. The confirmation flag is a flag indicating that the action information is confirmed. The ability to determine whether the action information is confirmed is, for example, that the storage areas for confirmed action information and unconfirmed action information are different. In addition, any method for making it possible to determine whether the action information is confirmed may be used.

[0218] The receiving unit 32 receives radio waves including a device identifier for identifying the communication device B from each of one or two or more communication devices B. The receiving unit 32 usually receives radio waves from each of three or more communication devices B. The receiving unit 32 has the same function as the receiving unit 22.

[0219] The processing unit 33 performs various processes. The various processes are, for example, processes performed by the intensity acquisition unit 231, the type determination unit 232, the time acquisition unit 331, and the like.

[0220] The time acquisition unit 331 acquires time. The time acquisition unit 331 acquires time from, for example, a clock (not shown). The time acquisition unit 331 receives time from, for example, the server device 4 or a device (not shown). The time may be in hours, minutes, and seconds, or in hours and minutes. The time may include information of one or two or more of "year", "month", and "day". The time acquisition unit 331 may acquire the day of the week. The day of the week may be considered as information included in the acquired time.

[0221] The position acquisition unit 332 acquires position information. The position information is associated with time. The position acquisition unit 332 usually acquires position information in association with the time acquired by the time acquisition unit 331. The position acquisition unit 332 may perform the same process as the position acquisition unit 234. In particular, when the behavior acquisition device 3 is present indoors or the like and a GPS signal cannot be received, it is preferable that the position acquisition unit 332 perform the same process as the position acquisition unit 234. It is preferable that the position acquisition unit 332 has a GPS receiver. The position acquisition unit 332 acquires position information by, for example, a GPS receiver. Such position information is absolute position information. The position information acquired by the position acquisition unit 332 may be position information either outdoors or indoors.

[0222] It is preferable that the position acquisition unit 332 acquires indoor position information using the radio wave intensity of the communication device B determined by the type determination unit 232 to be a fixed terminal. The radio wave intensity used here is preferably the radio wave intensity of three or more communication devices B, but may also be the radio wave intensity of one or two or more communication devices B.

[0223] The activity acquisition unit 333 acquires the user's activity data associated with the time. The activity acquisition unit 333 usually acquires the activity data in association with the time acquired by the time acquisition unit 331. The process of acquiring the activity data is a known technique.

[0224] The vital acquisition unit 334 acquires one or more types of vital data of the user associated with the time. The vital acquisition unit 334 usually acquires one or more types of vital data associated with the time acquired by the time acquisition unit 331. The process of acquiring the vital data is a known technique.

[0225] The action estimation unit 335 acquires action information for specifying the user's action in the time zone specified by the time included in two or more pieces of action source information including the position information associated with the time, using the two or more pieces of action source information. It is preferable that the action source information also includes the activity data. Further, it is preferable that the action source information also includes one or more types of vital data.

[0226] The action estimation unit 335, for example, detects an action condition in which two or more pieces of action source information including the position information associated with the time match, and acquires the action information paired with the action condition from the storage unit 31.

[0227] The action estimation unit 335, for example, when sitting at the position of the desk at home which is indoors at 13:15, acquires the action information "working". The action estimation unit 335, for example, when sitting in the living room at home which is indoors at 20:17, acquires the action information "watching TV". The action estimation unit 335, for example, when acquiring the position information "izakaya" at 20:17, acquires the action information "drinking party".

[0228] The action estimation unit 335 may also acquire the action information in the time zone, using the action learning information of the learning management unit 311 and two or more pieces of action source information including the position information associated with the time. Hereinafter, the processing of the action estimation unit 335 when the learning information is a learning model and when it is a correspondence table will be described.

[0229] The action estimation unit 335 may acquire one or more user attribute values, acquire from the learning management unit 311 the behavior learning information that pairs with the user attribute value conditions that match the one or more user attribute values, and use the behavior learning information to acquire behavior information. (1) When the behavior learning information is a behavior learning model

[0230] The action estimation unit 335 acquires the learning model of the learning management unit 311. Also, the action estimation unit 335 acquires the time and one or two or more pieces of action source information associated with the time. Next, the action estimation unit 335 provides the time, the action source information, and the learning model to a module that performs prediction processing of machine learning, executes the module, and acquires action information.

[0231] Note that when the score output by the module is less than or equal to or less than the threshold value, the action estimation unit 335 may not acquire the action information. (2) When the behavior learning information is a behavior correspondence table

[0232] The action estimation unit 335 acquires the time and the action source information associated with the time. Next, the action estimation unit 335 acquires an action source vector having the time and the action source information as elements. Next, the action estimation unit 335 calculates the similarity between the action source vector and the action source vectors of each of the two or more pieces of action correspondence information included in the action correspondence table. Next, the action estimation unit 335 acquires from the action correspondence table the action information that pairs with the action source vector having the maximum similarity. Note that even if the similarity is the maximum, when the similarity is less than or equal to or less than the threshold value, the action estimation unit 335 may not acquire the action information.

[0233] The emotion estimation unit 336 acquires emotion information regarding the user's emotion in a time zone using the action information or the action source information.

[0234] The emotion estimation unit 336, for example, detects an emotion condition that matches two or more pieces of emotion source information including position information associated with the time, and acquires from the storage unit 31 the emotion information that pairs with the emotion condition.

[0235] For example, when sitting at the position of a desk at home indoors at 13:15, the emotion estimation unit 336 acquires the emotion information "positive". For example, when sitting in the living room of a home indoors at 20:17, the emotion estimation unit 336 acquires the emotion information "positive". For example, when the location information "izakaya" is acquired at 20:17, the emotion estimation unit 336 acquires the emotion information "positive".

[0236] The emotion estimation unit 336 acquires the emotion learning information of the learning management unit 311 and the action information acquired by the action estimation unit 335 or one or more action source information from which the action information is obtained, and uses them to acquire emotion information in a time zone. Here, since the action information or one or more pieces of action source information are used for acquiring emotion information, they are referred to as emotion source information.

[0237] The emotion estimation unit 336 may acquire one or more user attribute values, acquire from the learning management unit 311 the emotion learning information that pairs with the user attribute value conditions that the one or more user attribute values match, and use the emotion learning information to acquire emotion information.

[0238] Hereinafter, the emotion estimation unit 336 will explain the processing of the action estimation unit 335 when the emotion learning information is an emotion learning model and when it is an emotion correspondence table. (1) When the emotion learning information is an emotion learning model

[0239] The emotion estimation unit 336 acquires the emotion learning model of the learning management unit 311. Also, the emotion estimation unit 336 acquires one or more pieces of action source information or action information associated with the time. Next, the emotion estimation unit 336 provides the acquired one or more pieces of action source information or action information and the emotion learning model to a module that performs prediction processing of machine learning, executes the module, and acquires emotion information.

[0240] Note that when the score output by the module is less than or equal to the threshold value or less than the threshold value, the emotion estimation unit 336 does not have to acquire emotion information. (2) When the emotion learning information is an emotion correspondence table

[0241] The emotion estimation unit 336 acquires one or more pieces of actor information or action information associated with the time. Next, the emotion estimation unit 336 acquires an emotion source vector having the one or more pieces of actor information or action information as elements. Next, the emotion estimation unit 336 calculates the similarity between the emotion source vector and the emotion source vectors of each of the two or more pieces of corresponding information in the emotion correspondence table. Next, the emotion estimation unit 336 acquires, from the emotion correspondence table, the emotion information paired with the emotion source vector having the maximum similarity. Note that even if the similarity is the maximum, the emotion estimation unit 336 does not have to acquire the emotion information when the similarity is less than or equal to the threshold value or less than the threshold value.

[0242] The location acquisition unit 337 refers to the map of the map management unit 312 and acquires location information corresponding to the location information acquired by the location acquisition unit 332. The location information in such a case is usually absolute location information (for example, (latitude, longitude)).

[0243] It is preferable that the location acquisition unit 337 acquires location information only for a time period when the action estimation unit 335 does not acquire action information.

[0244] The storage unit 338 associates the action information acquired by the action estimation unit 335 with the time period and stores it in the action management unit 313.

[0245] The storage unit 338 may associate the emotion information acquired by the emotion estimation unit 336 with the time period and store it in the action management unit 313.

[0246] The configuration unit 339 configures the information to be output using the action information of the action management unit 313. The configuration unit 339 configures the information to be output using, for example, the emotion information of the action management unit 313.

[0247] The component 339 constitutes output information having action information paired with each of two or more time zones on a calendar, for example, in the area specified by each of the two or more time zones. In the output information, it is preferable that two or more pieces of action information are arranged so that the confirmed action information and the unconfirmed action information can be visually distinguished. The component 339 preferably constitutes output information that visually reveals the emotion information corresponding to each of two or more time zones. The component 339 preferably constitutes output information in which, for example, a time zone with "positive" emotion information and a time zone with "negative" emotion information are visibly distinguishable. The component 339 preferably constitutes output information such that, for example, the background colors of a time zone with "positive" emotion information and a time zone with "negative" emotion information are different.

[0248] The output unit 34 outputs various types of information. The various types of information are, for example, action information, emotion information, and location information.

[0249] Here, output generally refers to display on a display, but it may also be a concept including projection using a projector, printing by a printer, transmission to an external device, storage in a recording medium, delivery of a processing result to another processing device or another program, etc.

[0250] The action output unit 341 outputs the action information in each of one or more time zones.

[0251] The action output unit 341 preferably outputs the location acquired by the location acquisition unit 337 when the action estimation unit 335 cannot acquire the action information.

[0252] The action output unit 341 preferably outputs two or more pieces of action information so that the confirmed action information and the unconfirmed action information can be visually distinguished.

[0253] The emotion output unit 342 outputs emotion information. The emotion output unit 342 outputs, for example, the emotion information acquired by the emotion estimation unit 336. The emotion output unit 342 preferably outputs the emotion information in each of one or more time zones.

[0254] In the server storage unit 41 that constitutes the server device 4, various types of information are stored. The various types of information are, for example, the learning information described above, the action source information and time associated with two or more user identifiers, two or more action teacher data, and two or more emotion teacher data.

[0255] The server reception unit 42 receives various instructions and information. The various instructions and information are, for example, an instruction to transmit information. The information here is, for example, learning information and action source information.

[0256] The server processing unit 43 performs various processes. The various processes are, for example, a learning process. The learning process is, for example, an action learning process and an emotion learning process.

[0257] The action learning process is a process of obtaining an action learning model using two or more action teacher data. For example, the server processing unit 43 provides two or more action teacher data to a learning processing module of machine learning, executes the module, obtains an action learning model, and accumulates it in the server storage unit 41. For example, for each candidate of two or more pieces of action information, the server processing unit 43 provides two or more positive examples, which are teacher data including the action information, and two or more negative examples, which are teacher data not including the action information, to a learning processing module of machine learning, executes the module, obtains an action learning model for each candidate of the action information, and accumulates it in the server storage unit 41 in association with the action information. For example, the server processing unit 43 obtains an action correspondence table, which is a table having two or more pieces of action teacher data as records, and accumulates it in the server storage unit 41.

[0258] Affective learning processing is a process of obtaining an affective learning model using two or more pieces of affective teacher data. The server processing unit 43, for example, provides two or more pieces of affective teacher data to a learning processing module of machine learning, executes the module, obtains an affective learning model, and stores it in the server storage unit 41. The server processing unit 43, for example, for each candidate of two or more pieces of each affective information, provides two or more positive examples that are teacher data including the affective information and two or more negative examples that are teacher data not including the affective information to a learning processing module of machine learning, executes the module, obtains an affective learning model for each candidate of the affective information, and stores it in the server storage unit 41 in association with the affective information. The server processing unit 43, for example, obtains an affect correspondence table that is a table having two or more pieces of each affective teacher data as records, and stores it in the server storage unit 41.

[0259] The server transmission unit 44 transmits various types of information. The various types of information are, for example, a behavior learning model, an affective learning model, a behavior correspondence table, and an affect correspondence table.

[0260] The storage unit 31, the learning management unit 311, the map management unit 312, the behavior management unit 313, and the server storage unit 41 are preferably non-volatile recording media, but can also be realized by volatile recording media.

[0261] The process by which information is stored in the storage unit 31 or the like is not limited. For example, information may be stored in the storage unit 31 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 31 or the like, or information input via an input device may be stored in the storage unit 31 or the like.

[0262] The reception unit 32, the server reception unit 42, and the server transmission unit 44 are usually realized by wireless or wired communication means.

[0263] The processing unit 33, time acquisition unit 331, position acquisition unit 332, activity acquisition unit 333, vital acquisition unit 334, action estimation unit 335, emotion estimation unit 336, location acquisition unit 337, storage unit 338, configuration unit 339, and server processing unit 43 can generally be realized from a processor, memory, etc. The processing procedures of the processing unit 33 etc. are generally realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). Note that the processor may be a CPU, MPU, GPU, etc., and its type is not limited.

[0264] The output unit 34, action output unit 341, and emotion output unit 342 may or may not be considered to include output devices such as a display and a speaker. The output unit 34 can be realized by the driver software of the output device or the driver software of the output device and the output device etc.

[0265] Next, an operation example of the action acquisition device 3 constituting the information system D will be described using the flowchart of FIG. 19.

[0266] (Step S1901) The processing unit 33 determines whether to acquire information. If acquiring information, it proceeds to step S1902, and if not acquiring information, it proceeds to step S1916. Note that the processing unit 33 may always determine to acquire information, or may determine to acquire information when a flag indicating information acquisition is stored in the storage unit 31 etc. The conditions for such determination are not limited.

[0267] (Step S1902) The time acquisition unit 331 acquires the time from a clock (not shown). Here, the time acquisition unit 331 may acquire the day of the week.

[0268] (Step S1903) The processing unit 33 acquires one or two or more action source information. An example of such action source acquisition processing will be described using the flowchart of FIG. 20.

[0269] (Step S1904) The action estimation unit 335 estimates action information for specifying the user's action using the one or more action source information acquired in step S1903. An example of such action estimation processing will be described using the flowcharts of FIGS. 22 to 24.

[0270] (Step S1905) The location acquisition unit 337 determines whether action information could be acquired in step S1904. If action information could be acquired, it proceeds to step S1907, and if action information could not be acquired, it proceeds to step S1906.

[0271] (Step S1906) The location acquisition unit 337 refers to the map of the map management unit 312 and acquires location information corresponding to the position information acquired in step S1903. Note that it may not be possible to acquire location information here.

[0272] (Step S1907) The storage unit 338 determines whether the action information stored in a buffer (not shown), which is the action information accumulated immediately before, matches the action information acquired in step S1904. If they match, it proceeds to step S1908, and if they do not match, it proceeds to step S1912.

[0273] (Step S1908) The storage unit 338 associates the acquired action information, etc. with the time acquired in step S1902 and stores it in a buffer (not shown).

[0274] (Step S1909) The emotion estimation unit 336 performs processing to estimate emotion information. An example of such emotion estimation processing will be described using the flowcharts of FIGS. 25 to 27.

[0275] (Step S1910) The storage unit 338 determines whether emotion information could be acquired in step S1909. If emotion information could be acquired, it proceeds to step S1911, and if emotion information could not be acquired, it returns to step S1901.

[0276] (Step S1911) The storage unit 338 stores the emotion information acquired in step S1909 in a buffer (not shown) in association with the time acquired in step S1902. Return to step S1901.

[0277] (Step S1912) The storage unit 338 stores the time acquired in step S1902 and the acquired action information, etc. in a buffer (not shown) in association with each other.

[0278] (Step S1913) The storage unit 338 acquires the previous action information, etc.

[0279] (Step S1914) The storage unit 338 acquires the time zone specified by two or more times associated with the previous action information, etc.

[0280] (Step S1915) The storage unit 338 associates the time zone acquired in step S1914 with the previous action information, etc. acquired in step S1913, and stores them in the action management unit 313. Here, the storage unit 338 may also associate them with the user identifier and store the time zone and the previous action information, etc. in association with each other. In such a case, it is preferable that the storage destination is the server device 4.

[0281] (Step S1916) The action acquisition device 3 determines whether it has received an output instruction. If it has received an output instruction, go to step S1917; if not, go to step S1919.

[0282] (Step S1917) The configuration unit 339 configures output information using the action information, etc. of the action management unit 313. An example of such output configuration processing will be described with reference to the flowchart of FIG. 28.

[0283] (Step S1918) The output unit 34 outputs the output information configured in step S1917. Return to step S1901.

[0284] (Step S1919) The behavior acquisition device 3 determines whether it has received an input of information with respect to the output information being output. If it has received an input of information, it proceeds to step S1920, and if it has not received the input, it proceeds to step S1923.

[0285] (Step S1920) The processing unit 33 determines whether the information received in step S1919 is a confirmation instruction for the estimated behavior information or the estimated emotion information. If it is a confirmation instruction, it proceeds to step S1921, and if it is not a confirmation instruction, it proceeds to step S1922.

[0286] (Step S1921) The storage unit 338 performs a process for confirming the behavior information or emotion information corresponding to the confirmation instruction. It returns to step S1901. Note that such a process is, for example, a process of associating a confirmation flag with the behavior information corresponding to the confirmation instruction or the emotion information corresponding to the confirmation instruction.

[0287] (Step S1922) The storage unit 338 stores the input information. It returns to step S1901. Note that the input information is, for example, correct behavior information or correct emotion information. Then, the storage unit 338 updates the estimated behavior information or emotion information corresponding to the input information to the input behavior information or emotion information. Further, the storage unit 338 performs a process for confirming such behavior information or emotion information.

[0288] (Step S1923) The behavior acquisition device 3 determines whether it has received a learning instruction. If it has received a learning instruction, it proceeds to step S1924, and if it has not received the instruction, it returns to step S1901.

[0289] (Step S1924) A learning unit (not shown) or a learning device (not shown) of the behavior acquisition device 3 constructs behavior learning information using two or more behavior teacher data including behavior information and stores it in the learning management unit 311. An example of such behavior learning processing will be described with reference to the flowcharts of FIGS. 29 and 30.

[0290] (Step S1925) A learning unit (not shown) or a learning device (not shown) of the behavior acquisition device 3 constructs emotion learning information using two or more emotion teacher data including emotion information and accumulates it in the learning management unit 311. Return to Step S1901. An example of such emotion learning processing will be described using the flowcharts of FIGS. 31 and 32.

[0291] Note that in the flowchart of FIG. 19, the process ends due to a power-off or a processing end interrupt.

[0292] Next, an example of the action source acquisition process in Step S1903 will be described using the flowchart of FIG. 20.

[0293] (Step S2001) The processing unit 33 determines whether the receiving unit 32 can acquire a GPS signal. If the GPS signal can be acquired, go to Step S2002; if the GPS signal cannot be acquired, go to Step S2003.

[0294] (Step S2002) The position acquisition unit 332 acquires absolute position information based on the GPS signal received by the receiving unit 32.

[0295] (Step S2003) The position acquisition unit 332 acquires position information. An example of such position estimation processing will be described using the flowchart of FIG. 21.

[0296] (Step S2004) The activity acquisition unit 333 acquires the user's activity data.

[0297] (Step S2005) The vital acquisition unit 334 acquires one or two or more types of the user's vital data.

[0298] (Step S2006) The action estimation unit 335 acquires one or more pieces of past action information. Note that the one or more pieces of past action information include the action information immediately before in time.

[0299] (Step S2007) The action estimation unit 335 determines whether to use the emotion information for the action estimation process. If using the emotion information, it proceeds to step S2008; if not using the emotion information, it proceeds to step S2009. Note that whether to use the emotion information for the action estimation process is usually determined in advance.

[0300] (Step S2008) The emotion estimation unit 336 acquires the emotion information. An example of such emotion estimation processing will be described using the flowcharts of FIGS. 25 to 27.

[0301] (Step S2009) The action estimation unit 335 acquires the elapsed time since the start of the new action specified by the new action information.

[0302] (Step S2010) The action estimation unit 335 constructs the information to be used in the action estimation process using two or more types of action source information including time and position information. Then it returns to the upper-level process. Here, the constructed information is usually a set of action source information, for example, an action source vector with two or more pieces of action source information as elements. The two or more types of action source information are, for example, two or more types of information among time, day of the week, position information, activity data, vital data, past action information, emotion information, and elapsed time.

[0303] Note that in the flowchart of FIG. 20, even when the GPS signal can be acquired, the position acquisition unit 332 may acquire the position information by the position estimation process described using the flowchart of FIG. 21.

[0304] Also, in the flowchart of FIG. 20, the action estimation unit 335 may acquire one or two or more user attribute values, and may acquire action source information including the one or more user attribute values. Note that the user attribute value may be information stored in the storage unit 31, or information input by the user or the like. Next, an example of the position estimation process in step S2003 will be described with reference to the flowchart of FIG. 21. In the flowchart of FIG. 21, the description of the same steps as in FIG. 13 will be omitted. Also, the position estimation process in step S2003 may be the same process as the flowchart of FIG. 13.

[0305] (Step S2101) The position acquisition unit 332 acquires the similarity between two radio wave intensity vectors, associates the similarity with the i-th point information, and temporarily stores it in a buffer (not shown). Proceed to step S1308.

[0306] (Step S2102) The position acquisition unit 332 acquires the position information included in the point information paired with the maximum similarity. Return to the upper-level process. Note that the acquired position information is the terminal position.

[0307] Next, an example of the first action estimation process in step S1904 will be described with reference to the flowchart of FIG. 22. The flowchart of FIG. 22 is a process of estimating action information by prediction processing of machine learning using one action learning model. That is, the flowchart of FIG. 22 is a process of estimating action information by prediction processing of multi-class classification of machine learning.

[0308] (Step S2201) The action estimation unit 335 acquires two or more types of action source information (for example, action source vectors) acquired in step S1903.

[0309] (Step S2202) The action estimation unit 335 acquires an action learning model from the learning management unit 311.

[0310] (Step S2203) The action estimation unit 335 provides two or more types of action source information acquired in Step S2201 and the action learning model to the prediction processing module of machine learning, and executes the module.

[0311] (Step S2204) The action estimation unit 335 acquires the estimated action information and score, which are the execution results in Step S2203.

[0312] (Step S2205) The action estimation unit 335 determines whether the score acquired in Step S2204 is equal to or greater than the threshold value. If it is equal to or greater than the threshold value, it proceeds to Step S2206; if it is less than the threshold value, it proceeds to Step S2207.

[0313] (Step S2206) The action estimation unit 335 acquires the action information acquired in Step S2204 as the output action information. Return to the upper-level process.

[0314] (Step S2207) The action estimation unit 335 acquires "empty" action information. Return to the upper-level process.

[0315] Note that in the flowchart of FIG. 22, the processes from Step S2205 to Step S2207 may not be performed.

[0316] Next, an example of the second action estimation process in Step S1904 will be described with reference to the flowchart of FIG. 23. The flowchart of FIG. 23 is a process of estimating action information by using the action learning model for each candidate of two or more pieces of action information through the prediction process of machine learning. That is, the flowchart of FIG. 22 is a process of estimating action information by using the prediction process of binary classification of machine learning.

[0317] (Step S2301) The action estimation unit 335 acquires two or more types of action source information acquired in Step S1903.

[0318] (Step S2302) The action estimation unit 335 assigns 1 to the counter i.

[0319] (Step S2303) The action estimation unit 335 refers to the learning management unit 311 and determines whether there is a candidate for the i-th action information. If there is a candidate for the i-th action information, it proceeds to step S2304; if not, it proceeds to step S2309.

[0320] (Step S2304) The action estimation unit 335 acquires the i-th action learning model corresponding to the candidate for the i-th action information from the learning management unit 311.

[0321] (Step S2305) The action estimation unit 335 provides the two or more types of action source information acquired in step S2301 and the i-th action learning model acquired in step S2304 to a prediction processing module that performs binary classification of machine learning, and executes the module.

[0322] (Step S2306) The action estimation unit 335 determines whether the execution result in step S2305 is "true". If it is "true", it proceeds to step S2307; if it is "false", it proceeds to step S2308.

[0323] (Step S2307) The action estimation unit 335 temporarily stores in a buffer (not shown) a score that is part of the execution result in step S2305, in association with the candidate for the i-th action information.

[0324] (Step S2308) The action estimation unit 335 increments the counter i by 1. It returns to step S2303.

[0325] (Step S2309) The action estimation unit 335 acquires the maximum score and determines whether the maximum score is greater than or equal to the threshold. If the maximum score is greater than or equal to the threshold, it proceeds to step S2310; if it is less than the threshold, it proceeds to step S2311.

[0326] (Step S2310) The action estimation unit 335 commends the action information corresponding to the maximum score. It returns to the upper-level process.

[0327] (Step S2311) The action estimation unit 335 acquires the action information of "empty". Return to the upper-level process.

[0328] Note that in the flowchart of FIG. 23, the processes from step S2308 to step S2311 may not be performed.

[0329] Next, an example of the third action estimation process of step S1904 will be described using the flowchart of FIG. 24. The flowchart of FIG. 24 is a process for estimating action information using an action correspondence table.

[0330] (Step S2401) The action estimation unit 335 acquires two or more types of action source information acquired in step S1903. The two or more types of action source information here are action source vectors.

[0331] (Step S2402) The action estimation unit 335 assigns 1 to the counter i.

[0332] (Step S2403) The action estimation unit 335 determines whether the i-th action correspondence information exists in the action correspondence table of the learning management unit 311. If the i-th action correspondence information exists, it proceeds to step S2404; if it does not exist, it proceeds to step S2407.

[0333] (Step S2404) The action estimation unit 335 acquires the i-th action source vector included in the i-th action correspondence information.

[0334] (Step S2405) The action estimation unit 335 acquires the similarity between the action source vector acquired in step S2403 and the action source vector acquired in step S2404, and associates it with the i-th action correspondence information.

[0335] (Step S2406) The action estimation unit 335 increments the counter i by 1. Return to step S2403.

[0336] (Step S2407) The action estimation unit 335 acquires the maximum similarity.

[0337] (Step S2408) The action estimation unit 335 determines whether the maximum similarity acquired in Step S2407 is greater than or equal to a threshold value. If it is greater than or equal to the threshold value, it proceeds to Step S2409; if it is less than the threshold value, it proceeds to Step S2410.

[0338] (Step S2409) The action estimation unit 335 acquires the action information associated with the i-th action correspondence information paired with the maximum similarity. It returns to the upper-level process.

[0339] (Step S2410) The action estimation unit 335 acquires the action information of "empty". It returns to the upper-level process.

[0340] Note that in the flowchart of FIG. 24, the processes from Step S2408 to Step S2411 may not be performed.

[0341] Next, an example of the first emotion estimation process in Step S1909 will be described using the flowchart of FIG. 25. The flowchart of FIG. 25 is a process of estimating emotion information by machine learning prediction processing using one emotion learning model. That is, the flowchart of FIG. 25 is a process of estimating emotion information by the prediction process of multi-class classification of machine learning.

[0342] (Step S2501) The emotion estimation unit 336 acquires two or more types of action source information acquired in Step S1903. Here, each of the two or more types of action source information is emotion source information. Also, the two or more types of emotion source information are, for example, emotion source vectors.

[0343] (Step S2502) The emotion estimation unit 336 acquires an emotion learning model from the learning management unit 311.

[0344] (Step S2503) The emotion estimation unit 336 provides two or more types of emotion source information acquired in Step S2501 and the emotion learning model to the prediction processing module of machine learning and executes the module.

[0345] (Step S2504) The emotion estimation unit 336 acquires the estimated emotion information and score, which are the execution results in Step S2503.

[0346] (Step S2505) The emotion estimation unit 336 determines whether the score acquired in Step S2504 is equal to or greater than the threshold value. If it is equal to or greater than the threshold value, it proceeds to Step S2506; if it is less than the threshold value, it proceeds to Step S2507.

[0347] (Step S2506) The emotion estimation unit 336 acquires the emotion information acquired in Step S2504 as the output emotion information. Return to the upper-level process.

[0348] (Step S2507) The emotion estimation unit 336 acquires "empty" emotion information. Return to the upper-level process.

[0349] Note that in the flowchart of FIG. 25, the processes from Step S2505 to Step S2507 may not be performed.

[0350] Next, an example of the second emotion estimation process in Step S1909 will be described with reference to the flowchart of FIG. 26. The flowchart of FIG. 26 is a process of estimating emotion information by using the emotion learning model for each candidate of two or more types of emotion information through the prediction process of machine learning. That is, the flowchart of FIG. 26 is a process of estimating emotion information by the prediction process of binary classification of machine learning.

[0351] (Step S2601) The emotion estimation unit 336 acquires two or more types of emotion source information acquired in Step S1903.

[0352] (Step S2602) The emotion estimation unit 336 assigns 1 to the counter i.

[0353] (Step S2603) The emotion estimation unit 336 refers to the learning management unit 311 to determine whether there is a candidate for the i-th emotion information. If there is a candidate for the i-th emotion information, it proceeds to step S2604; if not, it proceeds to step S2609.

[0354] (Step S2604) The emotion estimation unit 336 obtains the i-th emotion learning model corresponding to the candidate for the i-th emotion information from the learning management unit 311.

[0355] (Step S2605) The emotion estimation unit 336 provides the two or more types of emotion source information obtained in step S2601 and the i-th emotion learning model obtained in step S2604 to a prediction processing module that performs binary classification of machine learning, and executes the module.

[0356] (Step S2606) The emotion estimation unit 336 determines whether the execution result in step S2605 is "true". If it is "true", it proceeds to step S2607; if it is "false", it proceeds to step S2608.

[0357] (Step S2607) The emotion estimation unit 336 temporarily stores in a buffer (not shown) a score that is part of the execution result in step S2605, associated with the candidate for the i-th emotion information.

[0358] (Step S2608) The emotion estimation unit 336 increments the counter i by 1. It returns to step S2603.

[0359] (Step S2609) The emotion estimation unit 336 obtains the maximum score and determines whether the maximum score is greater than or equal to a threshold value. If the maximum score is greater than or equal to the threshold value, it proceeds to step S2610; if it is less than the threshold value, it proceeds to step S2611.

[0360] (Step S2610) The emotion estimation unit 336 commends the emotion information corresponding to the maximum score. It returns to the upper-level process.

[0361] (Step S2611) The emotion estimation unit 336 acquires the emotion information of "empty". Return to the upper-level process.

[0362] Note that in the flowchart of FIG. 26, the processes from step S2608 to step S2611 may not be performed.

[0363] Next, an example of the third emotion estimation process in step S1909 will be described using the flowchart of FIG. 27. The flowchart of FIG. 27 is a process of estimating emotion information using an emotion correspondence table.

[0364] (Step S2701) The emotion estimation unit 336 acquires two or more types of emotion source information acquired in step S1903. The two or more types of emotion source information here are emotion source vectors.

[0365] (Step S2702) The emotion estimation unit 336 substitutes 1 for the counter i.

[0366] (Step S2703) The emotion estimation unit 336 determines whether the i-th emotion correspondence information exists in the emotion correspondence table of the learning management unit 311. If the i-th emotion correspondence information exists, go to step S2704; if not, go to step S2707.

[0367] (Step S2704) The emotion estimation unit 336 acquires the i-th emotion source vector included in the i-th emotion correspondence information.

[0368] (Step S2705) The emotion estimation unit 336 acquires the similarity between the emotion source vector acquired in step S2703 and the emotion source vector acquired in step S2704, and associates it with the i-th emotion correspondence information.

[0369] (Step S2706) The emotion estimation unit 336 increments the counter i by 1. Return to step S2703.

[0370] (Step S2707) The emotion estimation unit 336 acquires the maximum similarity.

[0371] (Step S2708) The emotion estimation unit 336 determines whether the maximum similarity acquired in Step S2707 is greater than or equal to the threshold value. If it is greater than or equal to the threshold value, it proceeds to Step S2709; if it is less than the threshold value, it proceeds to Step S2710.

[0372] (Step S2709) The emotion estimation unit 336 acquires the emotion information associated with the i-th emotion correspondence information paired with the maximum similarity. Returns to the upper-level process.

[0373] (Step S2710) The emotion estimation unit 336 acquires "empty" emotion information. Returns to the upper-level process.

[0374] Note that in the flowchart of FIG. 27, the processes from Step S2708 to Step S2711 do not have to be performed.

[0375] Next, an example of the output configuration process in Step S1917 will be described using the flowchart of FIG. 28.

[0376] (Step S2801) The configuration unit 339 acquires the calendar template from the storage unit 31.

[0377] (Step S2802) The configuration unit 339 substitutes 1 into the counter i.

[0378] (Step S2803) The configuration unit 339 determines whether the i-th time slot stored in the action management unit 313 exists. If the i-th time slot exists, it proceeds to Step S2804; if it does not exist, it returns to the upper-level process. Note that in the action management unit 313, action information and emotion information are stored in association with each time slot of 1 or more.

[0379] (Step S2804) The component 339 determines whether the i-th time period stored in the action management unit 313 is included in the period covered by the calendar template acquired in Step S2801. If it is included, it proceeds to Step S2805; if not, it proceeds to Step S2809.

[0380] (Step S2805) The component 339 acquires the action information corresponding to the i-th time period from the action management unit 313.

[0381] (Step S2806) The component 339 acquires the emotion information corresponding to the i-th time period from the action management unit 313. Here, it is not necessary to be able to acquire the emotion information.

[0382] (Step S2807) The component 339 constructs time period information that is the information to be arranged in the i-th time period of the calendar, can identify the action information acquired in Step S2805, and can identify the emotion information acquired in Step S2806.

[0383] (Step S2808) The component 339 arranges the time period information constructed in Step S2807 at the position in the calendar specified by the i-th time period.

[0384] (Step S2809) The component 339 increments the counter i by 1 and returns to Step S2803.

[0385] Next, an example of the first action learning process in Step S1924 will be described using the flowchart in FIG. 29. It is assumed that the action learning process is performed by a learning unit (not shown), for example. The learning unit may be a learning device different from the action acquisition device 3. The first action learning process is a process of acquiring an action learning model for multi-class classification.

[0386] (Step S2901) The learning unit substitutes 1 for the counter i.

[0387] (Step S2902) The learning unit determines whether the i-th action information and the like exist in the action management unit 313. If the i-th action information and the like exist, it proceeds to step S2903; if not, it proceeds to step S2905.

[0388] (Step S2903) The learning unit uses the i-th action information and the like to construct action teacher data and append it to a buffer (not shown). Note that the action teacher data usually has two or more types of action source information as explanatory variables and the action information as the target variable.

[0389] (Step S2904) The learning unit increments the counter i by 1 and returns to step S2902.

[0390] (Step S2905) The learning unit provides the two or more action teacher data stored in the buffer (not shown) to the learning processing module of machine learning, executes the module, and obtains an action learning model.

[0391] (Step S2906) The learning unit stores the action learning model obtained in step S2905 in the learning management unit 311.

[0392] In the flowchart of FIG. 29, the learning unit may store in the learning management unit 311 an action correspondence table in which the two or more pieces of action teacher data in the buffer (not shown) are records (action correspondence information) without performing the learning processes of steps S2905 and S2906.

[0393] Next, an example of the second action learning process in step S1924 will be described using the flowchart of FIG. 30. The second action learning process is a process of obtaining a binary classification action learning model for each of two or more action information candidates.

[0394] (Step S3001) The learning unit assigns 1 to the counter i.

[0395] (Step S3002) The learning unit determines whether there is action information of the i-th type. If there is action information of the i-th type, it proceeds to step S3003; if not, it returns to the upper-level process.

[0396] (Step S3003) The learning unit acquires the action information of the i-th type.

[0397] (Step S3004) The learning unit acquires two or more positive examples that are teacher data in the action management unit 313 and are action teacher data including the action information of the i-th type.

[0398] (Step S3005) The learning unit acquires two or more negative examples that are teacher data in the action management unit 313 and are action teacher data not including the action information of the i-th type.

[0399] (Step S3006) The learning unit provides the two or more positive examples acquired in step S3004 and the two or more negative examples acquired in step S3005 to the learning processing module of machine learning, executes the module, and acquires an action learning model.

[0400] (Step S3007) The learning unit stores the action learning model acquired in step S3006 in the learning management unit 311 in pair with the action information of the i-th type.

[0401] (Step S3008) The learning unit increments the counter i by 1 and returns to step S3002.

[0402] Next, an example of the first emotion learning process in step S1925 will be described with reference to the flowchart of FIG. 31. It is assumed that the action learning process is performed by a learning unit (not shown), for example. The learning unit may be a learning device different from the action acquisition device 3. The first emotion learning process is a process of acquiring a multi-class emotion learning model.

[0403] (Step S3101) The learning unit assigns 1 to the counter i.

[0404] (Step S3102) The learning unit determines whether the i-th emotion information or the like exists in the behavior management unit 313. If the i-th emotion information or the like exists, it proceeds to step S3103; if it does not exist, it proceeds to step S3105.

[0405] (Step S3103) The learning unit uses the i-th emotion information or the like to construct emotion teacher data and append it to a buffer (not shown). Note that the emotion teacher data is information that uses two or more types of emotion source information as explanatory variables and emotion information as the target variable.

[0406] (Step S3104) The learning unit increments the counter i by 1 and returns to step S3102.

[0407] (Step S3105) The learning unit provides the two or more pieces of emotion teacher data stored in a buffer (not shown) to the learning processing module of machine learning, executes the module, and obtains an emotion learning model.

[0408] (Step S3106) The learning unit stores the emotion learning model obtained in step S3105 in the learning management unit 311.

[0409] Note that in the flowchart of FIG. 31, the learning unit may store in the learning management unit 311 an emotion correspondence table in which two or more pieces of emotion teacher data in a buffer (not shown) are records (emotion correspondence information) without performing the learning processes of steps S3105 and S3106.

[0410] Next, an example of the second emotion learning process in step S1925 will be described using the flowchart of FIG. 32. The second emotion learning process is a process of obtaining a binary classification emotion learning model for each candidate of two or more pieces of emotion information.

[0411] (Step S3201) The learning unit assigns 1 to the counter i.

[0412] (Step S3202) The learning unit determines whether the i-th type of emotion information exists. If the i-th type of emotion information exists, it proceeds to step S3203; if it does not exist, it returns to the upper-level process.

[0413] (Step S3203) The learning unit acquires the i-th type of emotion information.

[0414] (Step S3204) The learning unit acquires two or more positive examples that are emotion teacher data in the action management unit 313 and include the i-th type of emotion information.

[0415] (Step S3205) The learning unit acquires two or more negative examples that are emotion teacher data in the action management unit 313 and do not include the i-th type of emotion information.

[0416] (Step S3206) The learning unit provides the two or more positive examples acquired in step S3204 and the two or more negative examples acquired in step S3205 to the learning processing module of machine learning, executes the module, and acquires an emotion learning model.

[0417] (Step S3207) The learning unit stores the emotion learning model acquired in step S3206 in the learning management unit 311 in pair with the i-th type of emotion information.

[0418] (Step S3208) The learning unit increments the counter i by 1 and returns to step S3202.

[0419] Hereinafter, a specific operation example of the information system D in this embodiment will be described. Now, in the storage unit 31 of the server device 4, an action learning model acquired by learning processing of machine learning using a large number of teacher data including action source information of one or two or more users is stored. Also, in the storage unit 31, an emotion learning model acquired by learning processing of machine learning using a large number of teacher data including emotion source information of one or two or more users is stored.

[0420] In addition, in the storage unit 31 of the behavior acquisition device 3, which is a terminal (e.g., a smartwatch) held by the user "U1", a large number of behavior source information at two or more consecutive times acquired by the processing unit 33 is stored in the behavior source management table shown in FIG. 33 by the above-described processing.

[0421] The behavior source management table (FIG. 33) is a record having "ID", "time information", "physical data", "location information", and "elapsed time". "ID" is information for identifying a record. "Time information" is information for specifying time, and here it has "month / day", "time", and "day of the week". "Physical data" has "activity data" and "vital data". "Vital data" has "heart rate", "systolic blood pressure", "diastolic blood pressure", and "body temperature". "Heart rate" is the heart rate per unit time (here, "1 minute"). "Systolic blood pressure" is the maximum blood pressure, and "diastolic blood pressure" is the minimum blood pressure. "Location information" is, for example, the location information acquired by the processing described in Embodiment 2. "Elapsed time" is the time elapsed since the start of the same behavior.

[0422] Then, it is assumed that the user "U1" inputs an output instruction to the behavior acquisition device 3. Then, the behavior acquisition device 3 receives the output instruction.

[0423] Next, the behavior estimation unit 335 accesses, for example, the server device 4, receives a behavior learning model and an emotion learning model from the server device 4, and stores the behavior learning model and the emotion learning model in the learning management unit 311.

[0424] Next, the action estimation unit 335 constructs an action source vector having time information, body data, position information, elapsed time, etc. for each record in FIG. 33 by the process described using the flowchart of FIG. 22, for example. Next, the action estimation unit 335 acquires an action learning model from the learning management unit 311. Next, the action estimation unit 335 provides the action source vector and the action learning model to the prediction processing module of machine learning and executes the module. Then, it is assumed that the action estimation unit 335 acquires action information "A1" for the action source vectors from the record of "ID=1" to the record of "ID=289", and acquires action information "A2" for the action source vectors from the record of "ID=290" to the record of "ID=N". Then, the storage unit 338 stores the action information and the action confirmation flag "0" in the action management unit 313 for each record. Note that "0" of the action confirmation flag and the emotion confirmation flag indicates uncertainty, and "1" indicates certainty.

[0425] Also, the emotion estimation unit 336 constructs an emotion source vector having time information, body data, position information, elapsed time, action information, etc. for each record in FIG. 33 by the process described using the flowchart of FIG. 25, for example. Next, the emotion estimation unit 336 acquires an emotion learning model from the learning management unit 311. Next, the emotion estimation unit 336 provides the emotion source vector and the emotion learning model to the prediction processing module of machine learning and executes the module. Then, it is assumed that the emotion estimation unit 336 acquires emotion information "E1" for the action source vectors from the record of "ID=1" to the record of "ID=289", and acquires emotion information "E2" for the action source vectors from the record of "ID=290" to the record of "ID=N". Then, the storage unit 338 stores the emotion information and the emotion confirmation flag "0" in the action management unit 313 for each record.

[0426] As a result of the above processing, the action-emotion management table shown in FIG. 34 is stored in the action management unit 313. The action-emotion management table has two or more records having "ID", "action information", "action confirmation flag", "emotion information", and "emotion confirmation flag".

[0427] Next, for each record in FIGS. 33 and 34, when the accumulation unit 338 pairs with the same action information (for example, "A1"), the times (for example, T 001 , ···, T 002 , T 289 ) of the time period (for example, "T 001 to T 289 ", which is "TZ1") are obtained, and the time period and the action information are associated and accumulated.

[0428] Also, for each record in FIGS. 33 and 34, when the accumulation unit 338 pairs with the same emotion information (for example, "E1"), the times (for example, T 001 , ···, T 002 , T 289 ) of the time period (for example, "T 001 to T 289 ", which is "TZ1") are obtained, and the time period and the emotion information are associated and accumulated. At this stage, both the action determination flag and the emotion determination flag corresponding to each time period are "0". And an example of the accumulated information is shown in FIG. 35. FIG. 35 is a time period information management table. The time period information management table has one or more records having "ID", "time period", "action information", "action determination flag", "emotion information", and "emotion determination flag".

[0429] Then, the component 339 uses one or more sets of the time period, the action information, and the emotion information accumulated by the accumulation unit 338 to perform the process described with reference to the flowchart of FIG. 28, constructs the time period information for each action information, arranges it in the form of a calendar template, and constructs the output information.

[0430] Next, the output unit 34 outputs the output information. Such an output example is shown in FIG. 36. In FIG. 36, the estimated action information for each time period of each day in the calendar is displayed.

[0431] Then, if the action information for each time period is correct, user "U1" inputs a "confirmation instruction" for the displayed action information. If the estimated action information is incorrect, user "U1" inputs the correct action information. Also, if the emotion information for each time period is correct, user "U1" inputs a "confirmation instruction" for the output emotion information. If the estimated emotion information is incorrect, user "U1" inputs the correct emotion information. With the user's input, the "action information", "action confirmation flag", "emotion information", and "emotion confirmation flag" in FIG. 35 will be changed.

[0432] As described above, according to this embodiment, the user's actions can be estimated using the location information corresponding to the time.

[0433] Also, according to this embodiment, the user's actions can be estimated using the location information corresponding to the time and the user's activity data corresponding to the time.

[0434] Also, according to this embodiment, the user's actions can be estimated with higher accuracy using the location information corresponding to the time, the user's activity data corresponding to the time, and the user's vital data corresponding to the time.

[0435] Also, according to this embodiment, the emotion of the user during the action can be estimated.

[0436] Also, according to this embodiment, the user's actions can be estimated with higher accuracy using past records.

[0437] Also, according to this embodiment, the user's actions can be estimated with higher accuracy using the past records of two or more users.

[0438] Also, according to this embodiment, when the user's actions cannot be estimated, the location where the user was can be output.

[0439] Furthermore, according to this embodiment, using the location information such as indoor location information obtained by the location information acquisition method described in Embodiment 2, the user's actions can be estimated with high accuracy even in a place where a GPS signal cannot be received.

[0440] Note that the processing in this embodiment may be implemented by software. And this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software for realizing the information system D in this embodiment is a program as follows. That is, this program causes a computer to function as a time acquisition unit that acquires time, a position acquisition unit that acquires position information associated with the time, and an action estimation unit that acquires action information for specifying the user's action in the time zone specified by the time included in two or more pieces of action source information including the position information associated with the time, using the two or more pieces of action source information, and an action output unit that outputs the action information in the time zone.

[0441] (Embodiment 4) The difference between this embodiment and Embodiment 3 is as follows. That is, in this embodiment, the action acquisition device is a server, and the action acquisition device estimates the user's action information and emotion information using the position information and the like received from the user's terminal device.

[0442] FIG. 37 is a conceptual diagram of the information system E in this embodiment. The information system E includes an action acquisition device 5, one or two or more terminal devices 6, and one or two or more communication devices B.

[0443] The action acquisition device 5 is a server, for example, a cloud server, an ASP server, and its type does not matter. The action acquisition device 5 is a device that receives action source information such as position information from the user's terminal device 6, estimates the user's action information using the action source information, and transmits the action information to the terminal device 6. The action acquisition device 5 is a device that receives emotion source information from the user's terminal device 6, estimates the user's emotion information using the emotion source information, and transmits the action information to the terminal device 6.

[0444] The terminal device 6 is a terminal used by the user. The terminal device 6 is, for example, a smartphone, a tablet terminal, a smartwatch, a so-called personal computer, etc., and its type is not limited. The terminal device 6 is a terminal that transmits action source information and emotion source information including position information, etc. to the action acquisition device 5, receives action information and emotion information from the action acquisition device 5, and outputs them. Note that the device that transmits the action source information and emotion source information to the action acquisition device 5 and the device that receives and outputs the action information and emotion information from the action acquisition device 5 may be different devices.

[0445] FIG. 38 is a block diagram of the information system E in the present embodiment. FIG. 39 is a block diagram of the action acquisition device 5.

[0446] The action acquisition device 5 includes a storage unit 51, a reception unit 52, a processing unit 53, and a transmission unit 54. The storage unit 51 includes a learning management unit 311 and an action management unit 313. The reception unit 52 includes a position acquisition unit 521, an activity acquisition unit 522, and a vital acquisition unit 523. The processing unit 53 includes a time acquisition unit 331, an action estimation unit 335, an emotion estimation unit 336, an accumulation unit 338, and a configuration unit 339. The transmission unit 54 includes an action output unit 341 and an emotion output unit 342.

[0447] The terminal device 6 includes a terminal storage unit 61, a terminal reception unit 62, a terminal reception unit 63, a terminal processing unit 64, a terminal transmission unit 65, and a terminal output unit 66. The terminal storage unit 61 includes a map management unit 312. The terminal processing unit 64 includes an intensity acquisition unit 231, a type determination unit 232, a position acquisition unit 332, an activity acquisition unit 333, a vital acquisition unit 334, and a location acquisition unit 337.

[0448] Various types of information are stored in the storage unit 51 that constitutes the action acquisition device 5. The various types of information are, for example, the learning information and the action information described above.

[0449] The reception unit 52 receives various instructions and information from the terminal device 6. The various instructions and information are, for example, position information, activity data, vital data, output instructions, confirmation instructions, action information to be corrected, and emotion information to be corrected.

[0450] The receiving unit 52 preferably receives position information, activity data, vital data, etc. from the terminal device 6 at once. The receiving unit 52 preferably receives position information etc. associated with the user identifier at once. The user identifier is information for identifying the user who uses the terminal device 6. The user identifier is, for example, a user ID, a telephone number, an email address, or an identifier of the terminal device 6. The identifier of the terminal device 6 is, for example, an IP address.

[0451] The position acquisition unit 521 receives position information from the terminal device 6. Such position information is associated with a time. When receiving the position information, the position acquisition unit 521 preferably acquires the time from a clock (not shown) and associates the time with the position information. Such position information is preferably associated with the user identifier.

[0452] The activity acquisition unit 522 receives activity data from the terminal device 6. Such activity data is associated with a time. When receiving the activity data, the activity acquisition unit 522 preferably acquires the time from a clock (not shown) and associates the time with the activity data. Such activity data is preferably associated with the user identifier.

[0453] The vital acquisition unit 523 receives one or two or more types of vital data from the terminal device 6. Such vital data is associated with a time. When receiving the vital data, the vital acquisition unit 523 preferably acquires the time from a clock (not shown) and associates the time with the vital data. Such vital data is preferably associated with the user identifier.

[0454] The processing unit 53 performs various processes. The various processes are, for example, processes performed by the time acquisition unit 331, the action estimation unit 335, the emotion estimation unit 336, and the storage unit 338.

[0455] The transmission unit 54 transmits various types of information to the terminal device 6. The various types of information are, for example, the estimated behavior information, the estimated emotion information, and the output information configured by the component 339.

[0456] The behavior output unit 341 is the behavior information acquired by the behavior estimation unit 335, and transmits the behavior information associated with the time zone to the terminal device 6.

[0457] The emotion output unit 342 is the emotion information acquired by the emotion estimation unit 336, and transmits the emotion information associated with the time zone to the terminal device 6.

[0458] Various types of information are stored in the terminal storage unit 61 that constitutes the terminal device 6. The various types of information are, for example, location information, activity data, and vital data.

[0459] The terminal reception unit 62 receives various types of instructions and information. The various types of instructions and information are, for example, an output instruction, a confirmation instruction, the behavior information modified by the user for the estimated behavior information, and the emotion information modified by the user for the estimated emotion information.

[0460] The input means for the various types of instructions and information can be anything, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0461] The terminal reception unit 63 receives various types of information from the behavior acquisition device 5. The various types of information are, for example, output information, behavior information, and emotion information.

[0462] The terminal processing unit 64 performs various types of processing. The various types of processing are, for example, the processing of changing the instructions and information received by the terminal reception unit 62 into the instructions and information of the transmission structure, and the processing of changing the information received by the terminal reception unit 63 into the output structure.

[0463] The terminal transmission unit 65 transmits various types of instructions and information. The various types of instructions and information are, for example, an output instruction, a confirmation instruction, the behavior information to be changed, and the emotion information to be changed.

[0464] The terminal output unit 66 outputs various types of information. The various types of information are, for example, output information, action information, and emotion information.

[0465] The storage unit 51 and the terminal storage unit 61 are preferably non-volatile recording media, but can also be realized with volatile recording media.

[0466] The process by which information is stored in the storage unit 51 or the like is not limited. For example, information may be stored in the storage unit 51 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 51 or the like, or information input via an input device may be stored in the storage unit 51 or the like.

[0467] The receiving unit 52, the position acquisition unit 521, the activity acquisition unit 522, the vital acquisition unit 523, the transmitting unit 54, the action output unit 341, the emotion output unit 342, the terminal receiving unit 63, and the terminal transmitting unit 65 are realized by, for example, wireless or wired communication means.

[0468] The processing unit 53 and the terminal processing unit 64 can usually be realized from a processor, memory, etc. The processing procedures of the processing unit 53 or the like are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (a dedicated circuit). The processor is, for example, a CPU, MPU, GPU, etc., and its type is not limited.

[0469] The terminal reception unit 62 can be realized by a device driver of an input means such as a touch panel or a keyboard, control software for a menu screen, etc.

[0470] The terminal output unit 66 may or may not be considered to include output devices such as a display and a speaker. The terminal output unit 66 can be realized by driver software for the output device or by driver software for the output device and the output device or the like.

[0471] Next, an operation example of the action acquisition device 5 will be described using the flowchart of FIG. 40. In the flowchart of FIG. 40, the description of the same steps as those in FIG. 19 will be omitted.

[0472] (Step S4001) The receiving unit 52 determines whether it has received position information or the like paired with a user identifier from the terminal device 6. If it has received the position information or the like, it proceeds to step S4002, and if it has not received it, it proceeds to step S4004. Note that the position information or the like is, for example, a user identifier and position information. The position information or the like is, for example, one or more types of information among a user identifier and position information, and activity data and vital data.

[0473] (Step S4002) The time acquisition unit 331 acquires the time from a clock (not shown). Here, the time acquisition unit 331 may acquire the day of the week. Also, the time usually includes hours and minutes. The time may include any one or more types of information among year, month, and day.

[0474] (Step S4003) The storage unit 338 stores the position information or the like received in step S4001 and the time in the action management unit 313 in association with the user identifier. It returns to step S4001.

[0475] (Step S4004) The receiving unit 52 determines whether it has received an output instruction from the terminal device 6. If it has received the output instruction, it proceeds to step S4005, and if it has not received it, it proceeds to step S4009. Note that the received output instruction is usually associated with a user identifier. Also, the output instruction usually includes period information (e.g., "from December 17, 2023 to December 23, 2023") that specifies the period for acquiring action information.

[0476] (Step S4005) The action estimation unit 335 substitutes 1 into the counter i.

[0477] (Step S4006) The action estimation unit 335 determines whether the i-th action source information paired with the user identifier associated with the output instruction received in step S4004 exists in the action management unit 313. If the i-th action source information exists, it proceeds to step S4007; if it does not exist, it proceeds to step S1917.

[0478] (Step S4007) The action estimation unit 335 determines whether action information associated with the i-th action source information exists. If the action information exists, it proceeds to step S4008; if it does not exist, it proceeds to step S1904. Note that when action information associated with the action source information exists, usually, action information has already been estimated using the action source information.

[0479] (Step S4008) The action estimation unit 335 increments the counter i by 1. It returns to step S4006.

[0480] (Step S4009) The reception unit 52 determines whether it has received information or the like from the terminal device 6. If it has received information or the like, it proceeds to step S1920; if it has not received it, it proceeds to step S4010. Note that the information or the like is, for example, a confirmation instruction, action information to be corrected, or emotion information to be corrected.

[0481] (Step S4010) The reception unit 52 determines whether it has received a learning instruction from the terminal device 6. If it has received a learning instruction, it proceeds to step S1924; if it has not received it, it returns to step S4001.

[0482] (Step S4011) The transmission unit 54 transmits the output information configured in step S1917 to the terminal device 6. Return to step S4001.

[0483] Note that in the flowchart of FIG. 40, the process ends due to a power-off or a processing end interrupt.

[0484] Next, an operation example of the terminal device 6 will be described using the flowchart of FIG. 41. In the flowchart of FIG. 41, the description of the same steps as those in FIG. 19 will be omitted.

[0485] (Step S4101) The terminal transmission unit 65 acquires the user identifier in the terminal storage unit 61, and transmits the position information and the like acquired in step S1903 to the behavior acquisition device 5 in association with the user identifier.

[0486] (Step S4102) The terminal transmission unit 65 transmits the output instruction received in step S1916 to the behavior acquisition device 5 in association with the user identifier in the terminal storage unit 61. Note that the output instruction usually includes period information.

[0487] (Step S4103) The terminal reception unit 63 determines whether or not it has received output information from the behavior acquisition device 5. If it has received the output information, it proceeds to step S4104, and if it has not received it, it returns to step S4103.

[0488] (Step S4104) The terminal processing unit 64 configures the output information to be output using the received output information. The terminal output unit 66 outputs the output information. It returns to step S1901.

[0489] (Step S4105) The terminal transmission unit 65 transmits the information and the like acquired from the information input received in step S1919 to the behavior acquisition device 5 in association with the user identifier in the terminal storage unit 61. It returns to step S1901.

[0490] Note that the information and the like are, for example, a confirmation instruction, the changed behavior information, and the changed emotion information. Note that the confirmation instruction includes information specifying the behavior information or emotion information to be confirmed. The changed behavior information is associated with the information specifying the behavior information to be corrected. The changed emotion information is associated with the information specifying the emotion information to be corrected.

[0491] Note that in the flowchart of FIG. 41, the process ends due to a power-off or a processing end interrupt.

[0492] As described above, according to the present embodiment, the behavior of the user can be estimated using the position information corresponding to the time.

[0493] Also, according to the present embodiment, the behavior of the user can be estimated using the position information corresponding to the time and the activity data of the user corresponding to the time.

[0494] Also, according to the present embodiment, the behavior of the user can be estimated with higher accuracy using the position information corresponding to the time, the activity data of the user corresponding to the time, and the vital data of the user corresponding to the time.

[0495] Also, according to the present embodiment, the emotion of the user during the behavior can be estimated.

[0496] Also, according to the present embodiment, the behavior of the user can be estimated with higher accuracy using past records.

[0497] Also, according to the present embodiment, the behavior of the user can be estimated with higher accuracy using the past records of two or more users.

[0498] Also, according to the present embodiment, when the behavior of the user cannot be estimated, the location where the user was can be output.

[0499] Furthermore, the software that realizes the behavior acquisition device 5 in the present embodiment is a program as follows. That is, this program causes a computer to function as a time acquisition unit that acquires time, a position acquisition unit that acquires position information associated with the time, an action estimation unit that acquires action information that specifies the behavior of the user in the time zone specified by the time included in two or more pieces of action source information including the position information associated with the time, and an action output unit that outputs the action information in the time zone.

[0500] In addition, each of the devices with reference numeral 4 in FIG. 16 and reference numeral 6 in FIG. 37 shows the appearance of a computer that executes the programs described in this specification to realize the action acquisition device and the like of various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. Each of the devices with reference numeral 4 in FIG. 16 and reference numeral 6 in FIG. 37 is an overview diagram of this computer system 300, and FIG. 42 is a block diagram of the system 300.

[0501] In each of the devices with reference numeral 4 in FIG. 16 and reference numeral 6 in FIG. 37, the computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0502] In FIG. 42, in addition to the CD-ROM drive 3012, the computer 301 includes an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012 and the like, a ROM 3015 for storing programs such as a boot-up program, and an RAM 3016 connected to the MPU 3013 for temporarily storing instructions of an application program and providing a temporary storage space, and a hard disk 3017 for storing an application program, a system program, and data. Here, although not shown, the computer 301 may further include a network card that provides a connection to a LAN.

[0503] A program for causing the computer system 300 to execute functions such as the action acquisition device of the above-described embodiment may be stored in a CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 when executed. The program may be loaded directly from the CD-ROM 3101 or the network.

[0504] The program does not necessarily include an operating system (OS) that causes the computer 301 to execute the functions of the action acquisition device and the like in the above-described embodiments, or a third-party program or the like. The program only needs to include only the part of the instructions that calls appropriate functions (modules) in a controlled manner so as to obtain a desired result. How the computer system 300 operates is well-known, and a detailed description thereof will be omitted.

[0505] In the above program, in steps such as transmitting information and receiving information, processes performed by hardware, for example, processes performed by a modem or an interface card in the transmission step (processes that can only be performed by hardware) are not included.

[0506] Also, the computer that executes the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

[0507] Also, in each of the above embodiments, it goes without saying that two or more communication means existing in one device may be physically realized by one medium.

[0508] Also, in each of the above embodiments, each process may be realized by being centrally processed by a single device, or may be realized by being distributedly processed by a plurality of devices.

[0509] It goes without saying that the present invention is not limited to the above embodiments, and various modifications are possible and are also included within the scope of the present invention.

Industrial Applicability

[0510] As described above, the action acquisition device according to the present invention has an effect that it can estimate the behavior of a user using position information corresponding to time, and is useful, for example, as a smart watch or a smart phone carried by the user.

Explanation of Symbols

[0511] D, E Information System 6 Terminal Devices 3, 5 Action Acquisition Devices 4 Server Devices 32, 52 Receiver 31, 51 Storage Unit 33, 53 Processing Unit 34 Output Unit 41 Server Storage Unit 42 Server Receiver 43 Server Processing Unit 44 Server Transmitter 54 Transmitter 61 Terminal Storage Unit 62 Terminal Reception Unit 63 Terminal Receiver 64 Terminal Processing Unit 65 Terminal Transmitter 66 Terminal Output Unit 231 Strength Acquisition Unit 232 Type Judgment Unit 332, 521 Position Acquisition Unit 311 Learning Management Unit 312 Map Management Unit 313 Action Management Unit 331 Time Acquisition Unit 333, 522 Activity Acquisition Unit 334, 523 Vital Acquisition Unit 335 Action Estimation Unit 336 Emotion Estimation Unit 337 Location Acquisition Unit 338 Accumulation Unit 339 Composition Unit 341 Action Output Unit 342 Emotion Output Unit

Claims

1. a time acquisition unit that acquires time; a position acquisition unit that acquires position information associated with the time; an action estimation unit that acquires action information for specifying the user's action in a time zone specified by the time included in the two or more pieces of action source information including the position information associated with the time; an action acquisition device comprising an action output unit that outputs the action information in the time zone.

2. further comprising an activity acquisition unit that acquires the user's activity data associated with the time or a vital acquisition unit that acquires one or two or more types of the user's vital data associated with the time, wherein the action estimation unit acquires the action information in the time zone by using two or more pieces of action source information including the position information associated with the time, the activity data, or one or more types of the vital data, according to the action acquisition device of Claim 1.

3. an activity acquisition unit that acquires the user's activity data associated with the time; a vital acquisition unit that acquires one or two or more types of the user's vital data associated with the time; wherein the action estimation unit acquires the action information in the time zone by using two or more pieces of action source information including the position information associated with the time, the activity data, and one or more types of the vital data, according to the action acquisition device of Claim 1.

4. a sentiment estimation unit that acquires sentiment information regarding the user's sentiment in the time zone by using the action source information; wherein the action estimation unit also acquires the action information in the time zone by using the sentiment information, according to the action acquisition device of any one of Claims 1 to 3.

5. wherein the action estimation unit also acquires the action information in the time zone by using one or more pieces of past action information including the immediately preceding action information, according to the action acquisition device of any one of Claims 1 to 3.

6. further comprising a sentiment estimation unit that acquires sentiment information regarding the user's sentiment in the time zone by using the action information or the action source information; and a sentiment output unit that outputs the sentiment information, according to the action acquisition device of Claim 1.

7. wherein the sentiment estimation unit The learning information stored in the learning management unit having the learning information based on two or more pieces of teacher data having one or two or more pieces of action source information or action information and emotion information, and the action information acquired by the action estimation unit or the action source information that is the source of acquiring the action information are used to acquire the emotion information in the time zone. The action acquisition device according to claim 6.

8. The action estimation unit The learning information stored in the learning management unit having the learning information based on two or more pieces of teacher data having one or two or more pieces of action source information and action information, and two or more pieces of the action source information including the position information associated with the time are used to acquire the action information in the time zone. The action acquisition device according to claim 1.

9. The action information included in at least one piece of teacher data among the two or more pieces of teacher data is the action information input by the user. The action acquisition device according to claim 7.

10. The action information included in at least two or more pieces of teacher data among the two or more pieces of teacher data is the action information for two or more users. The action acquisition device according to claim 9.

11. Further comprising a location acquisition unit that refers to a map management unit storing a map having location information corresponding to the position information, and acquires the location information corresponding to the position information acquired by the position acquisition unit, The action output unit When the action estimation unit cannot acquire the action information, outputs the location information acquired by the location acquisition unit. The action acquisition device according to claim 1.

12. A receiving unit that receives radio waves including device identifiers for identifying the communication devices from three or more communication devices, For each of the three or more communication devices, an intensity acquisition unit that acquires time-series radio wave intensities, A type determination unit that uses the time-series radio wave intensities acquired by the intensity acquisition unit to determine whether each of the three or more communication devices is a fixed terminal fixed or a mobile terminal moving, The position acquisition unit Uses the radio wave intensities of the three or more communication devices determined by the type determination unit to be fixed terminals to acquire the position information indoors. The action acquisition device according to claim 1.

13. Further comprising an accumulation unit that associates the action information acquired by the action estimation unit with the time zone and accumulates the action information associated with each of the two or more time zones in an action management unit, It is possible to determine whether the action information is determined action information, The action output unit The action acquisition device according to claim 1, which outputs two or more pieces of action information so that the determined action information and the undetermined action information can be visually distinguished.

14. An action acquisition method realized by a time acquisition unit, a position acquisition unit, an action estimation unit, and an action output unit, comprising: a time acquisition step in which the time acquisition unit acquires a time; a position acquisition step in which the position acquisition unit acquires position information associated with the time; an action estimation step in which the action estimation unit acquires action information for specifying the action of the user in a time zone specified by the time included in two or more pieces of action source information including the position information associated with the time, using the two or more pieces of action source information; and an action output step in which the action output unit outputs the action information in the time zone.

15. A program for causing a computer to function as a time acquisition unit that acquires a time, a position acquisition unit that acquires position information associated with the time, an action estimation unit that acquires action information for specifying the action of the user in a time zone specified by the time included in two or more pieces of action source information including the position information associated with the time, using the two or more pieces of action source information, and an action output unit that outputs the action information in the time zone.

Images