Speech output device, speech output method, and program

Abstract

Previously, it was not possible to output appropriate responses for a user based on their behavioral information. [Solution] A speech output device H1 comprises a receiving unit H12 that receives time-series information including one or more pieces of information on the user's actions in time, a speech acquisition unit H131 that acquires speech information that identifies speech directed to the user and corresponds to the time-series information received by the receiving unit H12, and a speech output unit H141 that outputs the speech information acquired by the speech acquisition unit H131. Using the user's action information, an appropriate speech can be output directed to the user by the speech output device H1.

Description

[Technical Field]

[0001] This invention relates to a speech output device, etc., that acquires and outputs speech directed at a user. [Background technology]

[0002] Conventionally, there have been information processing devices that use generative AI to improve search performance in order to create appropriate answers to user questions (see Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 7564400 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, conventional technology could not output appropriate responses to a user based on their behavioral information. Furthermore, conventional technology could not output appropriate responses to a user based on their physical or mental score derived from their behavioral information. [Means for solving the problem]

[0005] The speech output device of the first invention comprises a receiving unit that receives time-series information including one or more pieces of action information of the user in time series, a speech acquisition unit that acquires speech information that identifies a speech directed to the user and corresponds to the time-series information received by the receiving unit, and a speech output unit that outputs the speech information acquired by the speech acquisition unit.

[0006] With this configuration, it is possible to output appropriate responses for the user using the user's behavior information.

[0007] Furthermore, the speech output device of this second invention further comprises a feedback acquisition unit that acquires feedback information for the output of speech information, and the speech acquisition unit is a speech output device that acquires speech information using the feedback information acquired by the feedback acquisition unit.

[0008] This configuration allows for the output of more appropriate responses to the user by providing feedback on their speech.

[0009] Furthermore, the speech output device of this third invention differs from the second invention in that the feedback acquisition unit acquires the user's facial image when the speech output unit outputs speech information, and acquires feedback information based on the facial image.

[0010] This configuration allows for the output of more appropriate speech to a user by providing feedback based on the analysis of the user's facial image.

[0011] Furthermore, the speech output device of this fourth invention, compared to the second invention, is a speech output device in which the feedback acquisition unit acquires the user's voice information when the speech output unit outputs speech information, and acquires feedback information based on the voice information.

[0012] This configuration allows for the output of more appropriate responses to the user by providing feedback based on the analysis of the voice information emitted by the user.

[0013] Furthermore, the speech output device of this fifth invention is a speech output device in which, with respect to any one of the first to fourth inventions, the speech acquisition unit acquires input information based on time-series information, provides the input information to the LLM, and acquires speech information from the LLM.

[0014] With this configuration, by providing user behavior information to the LLM, it is possible to output appropriate utterances for that user.

[0015] Moreover, the speech output device of the sixth invention is a speech output device that is a dialogue robot for any one of the first to fifth inventions.

[0016] With such a configuration, it is possible to provide a robot that outputs an appropriate speech to the user by using the user's action information.

Effect of the Invention

[0017] According to the speech output device of the present invention, an appropriate speech to the user can be output by using the user's action information.

Brief Description of the Drawings

[0018] [Figure 1] Conceptual diagram of the information system A including the location information production device 1 in Embodiment 1 [Figure 2] Block diagram of the location information production device 1 [Figure 3] Flowchart for explaining an operation example of the location information production device 1 [Figure 4] Flowchart for explaining an example of the time-series intensity acquisition process [Figure 5] Flowchart for explaining an example of the fixed information acquisition process [Figure 6] Flowchart for explaining an example of the same type determination process [Figure 7] Diagram showing the time-series radio wave intensity management table [Figure 8] Diagram showing the location information management table [Figure 9] Conceptual diagram of the information system C in Embodiment 2 [Figure 10] Block diagram of the terminal device 2 [Figure 11] Flowchart for explaining a first operation example of the terminal device 2 [Figure 12] Flowchart for explaining an example of the movement determination process [Figure 13] Flowchart for explaining an example of the position estimation process [Figure 14]A flowchart illustrating a second example of operation for terminal device 2. [Figure 15] A flowchart illustrating a second example of same-type determination processing. [Figure 16] Conceptual diagram of information system D in Embodiment 3 [Figure 17] Block diagram of the same information system D [Figure 18] Block diagram of the same behavior acquisition device 3 [Figure 19] Flowchart illustrating an example of operation of the activity acquisition device 3. [Figure 20] A flowchart illustrating an example of the process for obtaining the source of the same action. [Figure 21] A flowchart illustrating an example of the same-location estimation process. [Figure 22] A flowchart illustrating an example of the first action estimation process. [Figure 23] A flowchart illustrating an example of the second action estimation process. [Figure 24] A flowchart illustrating an example of the third action estimation process. [Figure 25] A flowchart illustrating an example of the first emotion estimation process. [Figure 26] A flowchart illustrating the second example of emotion estimation processing. [Figure 27] A flowchart illustrating the third example of emotion estimation processing. [Figure 28] A flowchart illustrating an example of the output configuration process. [Figure 29] A flowchart illustrating an example of the first behavioral learning process. [Figure 30] A flowchart illustrating the second example of behavioral learning processing. [Figure 31] A flowchart illustrating the first example of emotion learning processing. [Figure 32] A flowchart illustrating the second example of emotion learning processing. [Figure 33] Diagram showing the action source management chart. [Figure 34] Diagram showing the same behavioral and emotional management chart. [Figure 35] Diagram showing information management table for the same time period. [Figure 36] Figure showing an example of the same output. [Figure 37] Conceptual diagram of information system E in Embodiment 4 [Figure 38] Block diagram of the same information system E [Figure 39] Block diagram of the same behavior acquisition device 5 [Figure 40] Flowchart illustrating an example of operation of the activity acquisition device 5. [Figure 41] Flowchart illustrating an example of operation of terminal device 6. [Figure 42] Conceptual diagram of information system F in Embodiment 5 [Figure 43] Block diagram of the same information system F [Figure 44] Block diagram of the behavioral analysis device 7 [Figure 45] A flowchart illustrating an example of the operation of the behavioral analysis device 7. [Figure 46] A flowchart illustrating the first example of the score acquisition process. [Figure 47] A flowchart illustrating a second example of the score acquisition process. [Figure 48] A flowchart illustrating a third example of the score acquisition process. [Figure 49] A flowchart illustrating an example of the process for obtaining the same factors. [Figure 50] A flowchart illustrating an example of the recommendation acquisition process. [Figure 51] A flowchart illustrating an example of the long-term score acquisition process. [Figure 52] A flowchart illustrating an example of the learning process. [Figure 53] A diagram showing an example of a time-slot information management table. [Figure 54] A diagram showing an example of the recommendation management table. [Figure 55] Figure showing an example of the same output. [Figure 56] Figure showing an example of the same output. [Figure 57] Block diagram of the behavioral analysis device 9 [Figure 58] Conceptual diagram of information system G in Embodiment 6 [Figure 59] Block diagram of the same information system G [Figure 60] Block diagram of the behavioral analysis device 10 [Figure 61] Flowchart illustrating an example of operation of the behavioral analysis device 10. [Figure 62] A flowchart illustrating an example of the process for obtaining scores for each action. [Figure 63] A flowchart illustrating the first example of the score acquisition process. [Figure 64] A flowchart illustrating a second example of the score acquisition process. [Figure 65] A flowchart illustrating the first example of the cumulative score acquisition process. [Figure 66] A flowchart illustrating a second example of the cumulative score acquisition process. [Figure 67] A flowchart illustrating the first example of the cumulative score correction process. [Figure 68] A flowchart illustrating a second example of the cumulative score correction process. [Figure 69] A flowchart illustrating the first example of the model update process. [Figure 70] A flowchart illustrating a second example of the model update process. [Figure 71] This figure shows an example of the operation model of the behavioral analysis device 10. [Figure 72] Figure showing an example of the same output. [Figure 73] Conceptual diagram of information system H in Embodiment 7 [Figure 74] Block diagram of the same information system H [Figure 75] Block diagram of the speech output device H1. [Figure 76] Flowchart illustrating an example of operation of the speech output device H1. [Figure 77] A flowchart illustrating an example of the score acquisition process. [Figure 78] Flowchart explaining the process of acquiring simultaneous utterances [Figure 79] A flowchart illustrating an example of the input information acquisition process. [Figure 80] A flowchart illustrating an example of the feedback acquisition process. [Figure 81] A flowchart illustrating an example of operation for terminal device H2. [Figure 82] A diagram showing an example of a time-slot information management table. [Figure 83] This diagram shows an example of a template for a simultaneous utterance acquisition prompt. [Figure 84] A diagram showing an example of a simultaneous utterance acquisition prompt. [Figure 85] A diagram showing an example of simultaneous utterance information. [Figure 86] A diagram showing an example of simultaneous utterance information. [Figure 87] This diagram shows an example of a template for a simultaneous utterance acquisition prompt. [Figure 88] A diagram showing an example of a simultaneous utterance acquisition prompt. [Figure 89] Block diagram of the computer system in the above embodiment [Modes for carrying out the invention]

[0019] The following describes embodiments of the speech output device and the like with reference to the drawings. Note that components denoted by the same reference numerals in these embodiments perform similar operations, and therefore, further explanation may be omitted.

[0020] (Embodiment 1) In this embodiment, a location information production device will be described. A location information production device is a device that acquires location information, which will be described later, for a specific location.

[0021] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is irrelevant. Information X and information Y may be linked, may reside in the same buffer, may information X be contained in information Y, or information Y may be contained in information X, and so on.

[0022] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag on information Z, etc., and it is sufficient to be able to access information Z.

[0023] Figure 1 is a conceptual diagram of information system A, which includes a location information production device 1 in this embodiment. Information system A comprises the location information production device 1 and three or more communication devices B.

[0024] Each of the three or more communication devices B is a device that transmits radio waves to other devices such as the location information production device 1. Communication device B transmits a device identifier that identifies the communication device B to the other devices. Communication device B may be, for example, a Wi-Fi router or a communication device using BLE (Bluetooth Low Energy), but is not limited to these.

[0025] Figure 2 is a block diagram of the location information production device 1 in this embodiment. The location information production device 1 comprises a storage unit 11, a receiving unit 12, a receiving unit 13, and a processing unit 14. The receiving unit 12 comprises a location receiving unit 121. The processing unit 14 comprises an intensity acquisition unit 141, a type determination unit 142, and a storage unit 143.

[0026] The reception unit 12 receives various instructions and information. These instructions and information include, for example, location information, which will be described later. Any means of inputting these instructions and information is acceptable, such as a touch panel, keyboard, mouse, or menu screen.

[0027] The location receiving unit 121 receives location information for specific points. The location receiving unit 121 typically receives location information for three or more specific points. The location receiving unit 121 receives location information, for example, as user input. The location receiving unit 121 reads location information, for example, from the storage unit 11.

[0028] A specific location refers to a specific point indoors, but it can also be a specific point outdoors. Location information here refers to information that identifies a location indoors or outdoors. Location information is, for example, a three-dimensional coordinate value (x, y, z) indicating a relative position indoors or outdoors, but it can also be a two-dimensional coordinate value (x, y). The origin of the coordinate values ​​used to identify the relative position indoors or outdoors is not specified. An outdoor specific location is preferably a place where GPS signals are difficult to receive, such as among tall buildings or in a forest, but this is not required. Location information can also be information that a person can recognize as a place (for example, a string of characters) or an ID. Such location information can also be a label such as "living room," "workroom," "conference room," "east side of the library," or "toy department of a department store."

[0029] The location reception unit 121 may generate a unique ID. Such a unique ID can be considered a label or location information.

[0030] The location receiving unit 121 does not need to receive location information. In such cases, the location receiving unit 121 is unnecessary.

[0031] The receiving unit 13 receives radio waves containing device identifiers from three or more communication devices B at a specific location. The receiving unit 13 typically receives radio waves containing device identifiers from three or more communication devices B in succession.

[0032] The device identifier is information that identifies the communication device B. For example, the device identifier could be the ID or name of communication device B. It can be considered that receiving radio waves is equivalent to receiving information.

[0033] The processing unit 14 performs various processes. These processes include, for example, those performed by the strength acquisition unit 141, the type determination unit 142, and the storage unit 143.

[0034] The intensity acquisition unit 141 acquires the intensity of the radio waves received from each of the three or more communication devices B. The intensity acquisition unit 141 acquires the radio wave intensity in conjunction with the device identifier of the communication device B. The intensity acquisition unit 141 acquires the time-series radio wave intensity. The time-series radio wave intensity refers to two or more radio wave intensities that are consecutive in time. It goes without saying that there may be time intervals between them.

[0035] The type determination unit 142 uses the time-series radio wave intensity acquired by the intensity acquisition unit 141 to determine whether each of the three or more communication devices B is a fixed terminal or a mobile terminal. A fixed terminal is a communication device whose installation location is fixed. A mobile terminal is a communication device whose installation location is not fixed and which is moving.

[0036] The type determination unit 142, for example, obtains the degree of variation of two or more temporally consecutive radio wave intensity data in a time series paired with a single device identifier, and determines that the communication device B identified by that single device identifier is a mobile terminal if the degree of variation is equal to or greater than a threshold. The type determination unit 142 also obtains the degree of variation of two or more temporally consecutive radio wave intensity data in a time series paired with a single device identifier, and determines that the communication device B identified by that single device identifier is a fixed terminal if the degree of variation is equal to or less than a threshold.

[0037] The degree of variability refers to information indicating the degree of variation or change in radio wave intensity over time. The degree of variability can be, for example, a number based on variance, standard deviation, or difference (e.g., difference, the sum of the differences between two consecutive radio wave intensities within a time-series sequence of 3 or more consecutive radio wave intensities).

[0038] The type determination unit 142 determines, for example, that the number of radio wave intensity levels acquired in a predetermined time period is less than or equal to a threshold value, if the number of temporally consecutive radio wave intensity levels in a time series paired with a device identifier is less than or equal to a threshold value, then the communication device B identified by that device identifier is determined to be a mobile terminal.

[0039] The storage unit 143 configures and stores location information, which includes the device identifier and radio wave strength of the communication device B that the type determination unit 142 has determined to be a fixed terminal. It is preferable for the storage unit 143 to configure and store location information, which includes the device identifier and radio wave strength, for each of the three or more communication devices B.

[0040] The storage unit 143, for example, configures and stores location information, which includes the device identifier, radio wave strength, and location information of a specific point for each communication device B that the type determination unit 142 has determined to be a fixed terminal. It is preferable for the storage unit 143 to configure and store location information, which includes the device identifier, radio wave strength, and location information of a specific point, for each of the three or more communication devices B. The storage unit 143, for example, stores the location information in the storage unit 11, but it may also be stored in other devices. It is preferable for the location information to include location information, but it does not have to include location information. The location information may consist only of the device identifier and radio wave strength.

[0041] The radio wave intensity stored by the storage unit 143 is typically a representative value of the time-series radio wave intensity from communication device B. The representative value is, for example, the median, mean, maximum, or minimum value.

[0042] The storage unit 11 is preferably made of a non-volatile recording medium, but it can also be made of a volatile recording medium.

[0043] The process by which information is stored in the storage unit 11 is irrelevant. For example, information may be stored in the storage unit 11 via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11, or information input via an input device may be stored in the storage unit 11.

[0044] The reception unit 12 and the position reception unit 121 can be implemented using device drivers for input means such as touch panels and keyboards, or control software for menu screens.

[0045] The receiving unit 13 is typically implemented using wireless or wired communication means.

[0046] The processing unit 14, intensity acquisition unit 141, type determination unit 142, and storage unit 143 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 14, etc., are usually implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0047] Next, an example of the operation of the location information production device 1 will be explained using the flowchart in Figure 3.

[0048] (Step S301) The location receiving unit 121 determines whether or not it has received location information for a specific point. If location information has been received, the unit proceeds to step S302; otherwise, it returns to step S301.

[0049] (Step S302) The storage unit 143 acquires the location information received in step S301.

[0050] (Step S303) The processing unit 14, etc., performs time-series intensity acquisition processing. Time-series intensity acquisition processing is the process of acquiring the time-series radio wave intensity of radio waves from each of the three or more communication devices B. An example of time-series intensity acquisition processing will be explained using the flowchart in Figure 4.

[0051] (Step S304) The processing unit 14, etc., performs fixed information acquisition processing. The process returns to step S301. Fixed information acquisition processing is the process of acquiring the strength of radio waves from a fixed terminal at the point specified by the location information received in step S301. An example of fixed information acquisition processing will be explained using the flowchart in Figure 5.

[0052] In the flowchart of Figure 3, it is preferable that the user holding the location information production device 1 moves to three or more specific locations, and that the location information production device 1 receives location information for each of the three or more specific locations and repeatedly performs the processes from S301 to S304.

[0053] Furthermore, in the flowchart of Figure 3, processing is terminated by power off or processing termination interrupts.

[0054] Next, an example of the time-series intensity acquisition process in step S303 will be explained using the flowchart in Figure 4.

[0055] (Step S401) The receiving unit 13 determines whether or not it has received radio waves from any of the communication devices B. If it has received radio waves, it proceeds to step S402; otherwise, it returns to step S401.

[0056] (Step S402) The storage unit 143 obtains the device identifier corresponding to the radio wave received in step S401.

[0057] (Step S403) The intensity acquisition unit 141 acquires the intensity of the radio waves received in step S401.

[0058] (Step S404) The storage unit 143 appends the radio wave intensity obtained in step S403 to a buffer (not shown), associating it with the device identifier obtained in step S402.

[0059] (Step S405) The storage unit 143 determines whether the location information storage conditions are met. If the storage conditions are met, it returns to the higher-level processing; otherwise, it returns to step S401. The storage conditions include, for example, that a threshold time has elapsed since the reception of location information for a specific location, or that a threshold number of radio wave intensities have been stored for each of the three or more device identifiers.

[0060] Next, an example of the fixed information acquisition process in step S304 will be explained using the flowchart in Figure 5.

[0061] (Step S501) The type determination unit 142 assigns 1 to counter i.

[0062] (Step S502) The type determination unit 142 determines whether the i-th device identifier exists in the buffer (not shown). If the i-th device identifier exists, the process proceeds to step S503; otherwise, the process returns to the higher-level processing.

[0063] (Step S503) The type determination unit 142 determines the type of communication device B identified by the i-th device identifier. An example of this type determination process will be explained using the flowchart in Figure 6.

[0064] (Step S504) If the result of the determination in step S503 is "fixed terminal", proceed to step S505; if it is a mobile terminal, proceed to step S508.

[0065] (Step S505) The storage unit 143 obtains two or more radio wave intensities that are paired with the i-th device identifier from a buffer (not shown).

[0066] (Step S506) The storage unit 143 acquires two or more representative values ​​of radio wave intensity.

[0067] (Step S507) The storage unit 143 stores in the storage unit 11 a pair of the i-th device identifier and the representative value of the radio wave intensity obtained in step S506, associated with the location information received in step S301.

[0068] (Step S508) The type determination unit 142 increments the counter i by 1. The process returns to step S502.

[0069] In the flowchart of Figure 5, the storage unit 143 may, in step S506, acquire the latest radio wave intensity instead of two or more representative values ​​of radio wave intensity.

[0070] Next, an example of the type determination process in step S503 will be explained using the flowchart in Figure 6.

[0071] (Step S601) The type determination unit 142 obtains two or more radio wave intensity values ​​from a buffer (not shown) that are paired with the i-th device identifier in step S502.

[0072] (Step S602) The type determination unit 142 obtains the variation in two or more radio wave intensity values ​​obtained in step S601.

[0073] (Step S603) The type determination unit 142 determines whether the degree of variation obtained in step S602 is below or equal to the threshold. If the degree of variation is below or equal to the threshold, the process proceeds to step S604; if it is above or equal to the threshold, the process proceeds to step S605.

[0074] (Step S604) The type determination unit 142 determines the type of communication device B as "fixed terminal". It returns to the higher-level processing.

[0075] (Step S605) The type determination unit 142 determines that the type of communication device B is "mobile terminal". It returns to the higher-level processing.

[0076] The following describes a specific example of the operation of the location information production device 1 in this embodiment. Here, the storage condition is assumed to be that a predetermined time (for example, 3 minutes) has elapsed since the location information of a specific location was received.

[0077] Let's assume that user A is in a certain indoor location (for example, user A's home or a department store that user A frequently visits). And let's assume that user A has input location information (x1, y1) to location information production device 1.

[0078] Next, the location receiving unit 121 of the location information production device 1 receives location information (x1, y1) of a specific location. Then, the storage unit 143 acquires the received location information (x1, y1) into a buffer (not shown).

[0079] The receiving unit 13 then receives radio waves containing device identifiers from three or more communication devices B for a predetermined period of time (for example, 3 minutes). The storage unit 143 then acquires the device identifiers contained in the received radio waves. The intensity acquisition unit 141 also acquires the intensity of the received radio waves. Next, the storage unit 143 appends the acquired radio wave intensity to a buffer (not shown) in association with the acquired device identifiers. As a result, the buffer (not shown) is configured with the time-series radio wave intensity management table shown in Figure 7. The time-series radio wave intensity management table shown in Figure 7 is a table for a specific location indicated by the location information (x1, y1).

[0080] The time-series radio wave strength management table is a table that manages the time-series radio wave strength for each communication device B. The time-series radio wave strength management table is a table that manages two or more records that have "ID", "device identifier", and "time-series radio wave strength". "ID" is information that identifies the record. "Time-series radio wave strength" is the radio wave strength that is continuous over time. 11 "R 12 "R 21 " indicates radio wave strength.

[0081] After the management table in Figure 7 is created, the location information (x1, y1) of a specific point is received by the location reception unit 121, and a predetermined time (for example, 3 minutes) has elapsed, so the storage unit 143 determines that the conditions for storing location information have been met.

[0082] Next, the type determination unit 142 obtains the degree of variation in the time-series radio wave intensity of each record in Figure 7, following the operation of the flowchart in Figure 6, and determines whether each communication device B is a fixed terminal or a mobile terminal. The type determination unit 142 then determines that the communication devices B identified by the device identifiers "device 1, device 3, device 4, device 6, ..." are fixed terminals, and that the communication devices B identified by the device identifiers "device 2, device 5, ..." are mobile terminals.

[0083] Next, the storage unit 143 constructs radio wave strength information by pairing the device identifier of the fixed terminal communication device B with a representative value of the radio wave strength. Then, the storage unit 143 stores each of the multiple radio wave strength records in association with location information (x1, y1). Through this process, the record "ID=1" in the location information management table in Figure 8 is constructed. Note that the storage unit 143 may store only the device identifier and radio wave strength. In this case, each record in Figure 8 does not contain location information. Also, in this case, the location reception unit 121 does not need to receive location information for a specific location.

[0084] A location information management table is a table used to manage location information. The location information management table stores multiple records that correspond to location information and each record has an "ID," "device identifier," and "radio wave strength information." The "radio wave strength information" includes both the "device identifier" and the "radio wave strength."

[0085] Through the above process, location information for specific point 1, indicated by the location information (x1, y1), was accumulated.

[0086] User A moves to specific point 2, the location indicated by the location information (x2, y2), with the location information production device 1, and performs the same procedure as above. As a result, the location information production device 1 creates and stores a record for "ID=2" in the location information management table shown in Figure 8. Furthermore, User A moves to one or more specific points, including specific point 3, with the location information production device 1, and performs the same procedure as above. As a result, the location information production device 1 creates and stores a record for "ID=3" and beyond (not shown) in the location information management table shown in Figure 8.

[0087] As described above, according to this embodiment, location information for determining the position of a terminal device indoors can be obtained. In other words, according to this embodiment, three or more locations for determining the position of a terminal device indoors can be produced.

[0088] The processing in this embodiment may be implemented by software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the location information production device 1 in this embodiment is the following program. In other words, this program is a program that causes a computer to function as a location receiving unit that receives location information of a specific location, a receiving unit that receives radio waves from three or more communication devices at the specific location, each containing a device identifier that identifies the communication device, an intensity acquisition unit that acquires the time-series radio wave intensity for each of the three or more communication devices, a type determination unit that uses the time-series radio wave intensity acquired by the intensity acquisition unit to determine whether each of the three or more communication devices is a fixed terminal, which is a stationary communication device, or a mobile terminal, which is a mobile communication device, and an storage unit that constructs and stores location information having the device identifier and radio wave intensity of each of the three or more communication devices that the type determination unit has determined to be fixed terminals, and the location information of the specific location.

[0089] (Embodiment 2) In this embodiment, we will describe a terminal device that receives radio waves from three or more communication devices B, determines the type of each communication device B using the radio wave intensity in a time series, and obtains and outputs a terminal position indicating the location of the terminal device indoors using only the radio wave intensity from communication devices B of type "fixed terminal".

[0090] Furthermore, in this embodiment, we will describe a terminal device that determines whether the terminal device is moving or stationary, and uses the determination result to acquire and output the terminal position.

[0091] Figure 9 is a conceptual diagram of information system C in this embodiment. Information system C comprises one or more terminal devices 2 and three or more communication devices B.

[0092] Terminal device 2 is a device capable of acquiring location information indoors. Terminal device 2 can be, for example, a smartphone, tablet device, smartwatch, or so-called personal computer, and its type is not limited.

[0093] Figure 10 is a block diagram of the terminal device 2 in this embodiment. The terminal device 2 comprises a storage unit 21, a receiving unit 22, a processing unit 23, and an output unit 24. The storage unit 21 comprises a location information storage unit 211. The processing unit 23 comprises an intensity acquisition unit 231, a type determination unit 232, a movement determination unit 233, and a position acquisition unit 234. The position acquisition unit 234 comprises an intensity acquisition means 2341, a location determination means 2342, and a position acquisition means 2343. The output unit 24 comprises a position output unit 241.

[0094] The storage unit 21 stores various types of information. These types of information include, for example, location information, which will be described later.

[0095] The location information storage unit 211 stores three or more pieces of location information. Preferably, the three or more pieces of location information in the location information storage unit 211 are information accumulated by the location information production device 1.

[0096] The location information storage unit 211 stores three or more location information entries, for example, each location information entry, device identifier, and radio wave intensity information. It is preferable that three or more radio wave intensity information entries are associated with each of the three or more location information entries. The radio wave intensity information entries include a device identifier and radio wave intensity. The three or more radio wave intensity information entries may constitute a radio wave intensity vector. The radio wave intensity vector is a vector using radio wave intensity information and has a structure such as (radio wave intensity of device identifier 1, radio wave intensity of device identifier 2, radio wave intensity of device identifier 3, ... radio wave intensity of device identifier n). The location information storage unit 211 stores, for example, a location information management table having the structure shown in Figure 8.

[0097] Note that terminal device 2 does not necessarily have a location information storage unit 211. In such a case, terminal device 2 refers to the location information storage unit 211 of an external device (not shown) and obtains the terminal position as described later.

[0098] The receiving unit 22 receives radio waves from each of the three or more communication devices B, which include a device identifier that identifies the communication device B. The receiving unit 22 usually performs the same function as the receiving unit 13 described above. The receiving unit 22 usually receives radio waves continuously from each of the three or more communication devices B.

[0099] The processing unit 23 performs various processes. These processes include, for example, those performed by the intensity acquisition unit 231, the type determination unit 232, the movement determination unit 233, and the position acquisition unit 234.

[0100] The signal strength acquisition unit 231 acquires the time-series signal strength for each of the three or more communication devices B. The signal strength acquisition unit 231 acquires the signal strength based on the radio waves received by the receiving unit 22. The signal strength acquisition unit 231 performs the same function as the signal strength acquisition unit 141 described above.

[0101] The type determination unit 232 uses the time-series radio wave intensity acquired by the intensity acquisition unit 231 to determine whether each of the three or more communication devices B is a fixed terminal or a mobile terminal. The type determination unit 232 performs the same function as the type determination unit 142 described above.

[0102] The movement determination unit 233 determines whether the terminal device 2 is moving or stopped, and obtains a movement determination result, which is the result of that determination. The movement determination result is, for example, "moving" or "stopped".

[0103] The movement determination unit 233 acquires, for example, sensor information from the terminal device 2 and uses the sensor information to obtain a movement determination result. The sensor information includes, for example, acceleration data from a gyroscope and time-series position information.

[0104] The movement determination unit 233 obtains a movement determination result of "Stopped" if, for example, the acceleration due to the gyroscope is "0" or less than or equal to a threshold. The movement determination unit 233 obtains a movement determination result of "Moving" if, for example, the acceleration due to the gyroscope is equal to or greater than a threshold.

[0105] The movement determination unit 233, for example, uses the time-series radio wave intensity data of each of the three or more communication devices B acquired by the intensity acquisition unit 231 to determine that the device is stopped if there is no change in the time-series radio wave intensity data of one or more communication devices B, and obtains the movement determination result "Stopped".

[0106] The position acquisition unit 234 acquires the radio wave strength of three or more communication devices B that the type determination unit 232 has determined to be fixed terminals, and uses the radio wave strength of the fixed terminals to acquire the terminal position, which is the location information of the terminal device 2 indoors.

[0107] The location acquisition unit 234 acquires, for example, the radio wave strength of three or more communication devices B that the type determination unit 232 has determined to be fixed terminals, and using these three or more radio wave strengths, it refers to three or more location information in the location information storage unit 211 and acquires the terminal location using the fingerprint method.

[0108] The position acquisition unit 234 preferably acquires position information using the movement determination result. For example, the position acquisition unit 234 preferably acquires the terminal position only when the movement determination result is "stopped".

[0109] The position acquisition unit 234 may acquire the terminal position using location information containing the device identifier, but only if the device identifier corresponding to the radio waves received by the receiving unit 22 is included in the location information of the location information storage unit 211. This is because the location information in the location information storage unit 211 is the location information of a fixed terminal.

[0110] The strength acquisition means 2341 acquires the radio wave strength of three or more communication devices that the type determination unit 232 has determined to be fixed terminals, associating them with device identifiers.

[0111] The location determination means 2342 determines one or more location information items that satisfy similar conditions to the radio wave intensity associated with each of the three or more device identifiers acquired by the intensity acquisition means 2341, from the location information stored in the location information storage unit 211.

[0112] The location determination means 2342 obtains, for example, a first radio wave intensity vector, which is a vector whose elements are the radio wave intensity paired with each of the three or more device identifiers in the location information. The location determination means 2342 also obtains, for example, a second radio wave intensity vector, which is a vector whose elements are the radio wave intensity associated with each of the three or more device identifiers obtained by the intensity acquisition means 2341. The location determination means 2342 obtains, for example, the similarity between the two radio wave intensity vectors, and if the similarity is equal to or greater than a threshold, it obtains the location information corresponding to the first radio wave intensity vector.

[0113] The position acquisition means 2343 acquires position information from one or more point pieces determined by the point determination means 2342, and uses one or more position pieces to acquire the terminal position.

[0114] The output unit 24 outputs various types of information. These types of information include, for example, the terminal location and an indoor map.

[0115] Here, "output" is a concept that includes display on a screen, projection using a projector, printing with a printer, sound output, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0116] The location output unit 241 outputs the terminal location acquired by the location acquisition unit 234. The location output unit 241 displays, for example, a graphic on an indoor map that clearly indicates the location identified by the terminal.

[0117] The storage unit 21 and the location information storage unit 211 are preferably made of a non-volatile recording medium, but can also be made of a volatile recording medium.

[0118] The process by which information is stored in the storage unit 21, etc. is not relevant. For example, information may be stored in the storage unit 21, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 21, etc., or information input via an input device may be stored in the storage unit 21, etc.

[0119] The receiving unit 22 is typically implemented using wireless or wired communication means.

[0120] The processing unit 23, intensity acquisition unit 231, type determination unit 232, movement determination unit 233, position acquisition unit 234, intensity acquisition means 2341, point determination means 2342, and position acquisition means 2343 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 23, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0121] The output unit 24 and the position output unit 241 can be implemented, for example, by driver software for an output device such as a display or speaker, or by driver software for an output device and an output device.

[0122] Next, we will explain the first example of operation of terminal device 2 using the flowchart in Figure 11.

[0123] (Step S1101) The movement determination unit 233 determines whether the terminal device 2 is moving or stopped. An example of this movement determination process will be explained using the flowchart in Figure 12.

[0124] (Step S1102) If the result of the judgment in step S1101 is "stopped", proceed to step S1103; otherwise, return to step S1101.

[0125] (Step S1103) The intensity acquisition means 2341 performs time-series intensity acquisition processing. An example of time-series intensity acquisition processing is explained using the flowchart in Figure 4.

[0126] (Step S1104) The type determination unit 232, the location determination means 2342, etc. perform fixed information acquisition processing. An example of the fixed information acquisition processing is explained using the flowchart in Figure 5.

[0127] (Step S1105) The position acquisition means 2343 performs a position estimation process to acquire the terminal position. An example of the position estimation process will be explained using the flowchart in Figure 13.

[0128] (Step S1106) The position output unit 241 outputs the terminal position acquired in step S1105. Return to step S1101.

[0129] In the flowchart shown in Figure 11, processing is terminated by power-off or processing termination interrupts.

[0130] Next, an example of the movement decision process in step S1101 will be explained using the flowchart in Figure 12.

[0131] (Step S1201) The movement determination unit 233 acquires sensor values ​​(e.g., acceleration) from the terminal device 2 and temporarily stores them in a buffer (not shown).

[0132] (Step S1202) The movement determination unit 233 uses the sensor values ​​in a buffer (not shown) to determine whether or not to make a movement determination. If a movement determination is made, the unit proceeds to step S1203; otherwise, it returns to step S1201. The movement determination unit 233 may always make a movement determination, or it may make a movement determination after accumulating a predetermined number of sensor values ​​in the buffer, or after a predetermined time has elapsed since the acquisition of the sensor values.

[0133] (Step S1203) The movement determination unit 233 uses one or more sensor values ​​from a buffer (not shown) to determine whether the terminal device 2 is moving or stopped. If it is stopped, proceed to step S1204; if it is moving, proceed to step S1205.

[0134] (Step S1204) The movement determination unit 233 sets the movement determination result to "Stopped". Proceed to step S1206.

[0135] (Step S1205) The movement determination unit 233 sets the movement determination result to "moving".

[0136] (Step S1206) The movement determination unit 233 clears a buffer (not shown). It returns to the higher-level processing unit.

[0137] Next, an example of the position estimation process in step S1105 will be explained using the flowchart in Figure 13.

[0138] (Step S1301) The position acquisition means 2343 acquires three or more radio wave strength information (a pair of device identifier and radio wave strength) of the terminal device 2.

[0139] (Step S1302) The position acquisition means 2343 vectorizes three or more radio wave intensity information and obtains a radio wave intensity vector. The radio wave intensity vector is, for example, (radio wave intensity of device identifier 1, radio wave intensity of device identifier 2, radio wave intensity of device identifier 3, ... radio wave intensity of device identifier n).

[0140] (Step S1303) The position acquisition means 2343 assigns 1 to counter i.

[0141] (Step S1304) The position acquisition means 2343 determines whether or not the i-th location information exists in the location information storage unit 211. If the i-th location information exists, the process proceeds to step S1305; otherwise, the process proceeds to step S1309.

[0142] (Step S1305) The position acquisition means 2343 acquires the i-th radio wave intensity vector of the i-th point information from the point information storage unit 211.

[0143] (Step S1306) The position acquisition means 2343 obtains the similarity between the radio wave intensity vector obtained in step S1302 and the i-th radio wave intensity vector obtained in step S1305. Next, the position acquisition means 2343 determines whether the similarity satisfies the similarity condition (for example, the similarity is greater than or equal to a threshold). If the similarity satisfies the similarity condition, the process proceeds to step S1307; otherwise, the process proceeds to step S1308.

[0144] (Step S1307) The position acquisition means 2343 acquires the position information and similarity of the i-th point information and stores it in a buffer (not shown).

[0145] (Step S1308) The position acquisition means 2343 increments the counter i by 1. Return to step S1304.

[0146] (Step S1309) The position acquisition means 2343 acquires a terminal position that identifies the position of the terminal device 2 using a set of three or more position information and similarity values ​​stored in a buffer (not shown). The process returns to the higher-level processing.

[0147] Next, a second example of operation of terminal device 2 will be explained using the flowchart in Figure 14.

[0148] (Step S1401) The intensity acquisition means 2341 performs time-series intensity acquisition processing. An example of time-series intensity acquisition processing is explained using the flowchart in Figure 4.

[0149] (Step S1402) The location determination means 2342 performs fixed information acquisition processing. An example of the fixed information acquisition processing is explained using the flowchart in Figure 5.

[0150] (Step S1403) The position acquisition means 2343 performs a position estimation process to acquire the terminal position. An example of the position estimation process is explained using the flowchart in Figure 13.

[0151] (Step S1404) The position output unit 241 outputs the terminal position acquired in step S1403. Return to step S1401.

[0152] In the flowchart in Figure 14, processing is terminated by power-off or processing termination interrupts.

[0153] Next, a second example of the type determination process in the fixed information acquisition process at step S1402 in the flowchart of Figure 14 will be explained using the flowchart of Figure 15. The first example of the type determination process was explained using the flowchart of Figure 6.

[0154] (Step S1501) The type determination unit 232 obtains the device identifier of the communication device B to be determined as a type.

[0155] (Step S1502) The type determination unit 232 determines whether the device identifier obtained in step S1501 exists in any of the location information in the location information storage unit 211. If it exists in any of the location information, the unit proceeds to step S1503; otherwise, the unit proceeds to step S1504.

[0156] (Step S1503) The type determination unit 232 determines the type as "fixed terminal". It returns to the higher-level processing.

[0157] (Step S1504) The type determination unit 232 determines the type to be "mobile terminal". It returns to the higher-level processing.

[0158] The following describes a specific example of the operation of terminal device 2 in this embodiment.

[0159] Assume that user B is holding their terminal device 2 and has entered an indoor location identified by a location identifier (P). Assume that the receiving unit 22 of terminal device 2 transmits a location information request containing the location identifier (P) to an external device (not shown), and receives the location information management table shown in Figure 8 from that device. Assume that the processing unit 23 has temporarily stored the location information management table in the location information storage unit 211.

[0160] Then, terminal device 2 operates as follows, according to the processes from steps S1103 to S1106 in Figure 11, or the processes in the flowchart in Figure 14.

[0161] In other words, the intensity acquisition means 2341 performs a time-series intensity acquisition process as explained using the flowchart in Figure 4, acquires the radio wave intensity of each of the three or more communication devices B at location X where the terminal device 2 is located, and constructs a time-series radio wave intensity management table having the structure shown in Figure 7.

[0162] Next, the intensity acquisition means 2341 performs time-series intensity acquisition processing to acquire the time-series radio wave intensity of each communication device B that can receive radio waves at point X, and constructs a time-series radio wave intensity management table with the structure shown in Figure 7.

[0163] Next, the type determination unit 142 refers to the time-series radio wave intensity management table and determines whether each communication device B is a "fixed terminal" or a "mobile terminal" based on the type determination process described using the flowchart in Figure 4.

[0164] Next, the location determination means 2342 acquires the radio wave strength of "device 1", "device 3", "device 4", "device 6", etc., which are determined to be fixed terminals. The location determination means 2342 then acquires a radio wave strength vector "(radio wave strength of device 1, radio wave strength of device 3, radio wave strength of device 4, radio wave strength of device 6, ...) = (P1, P3, P4, P6, ...)". Note that the radio wave strength of communication device B acquired by the location determination means 2342 here may be a representative value of two or more radio wave strengths, or it may be a single radio wave strength such as the latest radio wave strength of communication device B.

[0165] Next, the position acquisition means 2343 performs the position estimation process described using the flowchart of FIG. 13, and calculates the similarity between the radio wave intensity vector at point X and the radio wave intensity vectors of each record in FIG. 8 (vectors constituted by radio wave intensity information). Next, the position acquisition means 2343 determines radio wave intensity vectors that satisfy the similarity condition "similarity >= threshold value" (for example, the radio wave intensity vector of "ID=1" in FIG. 8 (S 11 , S 12 , S 13 , ···), the radio wave intensity vector of "ID=2" (S 21 , S 22 , S 23 , ···), ···). Next, the position acquisition means 2343 acquires pairs of position information and similarity that correspond to the radio wave intensity vectors that satisfy the similarity condition (for example, "(x1, y1), DS1", "(x2, y2), DS2", ···). Next, the position acquisition means 2343 acquires the position in the indoor area of point X (x1×DS1 / sum of similarities + x2×DS2 / sum of similarities + ···, y1×DS1 / sum of similarities + y2×DS2 / sum of similarities + ···). Note that the sum of similarities is "DS1 + DS2 + ···".

[0166] Next, the position output unit 241 outputs the indoor or outdoor map stored in the storage unit 21, and arranges a symbol at the position indicated by the position information (terminal position) acquired by the position acquisition means 2343 on the map.

[0167] As described above, according to the present embodiment, the position of the terminal device 2 indoors or outdoors can be easily acquired.

[0168] The processing in this embodiment may be implemented in software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the terminal device 2 in this embodiment is the following program. In other words, this program is a program that causes a computer to function as a receiving unit that receives radio waves from three or more communication devices, each containing a device identifier that identifies the communication device; an intensity acquisition unit that acquires the time-series radio wave intensity for each of the three or more communication devices; a type determination unit that uses the time-series radio wave intensity acquired by the intensity acquisition unit to determine whether each of the three or more communication devices is a fixed terminal or a moving terminal; a location acquisition unit that uses the radio wave intensity of the three or more communication devices that the type determination unit has determined to be fixed terminals to acquire the terminal location, which is the location information of the terminal device; and a location output unit that outputs the terminal location acquired by the location acquisition unit.

[0169] (Embodiment 3) In this embodiment, an activity acquisition device is described that uses the user's location information corresponding to the time to acquire user activity information and time zones, and outputs the activity information corresponding to those time zones. It is preferable that the activity acquisition device also uses user activity data and vital data to acquire user activity information and time zones. Furthermore, the location information used when acquiring activity information may be indoor location information acquired by the terminal device 2 described in Embodiment 2.

[0170] Furthermore, in this embodiment, an action acquisition device that acquires action information using past recorded information will be described. Note that the past recorded information may be information based on input from one or more users.

[0171] Furthermore, in this embodiment, we will describe an action acquisition device that also acquires and outputs user emotion information. In addition, in this embodiment, it is preferable to acquire emotion information using past recorded information.

[0172] In this embodiment, the behavior acquisition device is a terminal. However, as will be explained in Embodiment 4, the behavior acquisition device may also be a server. In other words, the processes for acquiring behavioral information and emotional information, which will be described later, may be performed by the user's terminal or by the server.

[0173] Furthermore, in this embodiment, if behavioral information cannot be obtained, a behavioral information acquisition device that uses a map to acquire and output location information will be described.

[0174] Furthermore, in this embodiment, an action acquisition device that displays estimated action information and confirmed action information in a manner that visually distinguishes them will be described.

[0175] Figure 16 is a conceptual diagram of the information system D in this embodiment. The information system D comprises one or more action acquisition devices 3, a server device 4, and three or more communication devices B.

[0176] The behavior acquisition device 3 is a terminal. The behavior acquisition device 3 is a device that acquires and outputs behavioral information. The behavior acquisition device 3 can be, for example, a smartphone, tablet terminal, smartwatch, or so-called personal computer, and its type is not limited.

[0177] Server device 4 is, for example, a device that stores sets of source information and action information for two or more users, for each time period. Server device 4 also stores, for example, learning information (described later), and provides this learning information to the action acquisition device 3. Server device 4 can be, for example, a cloud server or an ASP server, but the type is not limited.

[0178] Figure 17 is a block diagram of the information system D in this embodiment. Figure 18 is a block diagram of the behavior acquisition device 3.

[0179] The behavior acquisition device 3 comprises a storage unit 31, a receiving unit 32, a processing unit 33, and an output unit 34. The storage unit 31 comprises a learning management unit 311, a map management unit 312, and a behavior management unit 313. The behavior management unit 313 may reside in an external device not shown. The processing unit 33 comprises an intensity acquisition unit 231, a type determination unit 232, a time acquisition unit 331, a position acquisition unit 332, an activity acquisition unit 333, a vital sign acquisition unit 334, a behavior estimation unit 335, an emotion estimation unit 336, a location acquisition unit 337, a storage unit 338, and a component unit 339. The output unit 34 comprises a behavior output unit 341 and an emotion output unit 342.

[0180] The server device 4 comprises a server storage unit 41, a server receiving unit 42, a server processing unit 43, and a server transmitting unit 44.

[0181] The behavior acquisition device 3 accepts, for example, output instructions, confirmation instructions, and information input. An output instruction is an instruction to output the output information described later. Output instructions usually include period information. Period information is information that specifies the period for which behavior information, etc., will be output. A confirmation instruction is an instruction to confirm the estimated behavior information or emotional information. Information input is the input of information to change the estimated behavior information or emotional information if it is incorrect. Information input is the updated behavior information or updated emotional information.

[0182] The storage unit 31, which constitutes the action acquisition device 3, stores various types of information. These types of information include, for example, learning information (described later), maps (described later), action information (described later), location information, a calendar template, action information corresponding to two or more action conditions, and emotion information corresponding to two or more emotion conditions.

[0183] The calendar template is information that indicates the template for the output calendar. The calendar template can be, for example, an ICS file, an HTML file, or an XML file, but its data structure is not specified.

[0184] Behavioral conditions are the conditions for acquiring behavioral information. Behavioral conditions are conditions that use two or more source information for each behavior. Behavioral conditions are associated with behavioral information. For example, a behavioral condition might be "Location information = Office AND 8:00 <= Time <= 19:00", and the behavioral information associated with this condition is "Work". For example, a behavioral condition might be "Location information = Kitchen AND Activity data = Standing AND 7:00 <= Time <= 8:00", and the behavioral information associated with this condition is "Cooking". For example, a behavioral condition might be "Location information = Park AND Activity data = Standing AND 120 <= Heart rate", and the behavioral information associated with this condition is "Running".

[0185] An emotional condition is a condition for acquiring emotional information. An emotional condition is a condition that uses two or more emotional source information. An emotional condition is associated with emotional information. For example, an emotional condition might be "Location information = Office AND 8:00 <= Time <= 19:00", and the emotional information associated with this behavioral condition is "Positive". For example, an emotional condition might be "Location information = Kitchen AND Activity data = Standing AND 7:00 <= Time <= 8:00", and the emotional information associated with this behavioral condition is "Positive". For example, a behavioral condition might be "Location information = Park AND Activity data = Standing AND 120 <= Heart rate", and the emotional information associated with this behavioral condition is "Negative".

[0186] The learning management unit 311 stores learning information. Learning information is information based on two or more training data. Examples of learning information in the learning management unit 311 include behavioral learning information and emotional learning information. Each of the two or more learning information in the learning management unit 311 may be associated with a different user attribute value condition. A user attribute value condition is a condition relating to one or more user attribute values.

[0187] User attribute values ​​are the attributes of a user. User attribute values ​​include, for example, occupation, family structure, marital status (single or married), gender, age, age group, morning or night owl status, and residential area, but are not restricted.

[0188] Behavioral learning information is information based on two or more behavioral training data. Examples of behavioral learning information include behavioral learning models or behavioral correspondence tables. Emotional learning information is information based on two or more emotional training data. Examples of emotional learning information include emotion learning models or emotion correspondence tables.

[0189] Behavioral training data includes, for example, one or more source information for behavior and behavioral information. Emotional training data includes, for example, one or more source information for behavior, behavioral information, and emotional information.

[0190] Emotion training data, for example, includes one or more source information or behavioral information, and emotion information. In emotion training data, one or more source information, behavioral information, or one or more source information and behavioral information are explanatory variables, and emotion information is the dependent variable.

[0191] Behavioral information refers to information that identifies a user's actions. Examples of behavioral information include "work," "watching TV," "walking," "running," "gym," "bathing," and "sleeping."

[0192] Emotional information is information about the user's emotions. Emotional information can be positive (e.g., "1") or negative (e.g., "0"). Emotional information can be the degree of positivity or negativity. Emotional information can be joy (e.g., "1"), anger (e.g., "2"), sadness (e.g., "3"), or happiness (e.g., "4").

[0193] Behavioral source information is the information used to obtain behavioral information. Behavioral source information includes location information. Preferably, behavioral source information includes activity data or one or more types of vital data. Behavioral source information may also include emotional information. Behavioral source information may include one or more past behavioral information. Past behavioral information usually includes the most recent behavioral information. Behavioral source information may include one or more future schedule information of the user. Schedule information is, for example, information stored in a calendar server (not shown) (e.g., "Google Calendar® server"). Behavioral source information may include the elapsed time since arriving at the same location. Behavioral source information may include one or more user attribute values.

[0194] Location information refers to information that identifies the location of the behavior acquisition device 3. Location information includes, for example, (latitude, longitude), (latitude, longitude, altitude), a three-dimensional relative position (x, y, z) indoors, or a two-dimensional relative position (x, y) indoors, or place information. Place information refers to information that expresses the meaning of a place. Place information includes, for example, indoor place information or outdoor place information. Indoor place information refers to information that identifies a place indoors. Examples of indoor place information include "living room," "kitchen," "workroom," and "office." Outdoor place information refers to information that identifies a place outdoors. Examples of outdoor place information include "ABC Station," "library," "izakaya (Japanese pub)," and "Point A."

[0195] Physical data refers to information about a user's body. Examples of physical data include activity data and vital data.

[0196] Activity data is information that identifies a user's activities. Examples of activity data include "standing" and "ground contact (e.g., sitting)."

[0197] Vital data refers to information obtained from a user's biological system. It can also be called biometric information. Examples of vital data include heart rate, heart rate variability, blood pressure (systolic and / or diastolic), respiratory rate, and body temperature per unit of time (e.g., 1 minute or 30 seconds). Learning information, on the other hand, is a learning model or a correspondence table. A learning model is information constructed through machine learning learning processes using two or more training data sets, and is used in machine learning prediction processes. A learning model can also be called a learner, classifier, or classification model. The machine learning algorithm can be deep learning, random forest, decision tree, SVM, etc. Furthermore, various machine learning functions and existing libraries can be used in machine learning, such as the TensorFlow® library, the R language's random forest module, and TinySVM. When learning information is a learning model, one or more source data points are explanatory variables, and the action data is the target variable.

[0198] The learning models referred to here are, for example, behavioral learning models or emotion learning models. A behavioral learning model is a learning model for acquiring behavioral information, and the information is obtained through machine learning training using behavioral training data. Behavioral training data consists of one or more source information pieces and behavioral information.

[0199] An emotion learning model is a learning model for acquiring emotion information, and the information is obtained through machine learning training using emotion training data. Emotion training data consists of one or more emotion source data and emotion data.

[0200] A correspondence table is either an action correspondence table or an emotion correspondence table. An action correspondence table is a table for obtaining action information. An action correspondence table has two or more action correspondence information entries. Action correspondence information is information that shows the correspondence between one or more action source information entries and action information. One or more action source information entries have, for example, a vector structure. Such a vector is called an action source vector. An action source vector is a vector that has one or more action source information entries as elements. An emotion correspondence table has two or more emotion correspondence information entries. Emotion correspondence information is information that shows the correspondence between one or more emotion source information entries and emotion information. One or more emotion source information entries have, for example, a vector structure. Such a vector is called an emotion source vector. An emotion source vector is a vector that has one or more emotion source information entries as elements.

[0201] The map management unit 312 stores the map. The map has location information corresponding to one or more location information points. The map is, for example, in KIWI format, but its structure is not limited.

[0202] The behavior management unit 313 stores behavior information corresponding to two or more time periods. This behavior information is, for example, information acquired by the behavior estimation unit 335. The behavior information is information acquired by the behavior estimation unit 335 that has been modified by the user.

[0203] It is preferable that the action information here be identifiable as to whether or not it is confirmed action information. For example, action information may be associated with a confirmed flag. A confirmed flag is a flag that indicates that the action information is confirmed. Being able to identify whether or not it is confirmed action information means, for example, that the storage areas for confirmed action information and unconfirmed action information are different. Any other method for making it possible to identify whether or not it is confirmed action information is not specified.

[0204] The receiving unit 32 receives radio waves from one or more communication devices B, each containing a device identifier that identifies the communication device B. Typically, the receiving unit 32 receives radio waves from three or more communication devices B. The receiving unit 32 has the same functions as the receiving unit 22.

[0205] The processing unit 33 performs various processes. These processes include, for example, those performed by the intensity acquisition unit 231, the type determination unit 232, and the time acquisition unit 331.

[0206] The time acquisition unit 331 acquires the time. The time acquisition unit 331 acquires the time from, for example, a clock (not shown). The time acquisition unit 331 receives the time from, for example, a server device 4 or a device (not shown). The time may be in the form of hours, minutes, and seconds, or just hours and minutes. The time may include one or more pieces of information from "year," "month," and "day." The time acquisition unit 331 may also acquire the day of the week. The day of the week can be considered as information included in the acquired time.

[0207] The position acquisition unit 332 acquires position information. This position information is associated with a time. Normally, the position acquisition unit 332 acquires position information associated with the time acquired by the time acquisition unit 331. The position acquisition unit 332 may perform the same processing as the position acquisition unit 234. In particular, when GPS signals cannot be received, such as when the activity acquisition device 3 is indoors, it is preferable for the position acquisition unit 332 to perform the same processing as the position acquisition unit 234. It is preferable for the position acquisition unit 332 to have a GPS receiver. For example, the position acquisition unit 332 acquires position information using a GPS receiver. Such position information is absolute position information. The position information acquired by the position acquisition unit 332 may be outdoor or indoor position information.

[0208] The location acquisition unit 332 preferably acquires indoor location information using the radio wave strength of the communication device B that the type determination unit 232 has determined to be a fixed terminal. The radio wave strength used here preferably comes from three or more communication devices B, but it may also come from one or two or more communication devices B.

[0209] The activity acquisition unit 333 acquires user activity data associated with a specific time. Normally, the activity acquisition unit 333 acquires activity data associated with the time acquired by the time acquisition unit 331. The process of acquiring activity data is based on publicly known technology.

[0210] The vital sign acquisition unit 334 acquires one or more types of vital data of the user corresponding to a given time. Typically, the vital sign acquisition unit 334 acquires one or more types of vital data corresponding to a given time acquired by the time acquisition unit 331. The process of acquiring vital data is based on publicly known technology.

[0211] The behavior estimation unit 335 uses two or more source information pieces, including location information associated with time, to acquire behavior information that identifies the user's behavior during the time period specified by the time of each of the two or more source information pieces. Preferably, the source information pieces also include activity data. Preferably, the source information pieces also include one or more types of vital data.

[0212] The behavior estimation unit 335 detects, for example, behavioral conditions in which two or more source pieces of behavioral information, including location information corresponding to a time, match, and acquires behavioral information that is paired with those behavioral conditions from the storage unit 31.

[0213] The behavior estimation unit 335 acquires behavior information "work" if, for example, at 13:15 the user is sitting at a desk indoors at home. The behavior estimation unit 335 acquires behavior information "watching TV" if, for example, at 20:17 the user is sitting in the living room indoors at home. The behavior estimation unit 335 acquires behavior information "drinking party" if, for example, at 20:17 the user acquires location information "izakaya" (Japanese pub).

[0214] The behavior estimation unit 335 may acquire behavior information for a given time period using the behavior learning information from the learning management unit 311 and two or more pieces of behavior source information, including location information associated with a given time. The processing of the behavior estimation unit 335 in the cases where the learning information is a learning model and in the case where it is a correspondence table will be described below.

[0215] The behavior estimation unit 335 may acquire one or more user attribute values, acquire behavior learning information from the learning management unit 311 that matches the user attribute value conditions to which the one or more user attribute values ​​match, and then acquire behavior information using the behavior learning information. (1) When behavioral learning information is a behavioral learning model

[0216] The behavior estimation unit 335 acquires the learning model from the learning management unit 311. The behavior estimation unit 335 also acquires the time and one or more source information for actions associated with that time. Next, the behavior estimation unit 335 provides the time, the source information for actions, and the learning model to a machine learning prediction processing module, executes the module, and acquires the action information.

[0217] Furthermore, if the score output by the module is below or equal to the threshold, the behavior estimation unit 335 does not need to acquire behavior information. (2) When behavioral learning information is a behavioral correspondence table

[0218] The behavior estimation unit 335 obtains the time and the source information of the action associated with that time. Next, the behavior estimation unit 335 obtains a source vector whose elements are the time and the source information of the action. Next, the behavior estimation unit 335 calculates the similarity between the source vector of the action and the source vectors of each of the two or more action correspondence information contained in the action correspondence table. Next, the behavior estimation unit 335 obtains the action information from the action correspondence table that is paired with the source vector with the highest similarity. Note that even if the similarity is the highest, the behavior estimation unit 335 does not need to obtain the action information if the similarity is below or equal to a threshold.

[0219] The emotion estimation unit 336 uses behavioral information or behavioral source information to obtain emotion information regarding the user's emotions during a given time period.

[0220] The emotion estimation unit 336 detects, for example, two or more emotion source information, including location information corresponding to a time, that match an emotion condition, and obtains the emotion information that is paired with that emotion condition from the storage unit 31.

[0221] For example, when sitting at the position of the desk at home indoors at 13:15, the emotion estimation unit 336 acquires the emotion information "positive". For example, when sitting in the living room of the home indoors at 20:17, the emotion estimation unit 336 acquires the emotion information "positive". For example, when the position information "izakaya" is acquired at 20:17, the emotion estimation unit 336 acquires the emotion information "positive".

[0222] The emotion estimation unit 336 uses the emotion learning information of the learning management unit 311 and the action information acquired by the action estimation unit 335 or one or more action source information from which the action information is obtained to acquire emotion information in a time zone. Here, since the action information or one or more pieces of action source information are used for acquiring emotion information, they are referred to as emotion source information.

[0223] The emotion estimation unit 336 may acquire one or more user attribute values, acquire from the learning management unit 311 the emotion learning information that pairs with the user attribute value conditions that match the one or more user attribute values, and use the emotion learning information to acquire emotion information.

[0224] The emotion estimation unit 336 will hereinafter explain the processing of the action estimation unit 335 when the emotion learning information is an emotion learning model and when it is an emotion correspondence table. (1) When the emotion learning information is an emotion learning model

[0225] The emotion estimation unit 336 acquires the emotion learning model of the learning management unit 311. Also, the emotion estimation unit 336 acquires one or more pieces of action source information or action information associated with the time. Next, the emotion estimation unit 336 provides the acquired one or more pieces of action source information or action information and the emotion learning model to a module that performs prediction processing of machine learning, executes the module, and acquires emotion information.

[0226] Note that when the score output by the module is below or less than the threshold value, the emotion estimation unit 336 does not have to acquire emotion information. (2) When the emotion learning information is an emotion correspondence table

[0227] The emotion estimation unit 336 acquires one or two or more pieces of actor information or action information associated with the time. Next, the emotion estimation unit 336 acquires an emotion source vector having the one or more pieces of actor information or action information as elements. Next, the emotion estimation unit 336 calculates the similarity between the emotion source vector and the emotion source vectors of each of the two or more pieces of corresponding information in the emotion correspondence table. Next, the emotion estimation unit 336 acquires the emotion information paired with the emotion source vector having the maximum similarity from the emotion correspondence table. Note that the emotion estimation unit 336 does not have to acquire the emotion information even if the similarity is the maximum and the similarity is less than or equal to the threshold value or less than the threshold value.

[0228] The location acquisition unit 337 refers to the map of the map management unit 312 and acquires location information corresponding to the location information acquired by the location acquisition unit 332. The location information in such a case is usually absolute location information (for example, (latitude, longitude)).

[0229] It is preferable that the location acquisition unit 337 acquires the location information only for the time period when the action estimation unit 335 does not acquire the action information.

[0230] The storage unit 338 associates the action information acquired by the action estimation unit 335 with the time period and stores it in the action management unit 313.

[0231] The storage unit 338 may associate the emotion information acquired by the emotion estimation unit 336 with the time period and store it in the action management unit 313.

[0232] The composition unit 339 composes the information to be output using the action information of the action management unit 313. The composition unit 339 composes the information to be output using, for example, the emotion information of the action management unit 313.

[0233] The component 339 configures output information having corresponding behavioral information for each time period in an area specified by each of two or more time periods on the calendar. It is preferable that the output information is arranged so that confirmed behavioral information and unconfirmed behavioral information can be visually distinguished. It is preferable that the component 339 configures output information that visually indicates emotional information corresponding to each of two or more time periods. It is preferable that the component 339 configures output information so that time periods where the emotional information is "positive" and time periods where the emotional information is "negative" are visually distinguishable. It is preferable that the component 339 configures the output information so that the background colors of time periods where the emotional information is "positive" and time periods where the emotional information is "negative" are different.

[0234] The output unit 34 outputs various types of information. These types of information include, for example, behavioral information, emotional information, and location information.

[0235] Here, "output" usually refers to display on a screen, but it may also be a concept that includes projection using a projector, printing with a printer, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0236] The behavior output unit 341 outputs behavior information for one or more time periods.

[0237] It is preferable for the behavior output unit 341 to output the location acquired by the location acquisition unit 337 when the behavior estimation unit 335 is unable to acquire behavior information.

[0238] The action output unit 341 preferably outputs two or more action information so that confirmed action information and unconfirmed action information can be visually distinguished.

[0239] The emotion output unit 342 outputs emotion information. For example, the emotion output unit 342 outputs emotion information acquired by the emotion estimation unit 336. It is preferable for the emotion output unit 342 to output emotion information for one or more time periods.

[0240] Various types of information are stored in the server storage unit 41 that constitutes the server device 4. These types of information include, for example, the learning information mentioned above, source information and time associated with two or more user identifiers, two or more behavioral training data, and two or more emotion training data.

[0241] The server receiving unit 42 receives various instructions and information. These instructions and information include, for example, instructions to transmit information. The information in this case includes, for example, learning information and action source information.

[0242] The server processing unit 43 performs various processes. These processes include, for example, learning processes. These learning processes include, for example, behavioral learning processes and emotion learning processes.

[0243] The behavioral learning process is a process that acquires a behavioral learning model using two or more behavioral training data. The server processing unit 43, for example, provides two or more behavioral training data to a machine learning learning processing module, executes the module, acquires a behavioral learning model, and stores it in the server storage unit 41. For example, for each of the two or more candidate behavioral information, the server processing unit 43 provides two or more positive examples, which are training data containing behavioral information, and two or more negative examples, which are training data not containing behavioral information, to the machine learning learning processing module, executes the module, acquires a behavioral learning model for each of the candidate behavioral information, and stores it in the server storage unit 41 in association with the behavioral information. For example, the server processing unit 43 acquires a behavioral correspondence table, which is a table in which each of the two or more behavioral training data is a record, and stores it in the server storage unit 41.

[0244] Sentiment learning processing is a process of obtaining a sentiment learning model using two or more sentiment teacher data. The server processing unit 43, for example, provides two or more sentiment teacher data to a learning processing module of machine learning, executes the module, obtains a sentiment learning model, and stores it in the server storage unit 41. The server processing unit 43, for example, for each candidate of two or more pieces of sentiment information, provides two or more positive examples, which are teacher data including sentiment information, and two or more negative examples, which are teacher data not including sentiment information, to a learning processing module of machine learning, executes the module, obtains a sentiment learning model for each candidate of sentiment information, and stores it in the server storage unit 41 in association with the sentiment information. The server processing unit 43, for example, obtains a sentiment correspondence table, which is a table having two or more pieces of each sentiment teacher data as records, and stores it in the server storage unit 41.

[0245] The server transmission unit 44 transmits various types of information. The various types of information are, for example, an action learning model, a sentiment learning model, an action correspondence table, and a sentiment correspondence table.

[0246] The storage unit 31, the learning management unit 311, the map management unit 312, the action management unit 313, and the server storage unit 41 are preferably non-volatile recording media, but can also be realized with volatile recording media.

[0247] The process of storing information in the storage unit 31 or the like is not limited. For example, information may be stored in the storage unit 31 or the like via a recording medium, or information transmitted via a communication line or the like may be stored in the storage unit 31 or the like, or information input via an input device may be stored in the storage unit 31 or the like.

[0248] The receiving unit 32, the server receiving unit 42, and the server transmission unit 44 are usually realized by wireless or wired communication means.

[0249] The processing unit 33, time acquisition unit 331, position acquisition unit 332, activity acquisition unit 333, vital sign acquisition unit 334, behavior estimation unit 335, emotion estimation unit 336, location acquisition unit 337, storage unit 338, configuration unit 339, and server processing unit 43 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 33, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0250] The output unit 34, the behavior output unit 341, and the emotion output unit 342 may or may not be considered to include output devices such as displays and speakers. The output unit 34 can be implemented using driver software for an output device, or driver software for an output device and an output device, etc.

[0251] Next, an example of the operation of the behavior acquisition device 3, which constitutes information system D, will be explained using the flowchart in Figure 19.

[0252] (Step S1901) The processing unit 33 decides whether or not to acquire the information. If it decides to acquire the information, it proceeds to step S1902; if it decides not to acquire the information, it proceeds to step S1916. The processing unit 33 may always decide to acquire the information, or it may decide to acquire the information only when a flag indicating the acquisition of information is stored in the storage unit 31, etc. The conditions for such a decision are not specified.

[0253] (Step S1902) The time acquisition unit 331 acquires the time from a clock (not shown). Here, the time acquisition unit 331 may also acquire the day of the week.

[0254] (Step S1903) The processing unit 33 acquires one or more action source information. An example of such action source acquisition process will be explained using the flowchart in Figure 20.

[0255] (Step S1904) The behavior estimation unit 335 estimates behavior information that identifies the user's behavior using one or more behavior source information obtained in step S1903. An example of such behavior estimation processing will be explained using the flowcharts in Figures 22 to 24.

[0256] (Step S1905) The location acquisition unit 337 determines whether or not it was able to acquire the action information in step S1904. If it was able to acquire the action information, it proceeds to step S1907; if it was not able to acquire the action information, it proceeds to step S1906.

[0257] (Step S1906) The location acquisition unit 337 refers to the map on the map management unit 312 and acquires location information corresponding to the location information acquired in step S1903. Note that there may be times when location information cannot be acquired at this stage.

[0258] (Step S1907) The storage unit 338 stores action information in a buffer (not shown), and determines whether the action information stored immediately before matches the action information acquired in step S1904. If they match, the process proceeds to step S1908; otherwise, the process proceeds to step S1912.

[0259] (Step S1908) The storage unit 338 associates the acquired action information, etc. with the time acquired in step S1902 and stores it in a buffer (not shown).

[0260] (Step S1909) The emotion estimation unit 336 performs a process to estimate emotion information. An example of such emotion estimation process will be explained using the flowcharts in Figures 25 to 27.

[0261] (Step S1910) The storage unit 338 determines whether or not emotional information was obtained in step S1909. If emotional information was obtained, the unit proceeds to step S1911; otherwise, it returns to step S1901.

[0262] (Step S1911) The storage unit 338 stores the emotion information acquired in step S1909 in a buffer (not shown) corresponding to the time acquired in step S1902. The process returns to step S1901.

[0263] (Step S1912) The storage unit 338 stores the time obtained in step S1902 and the acquired action information, etc., in a buffer (not shown) in association with each other.

[0264] (Step S1913) The storage unit 338 acquires information about the most recent action, etc.

[0265] (Step S1914) The storage unit 338 acquires a time period specified by two or more time points that correspond to the most recent action information, etc.

[0266] (Step S1915) The storage unit 338 associates the time period acquired in step S1914 with the most recent activity information acquired in step S1913 and stores it in the activity management unit 313. The storage unit 338 may also associate the time period with the most recent activity information and store it in association with a user identifier. In this case, the storage destination is preferably the server device 4.

[0267] (Step S1916) The action acquisition device 3 determines whether or not it has received an output instruction. If it has received an output instruction, it proceeds to step S1917; otherwise, it proceeds to step S1919.

[0268] (Step S1917) The component 339 uses the behavior information from the behavior management unit 313 to construct output information. An example of such output configuration processing will be explained using the flowchart in Figure 28.

[0269] (Step S1918) The output unit 34 outputs the output information configured in step S1917. Return to step S1901.

[0270] (Step S1919) The action acquisition device 3 determines whether or not it has received input information based on the output information being output. If it has received input information, it proceeds to step S1920; otherwise, it proceeds to step S1923.

[0271] (Step S1920) The processing unit 33 determines whether the information received in step S1919 is a confirmation instruction for estimated behavioral information or estimated emotional information. If it is a confirmation instruction, the process proceeds to step S1921; otherwise, the process proceeds to step S1922.

[0272] (Step S1921) The storage unit 338 performs processing to confirm the behavioral information or emotional information corresponding to the confirmation instruction. The process returns to step S1901. This processing, for example, involves associating a confirmation flag with the behavioral information or emotional information corresponding to the confirmation instruction.

[0273] (Step S1922) The storage unit 338 stores the input information. The process returns to step S1901. The input information is, for example, correct behavioral information or correct emotional information. The storage unit 338 then updates the input behavioral information or emotional information with the estimated behavioral information or emotional information corresponding to the input information. The storage unit 338 also performs processing to confirm such behavioral information or emotional information.

[0274] (Step S1923) The action acquisition device 3 determines whether or not it has received a learning instruction. If it has received a learning instruction, it proceeds to step S1924; otherwise, it returns to step S1901.

[0275] (Step S1924) The learning unit or learning device (not shown) of the behavior acquisition device 3 uses two or more behavior teacher data, including behavior information, to construct behavior learning information, and stores it in the learning management unit 311. An example of such behavior learning processing will be explained using the flowcharts in Figures 29 and 30.

[0276] (Step S1925) The learning unit or learning device (not shown) of the behavior acquisition device 3 uses two or more emotion training data, including emotion information, to construct emotion learning information, and stores it in the learning management unit 311. Return to step S1901. An example of such emotion learning processing will be explained using the flowcharts in Figures 31 and 32.

[0277] In the flowchart shown in Figure 19, processing is terminated by power-off or processing termination interrupts.

[0278] Next, an example of the action source acquisition process in step S1903 will be explained using the flowchart in Figure 20.

[0279] (Step S2001) The processing unit 33 determines whether the receiving unit 32 has been able to acquire a GPS signal. If a GPS signal has been acquired, the process proceeds to step S2002; otherwise, the process proceeds to step S2003.

[0280] (Step S2002) The position acquisition unit 332 acquires absolute position information based on the GPS signal received by the receiving unit 32.

[0281] (Step S2003) The position acquisition unit 332 acquires position information. An example of such position estimation processing will be explained using the flowchart in Figure 21.

[0282] (Step S2004) The activity acquisition unit 333 acquires user activity data.

[0283] (Step S2005) The vital signs acquisition unit 334 acquires one or more types of vital signs data from the user.

[0284] (Step S2006) The behavior estimation unit 335 acquires one or more past behavioral information. Note that the one or more past behavioral information includes the most recent behavioral information in terms of time.

[0285] (Step S2007) The behavior estimation unit 335 determines whether or not to use emotional information for the behavior estimation process. If emotional information is to be used, the process proceeds to step S2008; otherwise, it proceeds to step S2009. Note that whether or not emotional information is to be used for the behavior estimation process is usually predetermined.

[0286] (Step S2008) The emotion estimation unit 336 acquires emotion information. An example of such emotion estimation processing will be explained using the flowcharts in Figures 25 to 27.

[0287] (Step S2009) The behavior estimation unit 335 obtains the elapsed time since the start of the new behavior identified by the new behavior information.

[0288] (Step S2010) The behavior estimation unit 335 uses two or more types of behavior source information, including time and location information, to construct information to be used for behavior estimation processing. It returns to the higher-level processing. Here, the constructed information is usually a set of behavior source information, for example, a behavior source vector whose elements are two or more of each behavior source information. The two or more types of behavior source information are, for example, two or more types of information from time, day of the week, location information, activity data, vital data, past behavior information, emotional information, and elapsed time.

[0289] In addition, even if a GPS signal is acquired in the flowchart of Figure 20, the position acquisition unit 332 may acquire position information by the position estimation process described using the flowchart of Figure 21.

[0290] Furthermore, in the flowchart of Figure 20, the action estimation unit 335 may acquire one or more user attribute values ​​and acquire action source information including those one or more user attribute values. Note that the user attribute values ​​may be information stored in the storage unit 31, or information entered by the user, etc. Next, an example of the position estimation process in step S2003 will be explained using the flowchart of Figure 21. In the flowchart of Figure 21, the explanation of the same steps as in Figure 13 will be omitted. Also, the position estimation process in step S2003 may be the same process as in the flowchart of Figure 13.

[0291] (Step S2101) The position acquisition unit 332 acquires the similarity between the two radio wave intensity vectors, associates this similarity with the i-th location information, and temporarily stores it in a buffer (not shown). Proceed to step S1308.

[0292] (Step S2102) The location acquisition unit 332 acquires the location information of the point information that is paired with the point information with the highest similarity. It returns to the higher-level processing. The acquired location information is the terminal location.

[0293] Next, an example of the first behavior estimation process in step S1904 will be explained using the flowchart in Figure 22. The flowchart in Figure 22 shows a process that estimates behavioral information using a machine learning prediction process with a single behavioral learning model. In other words, the flowchart in Figure 22 shows a process that estimates behavioral information using a multi-class classification prediction process of machine learning.

[0294] (Step S2201) The action estimation unit 335 acquires two or more types of action source information (for example, action source vectors) obtained in step S1903.

[0295] (Step S2202) The behavior estimation unit 335 obtains the behavioral learning model from the learning management unit 311.

[0296] (Step S2203) The behavior estimation unit 335 provides the two or more types of behavior source information and the behavior learning model obtained in step S2201 to a machine learning prediction processing module and executes the module.

[0297] (Step S2204) The behavior estimation unit 335 obtains the estimated behavior information and score, which are the execution results in step S2203.

[0298] (Step S2205) The behavior estimation unit 335 determines whether the score obtained in step S2204 is above a threshold. If it is above the threshold, the unit proceeds to step S2206; otherwise, the unit proceeds to step S2207.

[0299] (Step S2206) The behavior estimation unit 335 acquires the behavior information obtained in step S2204 as output behavior information. It returns to the higher-level processing.

[0300] (Step S2207) The action estimation unit 335 obtains the action information for "empty". It returns to the higher-level processing.

[0301] Note that in the flowchart of Figure 22, steps S2205 to S2207 do not need to be performed.

[0302] Next, an example of the second behavior estimation process in step S1904 will be explained using the flowchart in Figure 23. The flowchart in Figure 23 shows a process in which behavior information is estimated by machine learning prediction using a behavior learning model for each of the two or more candidate behavior information. In other words, the flowchart in Figure 22 shows a process in which behavior information is estimated by machine learning binary classification prediction.

[0303] (Step S2301) The behavior estimation unit 335 acquires two or more types of behavior source information obtained in step S1903.

[0304] (Step S2302) The action estimation unit 335 assigns 1 to counter i.

[0305] (Step S2303) The action estimation unit 335 refers to the learning management unit 311 and determines whether or not a candidate for the i-th action information exists. If a candidate for the i-th action information exists, the unit proceeds to step S2304; otherwise, the unit proceeds to step S2309.

[0306] (Step S2304) The behavior estimation unit 335 obtains the i-th behavior learning model, which is paired with the i-th candidate behavior information, from the learning management unit 311.

[0307] (Step S2305) The behavior estimation unit 335 provides the two or more types of behavior source information obtained in step S2301 and the i-th behavior learning model obtained in step S2304 to a prediction processing module that performs binary classification using machine learning, and executes the module.

[0308] (Step S2306) The action estimation unit 335 determines whether the execution result in step S2305 is "true" or not. If it is "true", proceed to step S2307; if it is false, proceed to step S2308.

[0309] (Step S2307) The action estimation unit 335 associates the i-th candidate action information with a score which is part of the execution result in step S2305 and temporarily stores it in a buffer that is not shown.

[0310] (Step S2308) The action estimation unit 335 increments counter i by 1. Return to step S2303.

[0311] (Step S2309) The behavior estimation unit 335 obtains the maximum score and determines whether the score is above or below the threshold. If the maximum score is above or below the threshold, the unit proceeds to step S2310; otherwise, the unit proceeds to step S2311.

[0312] (Step S2310) The action estimation unit 335 retrieves the action information that corresponds to the highest score. It returns to the higher-level processing.

[0313] (Step S2311) The action estimation unit 335 obtains the action information for "empty". It returns to the higher-level processing.

[0314] Note that in the flowchart of Figure 23, the processes from steps S2308 to S2311 do not need to be performed.

[0315] Next, an example of the third behavior estimation process in step S1904 will be explained using the flowchart in Figure 24. The flowchart in Figure 24 shows the process of estimating behavior information using a behavior correspondence table.

[0316] (Step S2401) The action estimation unit 335 acquires two or more types of action source information obtained in step S1903. The two or more types of action source information here are action source vectors.

[0317] (Step S2402) The action estimation unit 335 assigns 1 to counter i.

[0318] (Step S2403) The action estimation unit 335 determines whether or not the i-th action correspondence information exists in the action correspondence table of the learning management unit 311. If the i-th action correspondence information exists, the unit proceeds to step S2404; otherwise, the unit proceeds to step S2407.

[0319] (Step S2404) The action estimation unit 335 obtains the i-th action source vector that the i-th action correspondence information has.

[0320] (Step S2405) The action estimation unit 335 obtains the similarity between the action source vector obtained in step S2403 and the action source vector obtained in step S2404, and associates it with the i-th action correspondence information.

[0321] (Step S2406) The action estimation unit 335 increments counter i by 1. Return to step S2403.

[0322] (Step S2407) The behavior estimation unit 335 obtains the highest similarity.

[0323] (Step S2408) The behavior estimation unit 335 determines whether the maximum similarity obtained in step S2407 is greater than or equal to a threshold. If it is greater than or equal to the threshold, it proceeds to step S2409; otherwise, it proceeds to step S2410.

[0324] (Step S2409) The behavior estimation unit 335 obtains the behavior information associated with the i-th behavior correspondence information that is paired with the highest similarity. It returns to the higher-level processing.

[0325] (Step S2410) The action estimation unit 335 obtains action information for "empty". It returns to the higher-level processing.

[0326] Note that in the flowchart of Figure 24, steps S2408 to S2411 do not need to be performed.

[0327] Next, an example of the first emotion estimation process in step S1909 will be explained using the flowchart in Figure 25. The flowchart in Figure 25 shows a process that estimates emotion information using a machine learning prediction process with a single emotion learning model. In other words, the flowchart in Figure 25 shows a process that estimates emotion information using a machine learning multi-class classification prediction process.

[0328] (Step S2501) The emotion estimation unit 336 acquires two or more types of behavior source information obtained in step S1903. Here, each of the two or more types of behavior source information is emotion source information. Furthermore, the two or more types of emotion source information are, for example, emotion source vectors.

[0329] (Step S2502) The emotion estimation unit 336 obtains the emotion learning model from the learning management unit 311.

[0330] (Step S2503) The emotion estimation unit 336 provides the two or more types of emotion source information obtained in step S2501 and the emotion learning model to a machine learning prediction processing module and executes the module.

[0331] (Step S2504) The emotion estimation unit 336 obtains the estimated emotion information and score, which are the execution results in step S2503.

[0332] (Step S2505) The emotion estimation unit 336 determines whether the score obtained in step S2504 is above a threshold. If it is above the threshold, the unit proceeds to step S2506; otherwise, the unit proceeds to step S2507.

[0333] (Step S2506) The emotion estimation unit 336 acquires the emotion information obtained in step S2504 as the output emotion information. It returns to the higher-level processing.

[0334] (Step S2507) The emotion estimation unit 336 obtains the emotion information for "empty". It returns to the higher-level processing.

[0335] Note that in the flowchart of Figure 25, steps S2505 to S2507 do not need to be performed.

[0336] Next, an example of the second emotion estimation process in step S1909 will be explained using the flowchart in Figure 26. The flowchart in Figure 26 shows a process in which emotion information is estimated by machine learning prediction using an emotion learning model for each of two or more candidate emotion information. In other words, the flowchart in Figure 26 shows a process in which emotion information is estimated by machine learning binary classification prediction.

[0337] (Step S2601) The emotion estimation unit 336 acquires two or more types of emotion source information obtained in step S1903.

[0338] (Step S2602) The emotion estimation unit 336 assigns 1 to counter i.

[0339] (Step S2603) The emotion estimation unit 336 refers to the learning management unit 311 and determines whether or not a candidate for the i-th emotion information exists. If a candidate for the i-th emotion information exists, the unit proceeds to step S2604; otherwise, the unit proceeds to step S2609.

[0340] (Step S2604) The emotion estimation unit 336 obtains the i-th emotion learning model, which is paired with the i-th candidate emotion information, from the learning management unit 311.

[0341] (Step S2605) The emotion estimation unit 336 provides the two or more types of emotion source information obtained in step S2601 and the i-th emotion learning model obtained in step S2604 to a prediction processing module that performs binary classification using machine learning, and executes the module.

[0342] (Step S2606) The emotion estimation unit 336 determines whether the result of the execution in step S2605 is "true" or not. If it is "true", proceed to step S2607; if it is false, proceed to step S2608.

[0343] (Step S2607) The emotion estimation unit 336 associates the i-th candidate emotion information with a score which is part of the execution result in step S2605 and temporarily stores it in a buffer that is not shown.

[0344] (Step S2608) The emotion estimation unit 336 increments counter i by 1. Return to step S2603.

[0345] (Step S2609) The emotion estimation unit 336 obtains the maximum score and determines whether the score is above or below the threshold. If the maximum score is above or below the threshold, the unit proceeds to step S2610; otherwise, the unit proceeds to step S2611.

[0346] (Step S2610) The emotion estimation unit 336 retrieves the emotion information that corresponds to the highest score. It returns to the higher-level processing.

[0347] (Step S2611) The emotion estimation unit 336 obtains the emotion information for "empty". It returns to the higher-level processing unit.

[0348] Note that in the flowchart of Figure 26, steps S2608 to S2611 do not need to be performed.

[0349] Next, an example of the third emotion estimation process in step S1909 will be explained using the flowchart in Figure 27. The flowchart in Figure 27 shows the process of estimating emotion information using an emotion correspondence table.

[0350] (Step S2701) The emotion estimation unit 336 acquires two or more types of emotion source information obtained in step S1903. The two or more types of emotion source information here are emotion source vectors.

[0351] (Step S2702) The emotion estimation unit 336 assigns 1 to counter i.

[0352] (Step S2703) The emotion estimation unit 336 determines whether or not the i-th emotion correspondence information exists in the emotion correspondence table of the learning management unit 311. If the i-th emotion correspondence information exists, the unit proceeds to step S2704; otherwise, the unit proceeds to step S2707.

[0353] (Step S2704) The emotion estimation unit 336 obtains the i-th emotion source vector that the i-th emotion correspondence information has.

[0354] (Step S2705) The emotion estimation unit 336 obtains the similarity between the emotion source vector obtained in step S2703 and the emotion source vector obtained in step S2704, and associates it with the i-th emotion correspondence information.

[0355] (Step S2706) The emotion estimation unit 336 increments counter i by 1. Return to step S2703.

[0356] (Step S2707) The emotion estimation unit 336 obtains the highest similarity.

[0357] (Step S2708) The emotion estimation unit 336 determines whether the maximum similarity obtained in step S2707 is equal to or greater than a threshold. If it is equal to or greater than the threshold, it proceeds to step S2709; otherwise, it proceeds to step S2710.

[0358] (Step S2709) The emotion estimation unit 336 obtains the emotion information associated with the i-th emotion correspondence information that is paired with the highest similarity. It returns to the higher-level processing.

[0359] (Step S2710) The emotion estimation unit 336 obtains the emotion information for "empty". It returns to the higher-level processing.

[0360] Note that in the flowchart of Figure 27, steps S2708 to S2711 do not need to be performed.

[0361] Next, an example of the output configuration process in step S1917 will be explained using the flowchart in Figure 28.

[0362] (Step S2801) The component 339 obtains a calendar template from the storage unit 31.

[0363] (Step S2802) Component 339 assigns 1 to counter i.

[0364] (Step S2803) The component 339 determines whether or not the i-th time period stored in the behavior management unit 313 exists. If the i-th time period exists, the process proceeds to step S2804; otherwise, it returns to the higher-level process. The behavior management unit 313 stores behavior information and emotion information associated with one or more time periods.

[0365] (Step S2804) The component 339 determines whether the i-th time period stored in the activity management unit 313 is included in the period covered by the calendar template obtained in step S2801. If it is included, proceed to step S2805; otherwise, proceed to step S2809.

[0366] (Step S2805) The component 339 obtains the behavior information corresponding to the i-th time period from the behavior management unit 313.

[0367] (Step S2806) The component 339 obtains emotional information corresponding to the i-th time period from the behavior management unit 313. Note that it is not necessary to obtain emotional information at this point.

[0368] (Step S2807) Component 339 constitutes time zone information that is to be placed in the i-th time zone of the calendar, and is information that can identify the behavioral information acquired in step S2805 and information that can identify the emotional information acquired in step S2806.

[0369] (Step S2808) The component 339 places the time zone information configured in step S2807 at the position in the calendar specified by the i-th time zone.

[0370] (Step S2809) Component 339 increments counter i by 1. Return to step S2803.

[0371] Next, an example of the first behavioral learning process in step S1924 will be explained using the flowchart in Figure 29. Note that the behavioral learning process is performed by a learning unit (not shown), for example. The learning unit may be a different learning device from the behavioral acquisition device 3. The first behavioral learning process is the process of acquiring a multi-class classification behavioral learning model.

[0372] (Step S2901) The learning unit assigns 1 to counter i.

[0373] (Step S2902) The learning unit determines whether or not the i-th action information exists in the action management unit 313. If the i-th action information exists, the unit proceeds to step S2903; otherwise, it proceeds to step S2905.

[0374] (Step S2903) The learning unit constructs behavioral training data using the i-th behavioral information, etc., and appends it to a buffer (not shown). Behavioral training data is usually information in which two or more behavioral source information are used as explanatory variables and behavioral information is used as the dependent variable.

[0375] (Step S2904) The learning unit increments counter i by 1. Return to step S2902.

[0376] (Step S2905) The learning unit provides two or more behavioral training data stored in a buffer (not shown) to a machine learning learning processing module, executes the module, and obtains a behavioral learning model.

[0377] (Step S2906) The learning unit stores the behavioral learning model acquired in step S2905 in the learning management unit 311.

[0378] In addition, in the flowchart of Figure 29, the learning unit may, without performing the learning processes of steps S2905 and S2906, store an action correspondence table in the learning management unit 311, in which two or more action teacher data are records (action correspondence information) in a buffer not shown.

[0379] Next, an example of the second behavioral learning process in step S1924 will be explained using the flowchart in Figure 30. The second behavioral learning process is the process of obtaining a binary classification behavioral learning model for each of the two or more candidate behavioral information.

[0380] (Step S3001) The learning unit assigns 1 to counter i.

[0381] (Step S3002) The learning unit determines whether or not the i-th type of action information exists. If the i-th type of action information exists, the unit proceeds to step S3003; otherwise, it returns to the higher-level processing.

[0382] (Step S3003) The learning unit obtains the i-th type of action information.

[0383] (Step S3004) The learning unit obtains two or more positive examples, which are training data in the behavior management unit 313 and are behavior training data containing the i-th type of behavior information.

[0384] (Step S3005) The learning unit obtains two or more negative examples, which are training data within the behavior management unit 313 and are behavioral training data that do not contain the i-th type of behavioral information.

[0385] (Step S3006) The learning unit provides the two or more positive examples obtained in Step S3004 and the two or more negative examples obtained in Step S3005 to a machine learning learning processing module, executes the module, and obtains an action learning model.

[0386] (Step S3007) The learning unit stores the behavioral learning model acquired in step S3006 in the learning management unit 311, paired with the i-th type of behavioral information.

[0387] (Step S3008) The learning unit increments counter i to 1. Return to step S3002.

[0388] Next, an example of the first emotion learning process in step S1925 will be explained using the flowchart in Figure 31. Note that the behavior learning process is performed by a learning unit (not shown in the diagram). The learning unit may be a different learning device from the behavior acquisition device 3. The first emotion learning process is the process of acquiring a multi-class classification emotion learning model.

[0389] (Step S3101) The learning unit assigns 1 to counter i.

[0390] (Step S3102) The learning unit determines whether or not the i-th emotion information exists in the behavior management unit 313. If the i-th emotion information exists, the unit proceeds to step S3103; otherwise, it proceeds to step S3105.

[0391] (Step S3103) The learning unit constructs emotion training data using the i-th emotion information, etc., and appends it to a buffer (not shown). The emotion training data is information in which two or more types of emotion source information are used as explanatory variables and emotion information is used as the dependent variable.

[0392] (Step S3104) The learning unit increments counter i by 1. Return to step S3102.

[0393] (Step S3105) The learning unit provides two or more emotion training data stored in a buffer (not shown) to a machine learning learning processing module, executes the module, and obtains an emotion learning model.

[0394] (Step S3106) The learning unit stores the emotion learning model acquired in step S3105 in the learning management unit 311.

[0395] In addition, in the flowchart of Figure 31, the learning unit may, without performing the learning processes of steps S3105 and S3106, store an emotion correspondence table in the learning management unit 311, in which two or more emotion teacher data are recorded (emotion correspondence information) in a buffer not shown.

[0396] Next, an example of the second emotion learning process in step S1925 will be explained using the flowchart in Figure 32. The second emotion learning process is the process of obtaining a binary classification emotion learning model for each of the two or more candidate emotion information.

[0397] (Step S3201) The learning unit assigns 1 to counter i.

[0398] (Step S3202) The learning unit determines whether or not the i-th type of emotion information exists. If the i-th type of emotion information exists, the unit proceeds to step S3203; otherwise, it returns to the higher-level processing.

[0399] (Step S3203) The learning unit acquires the i-th type of emotion information.

[0400] (Step S3204) The learning unit obtains two or more positive examples, which are emotion training data in the behavior management unit 313 and which contain the i-th type of emotion information.

[0401] (Step S3205) The learning unit obtains two or more negative examples, which are emotion training data within the behavior management unit 313 and do not contain the i-th type of emotion information.

[0402] (Step S3206) The learning unit provides the two or more positive examples obtained in step S3204 and the two or more negative examples obtained in step S3205 to a machine learning learning processing module, executes the module, and obtains an emotion learning model.

[0403] (Step S3207) The learning unit stores the emotion learning model acquired in step S3206 in the learning management unit 311, paired with the i-th type of emotion information.

[0404] (Step S3208) The learning unit increments counter i by 1. Return to step S3202.

[0405] The following describes a specific example of the operation of the information system D in this embodiment. Currently, the storage unit 31 of the server device 4 stores an action learning model acquired through machine learning processing using a large amount of training data containing the source information of one or more users' actions. The storage unit 31 also stores an emotion learning model acquired through machine learning processing using a large amount of training data containing the source information of one or more users' emotions.

[0406] Furthermore, in the storage unit 31 of the action acquisition device 3, which is a terminal (for example, a smartwatch) held by user "U1", a large amount of action source information for two or more consecutive time points acquired by the processing unit 33 through the above-described process is stored in the action source management table shown in Figure 33.

[0407] The activity source management table (Figure 33) is a record that has "ID," "Time Information," "Physical Data," "Location Information," and "Elapsed Time." "ID" is information that identifies the record. "Time Information" is information that specifies the time, and in this case it has "Month and Day," "Time of Day," and "Day of the Week." "Physical Data" has "Activity Data" and "Vital Data." "Vital Data" has "Heart Rate," "Blood Pressure (Upper)," "Blood Pressure (Lower)," and "Body Temperature." "Heart Rate" is the heart rate per unit of time (in this case, "1 minute"). "Blood Pressure (Upper)" is the systolic blood pressure, and "Blood Pressure (Lower)" is the diastolic blood pressure. "Location Information" is location information obtained by the process described in Embodiment 2, for example. "Elapsed Time" is the time that has elapsed since the same activity started.

[0408] Then, user "U1" inputs an output command to the action acquisition device 3. The action acquisition device 3 then accepts the output command.

[0409] Next, the behavior estimation unit 335 accesses, for example, the server device 4, receives the behavior learning model and the emotion learning model from the server device 4, and stores the behavior learning model and the emotion learning model in the learning management unit 311.

[0410] Next, the behavior estimation unit 335 constructs an action source vector for each record in Figure 33, for example, by processing as described using the flowchart in Figure 22, and includes time information, body data, location information, elapsed time, etc. Next, the behavior estimation unit 335 obtains an action learning model from the learning management unit 311. Next, the behavior estimation unit 335 provides the action source vector and the action learning model to a machine learning prediction processing module and executes the module. The behavior estimation unit 335 then obtains action information "A1" for the action source vectors from record "ID=1" to record "ID=289", and obtains action information "A2" for the action source vectors from record "ID=290" to record "ID=N". Finally, the storage unit 338 stores the action information and the action confirmation flag "0" for each record in the behavior management unit 313. Note that "0" for the action confirmation flag and emotion confirmation flag indicates that it is not yet confirmed, and "1" indicates that it is confirmed.

[0411] Furthermore, the emotion estimation unit 336 constructs an emotion source vector for each record in Figure 33, for example, by processing as described using the flowchart in Figure 25, which includes time information, physical data, location information, elapsed time, behavior information, etc. Next, the emotion estimation unit 336 obtains an emotion learning model from the learning management unit 311. Next, the emotion estimation unit 336 provides the emotion source vector and the emotion learning model to a machine learning prediction processing module and executes the module. The emotion estimation unit 336 then obtains emotion information "E1" for the behavior source vectors from record "ID=1" to record "ID=289", and obtains emotion information "E2" for the behavior source vectors from record "ID=290" to record "ID=N". Finally, the storage unit 338 stores the emotion information and emotion confirmation flag "0" for each record in the behavior management unit 313.

[0412] As a result of the above processing, the behavior and emotion management table shown in Figure 34 is stored in the behavior management unit 313. The behavior and emotion management table has two or more records, each containing an "ID," "behavior information," "behavior confirmation flag," "emotion information," and "emotion confirmation flag."

[0413] Next, the storage unit 338 stores the same action information (e.g., "A1") and the corresponding time (e.g., T) for each record in Figures 33 and 34. 001 ,···,T 002 ,T 289 ) time slot (for example, "T 001 From T 289 The system acquires "TZ1" (which is the time zone) and stores it in association with the time zone and behavioral information.

[0414] Furthermore, the storage unit 338 stores, for each record in Figures 33 and 34, the same emotion information (e.g., "E1") and the corresponding time (e.g., T 001 ,···,T 002 ,T 289 ) time slot (for example, "T 001 From T 289 The system obtains "TZ1"), which is the time zone, and stores it in association with the time zone and emotion information. At this stage, both the action confirmation flag and the emotion confirmation flag corresponding to each time zone are "0". An example of the stored information is shown in Figure 35. Figure 35 is a time zone information management table. The time zone information management table has one or more records that have "ID", "time zone", "action information", "action confirmation flag", "emotion information", and "emotion confirmation flag".

[0415] Then, the component 339 uses one or more sets of time period, behavioral information, and emotional information stored by the storage unit 338 to perform the processing described using the flowchart in Figure 28, to construct time period information for each behavioral information, arrange it in a calendar template, and construct output information.

[0416] Next, the output unit 34 outputs output information. An example of such output is shown in Figure 36. In Figure 36, estimated activity information for each time period on each day in the calendar is displayed.

[0417] Then, if the behavioral information for each time period is correct, user "U1" enters a "confirmation instruction" for the displayed behavioral information, and if the estimated behavioral information is incorrect, user "U1" enters the correct behavioral information. Also, if the emotional information for each time period is correct, user "U1" enters a "confirmation instruction" for the outputted emotional information, and if the estimated emotional information is incorrect, user "U1" enters the correct emotional information. Based on the user's input, the "behavioral information," "behavioral confirmation flag," "emotional information," and "emotional confirmation flag" in Figure 35 will be changed.

[0418] As described above, according to this embodiment, it is possible to estimate a user's behavior using location information corresponding to the time.

[0419] Furthermore, according to this embodiment, the user's behavior can be estimated using location information corresponding to the time and user activity data corresponding to that time.

[0420] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy by using location information corresponding to the time, user activity data corresponding to the time, and user vital data corresponding to the time.

[0421] Furthermore, according to this embodiment, it is possible to estimate the user's emotions at the time of their actions.

[0422] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy using past records.

[0423] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy by using the past records of two or more users.

[0424] Furthermore, according to this embodiment, if the user's actions cannot be estimated, the location where the user was can be output.

[0425] Furthermore, according to this embodiment, even in locations where GPS signals cannot be received, the user's actions can be estimated with high accuracy using the indoor location information obtained by the location information acquisition method described in Embodiment 2.

[0426] The processing in this embodiment may be implemented in software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the information system D in this embodiment is the following program. In other words, this program is a program that causes a computer to function as a time acquisition unit that acquires the time, a location acquisition unit that acquires location information corresponding to the time, an action estimation unit that acquires action information that identifies the user's actions during the time period specified by the time, using two or more action source information including the location information corresponding to the time, and an action output unit that outputs the action information during the time period.

[0427] (Embodiment 4) The difference between this embodiment and Embodiment 3 is as follows: In this embodiment, the behavior acquisition device is a server, and the behavior acquisition device uses location information and the like received from the user's terminal device to estimate the user's behavior information and emotional information.

[0428] Figure 37 is a conceptual diagram of the information system E in this embodiment. The information system E comprises an action acquisition device 5, one or more terminal devices 6, and one or more communication devices B.

[0429] The behavior acquisition device 5 is a server, for example, a cloud server or an ASP server, but the type is not limited. The behavior acquisition device 5 receives behavior source information such as location information from the user's terminal device 6, estimates the user's behavior information using this behavior source information, and transmits this behavior information to the terminal device 6. The behavior acquisition device 5 also receives emotion source information from the user's terminal device 6, estimates the user's emotion information using this emotion source information, and transmits this behavior information to the terminal device 6.

[0430] Terminal device 6 is a device used by the user. Terminal device 6 can be, for example, a smartphone, tablet, smartwatch, or so-called personal computer, and its type is not limited. Terminal device 6 transmits behavior source information, including location information, and emotion source information to the behavior acquisition device 5, and receives and outputs behavior information and emotion information from the behavior acquisition device 5. Note that the device that transmits behavior source information and emotion source information to the behavior acquisition device 5 and the device that receives and outputs behavior information and emotion information from the behavior acquisition device 5 may be different devices.

[0431] Figure 38 is a block diagram of the information system E in this embodiment. Figure 39 is a block diagram of the behavior acquisition device 5.

[0432] The behavior acquisition device 5 comprises a storage unit 51, a receiving unit 52, a processing unit 53, and a transmitting unit 54. The storage unit 51 comprises a learning management unit 311 and a behavior management unit 313. The receiving unit 52 comprises a position acquisition unit 521, an activity acquisition unit 522, and a vital sign acquisition unit 523. The processing unit 53 comprises a time acquisition unit 331, a behavior estimation unit 335, an emotion estimation unit 336, a storage unit 338, and a component unit 339. The transmitting unit 54 comprises a behavior output unit 341 and an emotion output unit 342.

[0433] The terminal device 6 comprises a terminal storage unit 61, a terminal receiving unit 62, a terminal receiving unit 63, a terminal processing unit 64, a terminal transmission unit 65, and a terminal output unit 66. The terminal storage unit 61 comprises a map management unit 312. The terminal processing unit 64 comprises an intensity acquisition unit 231, a type determination unit 232, a location acquisition unit 332, an activity acquisition unit 333, a vital sign acquisition unit 334, and a location acquisition unit 337.

[0434] The storage unit 51, which constitutes the action acquisition device 5, stores various types of information. These types of information include, for example, the learning information and the action information mentioned above.

[0435] The receiving unit 52 receives various instructions and information from the terminal device 6. These instructions and information include, for example, location information, activity data, vital data, output instructions, confirmation instructions, information on actions to be modified, and information on emotions to be modified.

[0436] The receiving unit 52 is preferably able to receive location information, activity data, vital data, etc., from the terminal device 6 all at once. The receiving unit 52 is preferably able to receive location information etc., associated with a user identifier, all at once. The user identifier is information that identifies the user using the terminal device 6. The user identifier may be, for example, a user ID, telephone number, email address, or identifier of the terminal device 6. The identifier of the terminal device 6 may be, for example, an IP address.

[0437] The location acquisition unit 521 receives location information from the terminal device 6. This location information is associated with a time. When the location acquisition unit 521 receives location information, it is preferable to acquire the time from a clock (not shown) and associate that time with the location information. It is preferable that this location information is associated with a user identifier.

[0438] The activity acquisition unit 522 receives activity data from the terminal device 6. This activity data is associated with a time. When the activity acquisition unit 522 receives activity data, it is preferable to acquire the time from a clock (not shown) and associate that time with the activity data. It is preferable that this activity data is associated with a user identifier.

[0439] The vital sign acquisition unit 523 receives one or more types of vital data from the terminal device 6. Such vital data is associated with time. When the vital sign acquisition unit 523 receives vital data, it is preferable to acquire the time from a clock (not shown) and associate that time with the vital data. It is preferable that such vital data is associated with a user identifier.

[0440] The processing unit 53 performs various processes. These processes include, for example, those performed by the time acquisition unit 331, the behavior estimation unit 335, the emotion estimation unit 336, and the storage unit 338.

[0441] The transmitting unit 54 transmits various types of information to the terminal device 6. These types of information include, for example, estimated behavioral information, estimated emotional information, and output information generated by the component unit 339.

[0442] The behavior output unit 341 contains behavior information acquired by the behavior estimation unit 335, and transmits behavior information associated with a time period to the terminal device 6.

[0443] The emotion output unit 342 transmits emotion information, which is acquired by the emotion estimation unit 336 and associated with a specific time period, to the terminal device 6.

[0444] The terminal storage unit 61, which constitutes the terminal device 6, stores various types of information. These types of information include, for example, location information, activity data, and vital data.

[0445] The terminal reception unit 62 receives various instructions and information. These instructions and information include, for example, output instructions, confirmation instructions, behavioral information that the user modifies based on estimated behavioral information, and emotional information that the user modifies based on estimated emotional information.

[0446] Any means of inputting instructions and information is acceptable, such as a touch panel, keyboard, mouse, or menu screen.

[0447] The terminal receiving unit 63 receives various types of information from the behavior acquisition device 5. These types of information include, for example, output information, behavior information, and emotion information.

[0448] The terminal processing unit 64 performs various processes. These processes include, for example, changing instructions and information received by the terminal receiving unit 62 into instructions and information in a structure to be transmitted, and changing the information received by the terminal receiving unit 63 into a structure to be output.

[0449] The terminal transmission unit 65 transmits various instructions and information. These instructions and information include, for example, output instructions, confirmation instructions, information on actions to be changed, and information on emotions to be changed.

[0450] The terminal output unit 66 outputs various types of information. These types of information include, for example, output information, behavioral information, and emotional information.

[0451] The storage unit 51 and the terminal storage unit 61 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0452] The process by which information is stored in the storage unit 51, etc. is not relevant. For example, information may be stored in the storage unit 51, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 51, etc., or information input via an input device may be stored in the storage unit 51, etc.

[0453] The receiving unit 52, location acquisition unit 521, activity acquisition unit 522, vital sign acquisition unit 523, transmitting unit 54, behavior output unit 341, emotion output unit 342, terminal receiving unit 63, and terminal transmitting unit 65 are implemented, for example, by wireless or wired communication means.

[0454] The processing unit 53 and the terminal processing unit 64 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 53, etc., are usually implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0455] The terminal reception unit 62 can be implemented using device drivers for input means such as touch panels and keyboards, or control software for menu screens, etc.

[0456] The terminal output unit 66 may or may not be considered to include output devices such as displays and speakers. The terminal output unit 66 can be implemented using driver software for an output device, or driver software for an output device and an output device.

[0457] Next, an example of the operation of the behavior acquisition device 5 will be explained using the flowchart in Figure 40. In the flowchart of Figure 40, the explanation of the same steps as in Figure 19 will be omitted.

[0458] (Step S4001) The receiving unit 52 determines whether or not it has received location information, etc., paired with a user identifier from the terminal device 6. If location information, etc., has been received, the unit proceeds to step S4002; otherwise, it proceeds to step S4004. Location information, etc., is, for example, a user identifier and location information. Location information, etc., is, for example, one or more types of information from a user identifier and location information, and activity data and vital data.

[0459] (Step S4002) The time acquisition unit 331 acquires the time from a clock (not shown). Here, the time acquisition unit 331 may also acquire the day of the week. The time usually includes hours and minutes. The time may also include one or more pieces of information from the year, month, and day.

[0460] (Step S4003) The storage unit 338 stores the location information and time received in step S4001 in the activity management unit 313, associating them with the user identifier. Return to step S4001.

[0461] (Step S4004) The receiving unit 52 determines whether or not it has received an output instruction from the terminal device 6. If an output instruction is received, the process proceeds to step S4005; otherwise, it proceeds to step S4009. The received output instruction is usually associated with a user identifier. The output instruction also usually includes period information that specifies the period for which behavioral information is to be acquired (for example, "from December 17, 2023 to December 23, 2023").

[0462] (Step S4005) The action estimation unit 335 assigns 1 to counter i.

[0463] (Step S4006) The action estimation unit 335 determines whether the i-th action source information, which is paired with the user identifier associated with the output instruction received in step S4004, exists in the action management unit 313. If the i-th action source information exists, the process proceeds to step S4007; otherwise, the process proceeds to step S1917.

[0464] (Step S4007) The action estimation unit 335 determines whether or not action information exists that corresponds to the i-th action source information. If action information exists, the unit proceeds to step S4008; otherwise, it proceeds to step S1904. Note that if action information exists that corresponds to the action source information, it usually means that the action information has already been estimated using that action source information.

[0465] (Step S4008) The action estimation unit 335 increments counter i by 1. Return to step S4006.

[0466] (Step S4009) The receiving unit 52 determines whether or not it has received information from the terminal device 6. If information is received, the process proceeds to step S1920; otherwise, the process proceeds to step S4010. The information may include, for example, confirmation instructions, information on actions to be modified, or information on emotions to be modified.

[0467] (Step S4010) The receiving unit 52 determines whether or not it has received a learning instruction from the terminal device 6. If a learning instruction is received, the unit proceeds to step S1924; otherwise, it returns to step S4001.

[0468] (Step S4011) The transmitting unit 54 transmits the output information configured in step S1917 to the terminal device 6. Return to step S4001.

[0469] In the flowchart shown in Figure 40, processing is terminated by power-off or processing termination interrupts.

[0470] Next, an example of the operation of terminal device 6 will be explained using the flowchart in Figure 41. In the flowchart of Figure 41, the explanation of the same steps as in Figure 19 will be omitted.

[0471] (Step S4101) The terminal transmission unit 65 obtains the user identifier of the terminal storage unit 61 and transmits the location information and other data obtained in step S1903 to the activity acquisition device 5, associating it with the user identifier.

[0472] (Step S4102) The terminal transmission unit 65 transmits the output instruction received in step S1916 to the action acquisition device 5, associating it with the user identifier of the terminal storage unit 61. The output instruction usually includes time period information.

[0473] (Step S4103) The terminal receiving unit 63 determines whether or not it has received output information from the action acquisition device 5. If output information has been received, the unit proceeds to step S4104; otherwise, it returns to step S4103.

[0474] (Step S4104) The terminal processing unit 64 uses the received output information to configure the output information to be output. The terminal output unit 66 outputs the output information. The process returns to step S1901.

[0475] (Step S4105) The terminal transmission unit 65 transmits the information obtained from the information input received in step S1919 to the action acquisition device 5, associating it with the user identifier of the terminal storage unit 61. Return to step S1901.

[0476] The information includes, for example, confirmation instructions, modified behavioral information, and modified emotional information. Confirmation instructions include information that identifies the behavioral information or emotional information to be confirmed. Modified behavioral information corresponds to information that identifies the behavioral information to be modified. Modified emotional information corresponds to information that identifies the emotional information to be modified.

[0477] In the flowchart shown in Figure 41, processing is terminated by power-off or processing termination interrupts.

[0478] As described above, according to this embodiment, it is possible to estimate a user's behavior using location information corresponding to the time.

[0479] Furthermore, according to this embodiment, the user's behavior can be estimated using location information corresponding to the time and user activity data corresponding to that time.

[0480] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy by using location information corresponding to the time, user activity data corresponding to the time, and user vital data corresponding to the time.

[0481] Furthermore, according to this embodiment, it is possible to estimate the user's emotions at the time of their actions.

[0482] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy using past records.

[0483] Furthermore, according to this embodiment, user behavior can be estimated with greater accuracy by using the past records of two or more users.

[0484] Furthermore, according to this embodiment, if the user's actions cannot be estimated, the location where the user was can be output.

[0485] Furthermore, the software that realizes the action acquisition device 5 in this embodiment is the following program. In other words, this program causes the computer to function as a time acquisition unit that acquires the time, a location acquisition unit that acquires location information corresponding to the time, an action estimation unit that acquires action information that identifies the user's actions during the time period specified by the time, using two or more action source information including the location information corresponding to the time, and an action output unit that outputs the action information during the time period.

[0486] (Embodiment 5) This embodiment describes a behavioral analysis device that acquires and outputs a disorder score, which identifies the degree of disorder in a user's behavior. In particular, this embodiment describes a behavioral analysis device that acquires and outputs a disorder score using the user's time-series behavioral information and reference information. The reference information is, for example, one or more types of information from among information based on the user's past time-series information, a learning model based on time-series behavioral information, recommended behavioral information, and other people's reference information. The reference information may also include, for example, environmental information.

[0487] In this embodiment, a behavioral analysis device that acquires and outputs the factors causing behavioral disturbances will be described.

[0488] In this embodiment, a behavioral analysis device that provides recommendations for improving behavioral disorders will be described.

[0489] In this embodiment, a behavioral analysis device that acquires and outputs the degree of improvement in behavioral disorders will be described. In this embodiment, a behavioral analysis device that acquires and outputs the degree of improvement when the conditions for outputting the degree of improvement are met will be described.

[0490] In this embodiment, a behavioral analysis device that acquires and outputs the recovery period from behavioral disturbances will be described.

[0491] In this embodiment, a behavioral analysis device that acquires and outputs long-term disorder scores will be described.

[0492] Figure 42 is a conceptual diagram of the information system F in this embodiment. The information system F comprises a behavioral analysis device 7, one or more terminal devices 8, and two or more communication devices B.

[0493] The behavioral analysis device 7 is a device that acquires one or more pieces of information from the following: disturbance score, disturbance factors, recommendation information, improvement level, recovery period, and long-term disturbance score, and transmits them to the terminal device 8. Furthermore, the behavioral analysis device 7 can, for example, receive source behavior information such as location information from the user's terminal device 8, use this source behavior information to estimate the user's behavior information, and transmit this behavior information to the terminal device 8. The behavioral analysis device 7 can also receive source emotion information from the user's terminal device 8, use this source emotion information to estimate the user's emotion information, and transmit this behavior information to the terminal device 8. The behavioral analysis device 7 is typically a server, such as a cloud server or an ASP server, but its type is not limited.

[0494] However, the behavior analysis device 7 may also be a terminal device. In this case, the behavior analysis device 7 may have all or part of the functions of terminal device 2 or behavior acquisition device 3. In this case, the behavior analysis device 7 may be, for example, a smartphone, tablet device, smartwatch, or so-called personal computer, and the type is not limited.

[0495] Terminal device 8 is a device used by the user. Terminal device 8 can be, for example, a smartphone, tablet, smartwatch, or so-called personal computer, and the type is not limited. Terminal device 8 is a device that transmits behavioral source information, including location information, and emotional source information to the behavioral analysis device 7, and receives and outputs disturbance score, disturbance factors, recommendation information, improvement degree, recovery period, long-term disturbance score, behavioral information, or emotional information from the behavioral analysis device 7. Note that the device that transmits behavioral source information and emotional source information to the behavioral analysis device 7 and the device that receives and outputs disturbance score, disturbance factors, etc. from the behavioral analysis device 7 may be different devices.

[0496] Figure 43 is a block diagram of the information system F in this embodiment. Figure 44 is a block diagram of the behavioral analysis device 7.

[0497] The behavior analysis device 7 comprises a storage unit 71, a reception unit 72, a processing unit 73, and an output unit 74. The behavior analysis device 7 may comprise all or some of the components of the behavior acquisition device 5. If the behavior analysis device 7 is a standalone device, it may comprise all or some of the components of the behavior acquisition device 3 or the terminal device 2.

[0498] The storage unit 71 includes a learning management unit 311, an action management unit 313, a standard management unit 711, and a recommendation management unit 712. The reception unit 72 includes a location acquisition unit 521, an activity acquisition unit 522, and a vital sign acquisition unit 523. The processing unit 73 includes a time acquisition unit 331, an action estimation unit 335, an emotion estimation unit 336, a storage unit 338, a configuration unit 339, a learning unit 731, a score acquisition unit 732, a long-term score acquisition unit 733, a judgment unit 734, a factor acquisition unit 735, a recommendation acquisition unit 736, an improvement degree acquisition unit 737, and a recovery period acquisition unit 738. The output unit 74 includes an action output unit 341, an emotion output unit 342, a score output unit 741, a long-term score output unit 742, a factor output unit 743, a recommendation output unit 744, an improvement degree output unit 745, and a recovery period output unit 746.

[0499] The terminal device 8 comprises a terminal storage unit 61, a terminal receiving unit 62, a terminal receiving unit 83, a terminal processing unit 64, a terminal transmission unit 65, and a terminal output unit 86. The terminal storage unit 61 comprises a map management unit 312. The terminal processing unit 64 comprises an intensity acquisition unit 231, a type determination unit 232, a location acquisition unit 332, an activity acquisition unit 333, a vital sign acquisition unit 334, and a location acquisition unit 337.

[0500] The storage unit 71, which constitutes the behavioral analysis device 7, stores various types of information. These types of information include, for example, the learning information mentioned above, the behavioral information mentioned above, the reference information described later, the recommendation information described later, the learning model described later, and various conditions described later. It goes without saying that these various conditions may also be embedded within the program.

[0501] The standard management unit 711 stores one or more standard information entries. The standard information may be managed separately for each of the two or more users. For example, the standard information may be associated with a user identifier.

[0502] Reference information refers to the information used as a baseline when obtaining a disorder score, which is the degree of disorder in a user's behavior. Note that disorder in behavior can also be described as disorder in daily life. Reference information is the source information for obtaining the degree of disorder in a user's behavior, corresponding to time-series information. Reference information may include, for example, one or more types of information such as self-reference information, learning models, recommended behavior information, and peer-reference information. Reference information may also include, for example, environmental reference information.

[0503] A disorder score is information that identifies the degree of disorder in behavior. Disorder in behavior can also be described as disorder in lifestyle. Disorder scores can be expressed on a scale of 0 to 100, a 5-point scale, a 10-point scale, etc. A disorder score can also be expressed on a 2-point scale (disordered or not). A disorder score can also represent the degree of well-being of one's lifestyle (degree of disorder).

[0504] Furthermore, the disorder score doesn't necessarily have to represent the degree of disorder, but rather the degree of lack of disorder. The degree of lack of disorder can be said to represent the degree of quality of life.

[0505] The following provides a detailed description of examples of standard information, including self-standard information, learning models, recommended behavior information, external standard information, and environmental standard information. (1) Self-standard information

[0506] Self-reference information is information based on a user's past time-series data. Reference information is, for example, a set of average (normal) behavioral information and time information for a user over a predetermined period (e.g., one day, one week). Reference information is, for example, a set of behavioral information and time information for a user over a predetermined period without any irregularities. The behavioral information contained in the reference information may correspond to the time (length) during which the behavior occurred. The behavioral information contained in the reference information includes, for example, the start time and end time of the behavior. Reference information is, for example, a vector. The vector is, for example, (time of behavioral information 1, start time of behavioral information 1, end time of behavioral information 1, time of behavioral information 2, start time of behavioral information 2, end time of behavioral information 2, ..., time of behavioral information n, start time of behavioral information n, end time of behavioral information n). The vector is, for example, (time-related information of behavioral information 1, time-related information of behavioral information 2, ..., time-related information of behavioral information n). Time-related information consists of one or more pieces of information from the following: time, start time, or end time. "Activity information 1," "Activity information 2," "Activity information 3," ... "Activity information n" are examples of "sleep," "breakfast," "travel," "work," "shopping," "dinner," "drinking party," and "game." (2) Learning Model

[0507] The learning model is a model obtained by performing a machine learning learning process using two or more training data sets. Here, the training data has explanatory variables based on the past time-series information of one or more users and a target variable which is disorder information regarding the degree of disorder in the user's behavior. The two or more training data sets are data for creating a learning model used for one user, and may be information based on the time-series information of that one user. Alternatively, the two or more training data sets are data for creating a learning model used for two or more users, and may be information based on the time-series information of one or more users, including other people. The learning model is usually a model created by the learning unit 731.

[0508] Disruption information refers to information about disruptions in a user's behavior. Disruption information includes, for example, a disruption score, information indicating whether or not there is disruption, information indicating the level of physical well-being, information indicating whether or not the physical well-being is good, information indicating the level of mental well-being, information indicating whether or not the mental well-being is good, information indicating the overall level of physical and mental well-being, and information indicating whether or not the overall physical and mental well-being is good. Disruption information is, for example, information entered by the user. Disruption information is, for example, a score obtained from one or more types of vital data of the user (e.g., heart rate, heart rate variability, blood pressure (systolic and / or diastolic), respiratory rate per unit time, body temperature) or information indicating whether or not the physical well-being is good. The technology for obtaining a score or information indicating whether or not the physical well-being is good from vital data is publicly known technology. Disruption information may also be information entered by the user in correspondence with time-series information.

[0509] A learning model is information constructed through the learning process of machine learning and is used in the prediction process of machine learning. A learning model can also be called a learner, classifier, or classification model. The machine learning algorithm can be deep learning, random forest, decision tree, SVR, SVM, etc. Furthermore, various machine learning functions and existing libraries can be used in machine learning, such as the TensorFlow® library, the R language's random forest module, fastText, and TinySVM.

[0510] Furthermore, a learning model based on the time-series data of a single user can be considered an example of self-referenced information. A learning model based on the time-series data of one or more users, including other users, can be considered an example of other-referenced information. (3) Recommended Actions

[0511] Recommended behavior information is information that identifies recommended behaviors. For example, recommended behavior information could be information that identifies common sense that is less likely to cause behavioral disorders, or information that identifies common sense that is more likely to cause behavioral disorders. Examples of recommended behavior information that identifies recommended behaviors include "number of meals = 3 times / day", "6 hours <= sleep time <= 10 hours", "8 PM <= bedtime <= midnight", and "5 AM <= wake-up time <= 9 AM". Examples of recommended behavior information that identifies unrecommended behaviors include "number of drinking parties >= 3 times / week", "drinking parties for 3 consecutive days", and "game time >= 3 hours / day". Recommended behavior information may also identify unrecommended behaviors. However, in this specification, recommended behavior information is generally described as information that identifies recommended behaviors. (4) Information based on other people's standards

[0512] External reference information is information based on the behavioral information of one or more other individuals other than the user. External reference information is usually information based on the behavioral information of two or more individuals. External reference information may also include self-reference information. External reference information is, for example, a collection of average behavioral information and time information of two or more users over a given period (e.g., one day, one week). The data structure of external reference information is, for example, the same as the data structure of self-reference information.

[0513] It is preferable that the standards management unit 711 stores other-party standards information corresponding to two or more user attribute value conditions. (5) Environmental standard information

[0514] Environmental standard information refers to standard environmental information. Environmental information is information that identifies the environment of the place of activity. Environmental information includes, for example, information that identifies the weather, temperature, humidity, and amount of ultraviolet radiation. Environmental standard information includes, for example, "sunny" for appropriate weather, "15 degrees Celsius <= temperature <= 25 degrees Celsius" for appropriate temperature range, "40% <= humidity <= 60%" for appropriate humidity range, and an appropriate amount of ultraviolet radiation.

[0515] The recommendation management unit 712 stores recommendation source information associated with one or more factor conditions. Factor conditions are conditions related to disruptive factors. Factor conditions may be the same as disruptive conditions. Disruptive factors are factors that disrupt behavior. Examples of disruptive factors include "sleep" (e.g., short sleep duration, late bedtime, late wake-up time), "breakfast" (e.g., late breakfast time, skipping breakfast), and "drinking parties" (frequent drinking parties).

[0516] Recommendation source information is the information that forms the basis of recommendation information. Recommendation source information is the information used to construct recommendation information. Recommendation source information may have, for example, one or more variables. Variables may contain, for example, disruption factors and element information corresponding to those disruption factors. Recommendation source information may also be recommendation information itself. Recommendation information is information recommended to the user in order to improve the disruption of the user's behavior. Recommendation information may be, for example, text, audio, still images, or videos, and its data type is not restricted.

[0517] The reception unit 72 receives one or more time-series information. The method by which the reception unit 72 receives the time-series information is irrelevant; the reception unit 72 only needs to acquire the time-series information.

[0518] Time-series information refers to a user's actions over time. Time-series information includes two or more time-series action records. Action records are information that identifies a user's actions. Action records are, for example, associated with time information. Time information identifies when a user performed an action. Time information includes, for example, a start time and an end time. Time information also includes, for example, time.

[0519] The reception unit 72 receives, for example, behavioral information for two or more time periods acquired by the behavior estimation unit 335.

[0520] Here, "reception" usually refers to the reception of information transmitted via wired or wireless communication lines, but it may also be a concept that includes the reception of information input from input devices such as keyboards, mice, and touch panels, as well as the reception of information read from recording media such as optical discs, magnetic discs, and semiconductor memory.

[0521] The processing unit 73 performs various processes. These processes include, for example, those performed by the time acquisition unit 331, the behavior estimation unit 335, the emotion estimation unit 336, the storage unit 338, the configuration unit 339, the learning unit 731, the score acquisition unit 732, the long-term score acquisition unit 733, the judgment unit 734, the factor acquisition unit 735, the recommendation acquisition unit 736, the improvement degree acquisition unit 737, and the recovery period acquisition unit 738.

[0522] The learning unit 731 creates a learning model using two or more training data sets and stores it in the standard management unit 711. The learning unit 731 creates a learning model using two or more training data sets through machine learning training. The training data here consists of explanatory variables based on the past time-series information of one or more users and a target variable which is disorder information relating to the degree of disorder in the user's behavior.

[0523] The learning unit 731 provides, for example, two or more training data sets for each of one or more users, each containing explanatory variables based on the user's past time-series information and a target variable which is disorder information relating to the degree of disorder in the user's behavior, to a module that performs machine learning processing, executes the module, and obtains a learning model. As mentioned above, the machine learning algorithm is not limited.

[0524] The score acquisition unit 732 uses the time-series information received by the reception unit 72 and the standard information from the standard management unit 711 to acquire a disorder score, which is the degree of disorder in the user's behavior. The score acquisition unit 732 usually acquires a disorder score, which is a score related to the difference between the time-series information received by the reception unit 72 and the standard information from the standard management unit 711.

[0525] The score acquisition unit 732 may acquire a first disturbance score and a second disturbance score, which are disturbance scores for each of the two time periods. The score acquisition unit 732 may acquire disturbance scores for two or more time series information.

[0526] The following describes examples of processing by the score acquisition unit 732 for each case where the reference information is self-reference information, a learning model, recommended behavior information, and external reference information. (1) Self-standard information

[0527] The score acquisition unit 732 acquires difference information regarding the difference between the time-series information for a predetermined period (e.g., one day, one week) received by the reception unit 72 and the self-reference information for the predetermined period, and acquires a larger disorder score the larger the difference information. (1-1) When the self-reference information is a vector (self-reference vector)

[0528] The score acquisition unit 732 obtains a vector from the time-series information for a predetermined period (e.g., one day, one week) received by the reception unit 72. This vector is called the test vector. Next, the score acquisition unit 732 obtains a larger disorder score the greater the difference between the test vector and the self-reference vector.

[0529] For example, the score acquisition unit 732 acquires the distance between the test vector and the self-reference vector. Next, the score acquisition unit 732 acquires a disorder score using this distance, or an increasing function that uses this distance as a parameter.

[0530] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the acquired test vector and each element of the self-reference vector, element by element, and for each element, if the difference between elements is equal to or greater than a threshold, it counts up the disorder score to obtain the final disorder score.

[0531] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the test vector and each element of the self-reference vector, element by element, and obtains a disorder score which is the sum of the absolute values ​​of the differences for each element. (1-2) When self-reference information is a set of pairs of information consisting of behavioral information and temporal information.

[0532] The score acquisition unit 732 determines whether or not each action information contained in the time-series information for a predetermined period received by the reception unit 72 exists in the self-reference information.

[0533] For example, if there is behavioral information that is not present in the self-reference information, the score acquisition unit 732 counts up the disorder score. Behavioral information that is not present in the self-reference information may be, for example, information about actions that are not normally performed, and can be a factor in behavioral disorder.

[0534] Furthermore, the score acquisition unit 732 acquires the difference between the time information that corresponds to the behavioral information contained in the time-series information for a predetermined period received by the reception unit 72 and the time information that corresponds to the behavioral information in the self-reference information (for example, one or more pieces of information from the difference in duration, difference in start time, and difference in end time). The score acquisition unit 732 acquires a larger disorder score the larger such difference is.

[0535] Furthermore, the score acquisition unit 732 acquires the frequency of behavioral information (e.g., "eating" and "drinking party") contained in the time-series information for a predetermined period received by the reception unit 72. The score acquisition unit 732 acquires the frequency of behavioral information in the self-reference information. Next, if the difference between the two frequencies is equal to or greater than a threshold, the score acquisition unit 732 acquires a large disturbance score. (2) Learning Model

[0536] The score acquisition unit 732 uses the time-series information received by the reception unit 72 and the learning model to perform machine learning prediction processing and acquire a disorder score.

[0537] For example, the score acquisition unit 732 constructs a test vector from the time-series information received by the reception unit 72. The test vector here can be, for example, (time of action information 1, start time of action information 1, end time of action information 1, time of action information 2, start time of action information 2, end time of action information 2, ..., time of action information n, start time of action information n, end time of action information n), or (time-related information of action information 1, time-related information of action information 2, ..., time-related information of action information n), but the structure is not limited.

[0538] Next, the score acquisition unit 732 provides the test vector and the learning model to a machine learning prediction processing module, executes the module, and obtains a disorder score. Alternatively, the score acquisition unit 732 may provide the vector and the learning model to a machine learning prediction processing module, execute the module, obtain information on whether or not the data is disordered, and if it obtains the target variable to be "disordered", it may obtain the score returned by the prediction processing module and obtain a disorder score based on that score. Such a disorder score is the score returned by the module, or a score obtained by an augmentation function that takes the score returned by the module as a parameter. For example, if the score acquisition unit 732 obtains the target variable to be "not disordered", it obtains a disorder score of "0".

[0539] Furthermore, the module that performs the machine learning prediction process may be a module that returns a randomness score, or a module that returns the predicted value that forms the basis of the randomness score.

[0540] Furthermore, the machine learning prediction algorithm can be any type, such as deep learning, random forest, decision tree, or SVM. However, a random forest is preferable for the machine learning prediction algorithm. This is because a random forest allows us to obtain the influence of each explanatory variable on the output target variable. The behavioral information corresponding to the explanatory variables with a large influence constitutes the disorder factors described later. Explanatory variables with a large influence are, for example, those with the highest influence rank (1st), those with a rank of N or higher, or those with an influence above or greater than the threshold. (3) Recommended Actions (3-1) When recommended behavior information is a vector (recommended behavior vector)

[0541] The score acquisition unit 732 obtains a test vector from the time-series information (for example, one day, one week) received by the reception unit 72. Next, the score acquisition unit 732 obtains a larger disorder score the greater the difference between the test vector and the recommended behavior vector.

[0542] For example, the score acquisition unit 732 acquires the distance between the test vector and the recommended action vector. Next, the score acquisition unit 732 acquires a disorder score using this distance, or an increasing function that uses this distance as a parameter.

[0543] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the test vector and each element of the recommended action vector, element by element, and for each element, if the difference between elements is equal to or greater than a threshold, it counts up the disorder score to obtain the final disorder score.

[0544] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the test vector and each element of the recommended action vector, element by element, and obtains a disorder score which is the sum of the absolute values ​​of the differences for each element. (3-2) When recommended behavioral information is a set of pairs of behavioral information and time information.

[0545] For example, the score acquisition unit 732 determines whether the behavioral information contained in the time-series information for a predetermined period received by the reception unit 72 is present in the recommended behavioral information. If the recommended behavioral information is information that identifies a recommended behavior, and the behavioral information is not present in the recommended behavioral information, the score acquisition unit 732 acquires a larger disorder score than if the behavioral information were present in the recommended behavioral information. If the recommended behavioral information is information that identifies a non-recommended behavior, and the behavioral information is present in the recommended behavioral information, the score acquisition unit 732 acquires a larger disorder score than if the behavioral information is not present in the recommended behavioral information.

[0546] For example, the score acquisition unit 732 acquires time information (e.g., time, start time, end time) that is paired with the behavior information contained in the time-series information for a predetermined period received by the reception unit 72. For each piece of behavior information, the score acquisition unit 732 determines whether the time information paired with the behavior information satisfies the conditions of the time information paired with the behavior information contained in the recommended behavior information (e.g., a suitable range for sleep duration). If the conditions of the time information paired with the behavior information contained in the recommended behavior information are not met, the score acquisition unit 732 acquires a larger disorder score than when the conditions are met. (4) Information based on other people's standards (4-1) When the information based on other people is a vector (vector based on other people)

[0547] The score acquisition unit 732 obtains a test vector from the time-series information (for example, one day, one week) received by the reception unit 72. Next, the score acquisition unit 732 obtains a larger disorder score the greater the difference between the test vector and the other-person reference vector.

[0548] For example, the score acquisition unit 732 acquires the distance between the test vector and the other reference vector. Next, the score acquisition unit 732 acquires a disorder score using this distance, or an increasing function that uses this distance as a parameter.

[0549] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the test vector and each element of the other reference vector, element by element, and for each element, if the difference between elements is equal to or greater than a threshold, it counts up the disorder score to obtain the final disorder score.

[0550] Furthermore, for example, the score acquisition unit 732 acquires the difference between each element of the test vector and each element of the other reference vector, element by element, and obtains a disorder score which is the sum of the absolute values ​​of the differences for each element. (4-2) When the information based on other people is a set of pairs of information consisting of behavioral information and temporal information.

[0551] The score acquisition unit 732 determines whether or not each action information contained in the time-series information for a predetermined period received by the reception unit 72 exists in the other party's reference information.

[0552] If there is behavioral information that is not present in the other-person reference information, the score acquisition unit 732 counts up the disorder score. Behavioral information that is not present in the other-person reference information is information about actions that two or more people do not normally perform.

[0553] Furthermore, the score acquisition unit 732 acquires the difference between the time information that corresponds to the behavioral information contained in the time-series information for a predetermined period received by the reception unit 72 and the time information that corresponds to the behavioral information present in the other party's reference information (for example, one or more pieces of information from the difference in duration, difference in start time, and difference in end time). The score acquisition unit 732 acquires a larger disorder score the larger such difference is.

[0554] Furthermore, the score acquisition unit 732 acquires the frequency of behavioral information (e.g., "eating" and "drinking party") contained in the time-series information for a predetermined period received by the reception unit 72. The score acquisition unit 732 acquires the frequency of behavioral information in the other-person reference information. Next, if the difference between the two frequencies is equal to or greater than a threshold, the score acquisition unit 732 acquires a large disturbance score.

[0555] Furthermore, the score acquisition unit 732 may, for example, acquire the results of a comparison of scores over two or more periods and acquire a score based on those comparison results. The score acquisition unit 732 may, for example, acquire the change in score (e.g., difference, percentage) between the same month of the previous year (June 2023 and June 2024) or one week of the previous month, and acquire a score using a function that takes that change as a parameter. Such a score is called the change score, and the greater the degree of improvement in the score, the larger the change score acquired.

[0556] The long-term score acquisition unit 733 uses two or more disturbance scores acquired by the score acquisition unit 732 to acquire a long-term disturbance score that identifies the degree of long-term behavioral disturbance. Here, "long-term" refers to periods such as 3 months (seasonally), 1 year, or 1 month. A long-term period is longer than one day. A long-term period is usually one week or longer.

[0557] The long-term score acquisition unit 733 obtains the number of times the judgment unit 734 has determined that the disorder condition is met, and uses this number to acquire a long-term disorder score. The longer the number of times the disorder condition is determined to be met, the larger the long-term disorder score the unit 733 acquires.

[0558] The determination unit 734 determines whether the disorder score obtained by the score acquisition unit 732 satisfies the disorder condition. The disorder condition is a condition for obtaining the disorder factor. For example, the disorder condition is that the disorder score is equal to or greater than a threshold.

[0559] Disruption factors are information about behavioral information that causes disruption. Disruption factors usually include behavioral information and time information. Disruption factors are information obtained using behavioral information corresponding to explanatory variables with a large influence obtained when prediction processing is performed using a random forest. Disruption factors are information obtained using behavioral information that is above or greater than a threshold for difference information that identifies the difference from baseline information. Disruption factors may be information about only specific behavioral information (e.g., "sleep," "drinking party," "eating"). Disruption factors may be information about behavioral information excluding specific behavioral information (e.g., "travel").

[0560] The determination unit 734 determines whether the first disturbance score and the second disturbance score satisfy the recovery conditions. The recovery conditions are the conditions for obtaining the recovery period. Examples of recovery conditions include: no longer meeting the disturbance conditions; no longer meeting the disturbance conditions after having met them; the degree of improvement is above or greater than the threshold; the degree of improvement is above or greater than the threshold and the second disturbance score is below or less than the threshold; and the second disturbance score is below or less than the threshold.

[0561] The determination unit 734 determines whether each of the two or more disturbance scores satisfies the disturbance condition. The disturbance condition is the condition for the factor acquisition unit 735 to acquire the disturbance factor. For example, the disturbance condition is that the disturbance score is equal to or greater than a threshold.

[0562] The factor acquisition unit 735 is time-series information corresponding to the disturbance score, and acquires disturbance factors related to the behavioral information that causes the disturbance from the behavioral information contained in the time-series information received by the reception unit 72. It is preferable for the factor acquisition unit 735 to acquire disturbance factors when the judgment unit 734 determines that the disturbance conditions are met.

[0563] The factor acquisition unit 735 acquires behavioral information corresponding to explanatory variables with a large influence obtained as a result of prediction processing by random forest, and acquires disorder factors having said behavioral information and time information paired with said behavioral information.

[0564] The factor acquisition unit 735 acquires behavioral information that caused the disorder score acquired by the score acquisition unit 732 to increase to the extent that it satisfies the adoption conditions. The factor acquisition unit 735 acquires disorder factors that include the behavioral information and the time information that is paired with the behavioral information. The adoption conditions are the conditions for behavioral information to be adopted as a disorder factor. For example, the adoption conditions are that the number of times the disorder score increases is above a threshold. For example, the adoption conditions are that the number of occurrences of a specific behavioral piece of information to be adopted as a disorder factor during a predetermined period is above a threshold (for example, the number of drinking parties is 3 or more per week).

[0565] The recommendation acquisition unit 736 refers to the recommendation management unit 712 and obtains recommendation source information from the recommendation management unit 712 that corresponds to the factor conditions that match the disruption factors obtained by the factor acquisition unit 735. The recommendation acquisition unit 736 substitutes the disruption factors into the recommendation information, which is the recommendation source information, or into the variables of the recommendation source information, and obtains the recommendation information.

[0566] The improvement degree acquisition unit 737 acquires the improvement degree using the first disturbance score and the second disturbance score. The first disturbance score and the second disturbance score are disturbance scores acquired by the score acquisition unit 732 using time-series information from different points in time.

[0567] The degree of improvement refers to information that identifies the extent to which a disturbance has improved. The degree of improvement is usually information about the difference between the first disturbance score and the second disturbance score. For example, if the disturbance score indicates the degree of disturbance, then "degree of improvement = first disturbance score - second disturbance score," and if the disturbance score indicates the quality of the behavior, then "degree of improvement = second disturbance score - first disturbance score."

[0568] The improvement level acquisition unit 737 preferably acquires the improvement level when the first disturbance score and the second disturbance score satisfy the improvement level output conditions. The improvement level output conditions are, for example, that the improvement level is equal to or greater than a threshold, or that the improvement level is equal to or less than a threshold.

[0569] The recovery period acquisition unit 738 acquires the recovery period, which is the difference between the first time information corresponding to the first disturbance score and the second time information corresponding to the second disturbance score. The recovery period acquisition unit 738 preferably acquires the recovery period when the determination unit 734 determines that the recovery condition is met. The recovery condition is the condition for acquiring the recovery period. For example, the recovery condition is that the second disturbance score does not satisfy the disturbance condition.

[0570] The output unit 74 outputs various types of information. These types of information include, for example, a disturbance score, a long-term disturbance score, behavioral information, emotional information, or location information.

[0571] Here, "output" usually refers to output to the terminal device 8, but it may also be a concept that includes display on a screen, projection using a projector, printing with a printer, sound output, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0572] The score output unit 741 outputs the disordered score acquired by the score acquisition unit 732. The method of outputting the disordered score is not limited. It is preferable for the score output unit 741 to output the disordered score in association with a user identifier. It is preferable for the score output unit 741 to output the disordered score for each predetermined period (e.g., one week, one day).

[0573] The long-term score output unit 742 outputs the long-term disorder score acquired by the long-term score acquisition unit 733. The output method of the long-term disorder score is not limited. It is preferable for the long-term score output unit 742 to output the long-term disorder score in association with a user identifier.

[0574] The factor output unit 743 outputs one or more disturbance factors acquired by the factor acquisition unit 735. It is preferable for the factor output unit 743 to output one or more disturbance factors in association with time series information.

[0575] The recommendation output unit 744 outputs one or more recommendation pieces of information acquired by the recommendation acquisition unit 736.

[0576] The improvement level output unit 745 outputs the improvement level acquired by the improvement level acquisition unit 737.

[0577] The recovery period output unit 746 outputs the recovery period acquired by the recovery period acquisition unit 738.

[0578] The terminal receiving unit 83, which constitutes the terminal device 8, receives various types of information. The terminal receiving unit 83 receives various types of information from the behavioral analysis device 7. These types of information include, for example, behavioral information, emotional information, disturbance score, long-term disturbance score, disturbance factors, recommendation information, degree of improvement, or recovery period.

[0579] The terminal output unit 86 outputs various types of information. These types of information include, for example, behavioral information, emotional information, disturbance score, long-term disturbance score, disturbance factors, recommendation information, degree of improvement, or recovery period.

[0580] The storage unit 71, learning management unit 311, reference management unit 711, and recommendation management unit 712 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0581] The process by which information is stored in the storage unit 71, etc. is not relevant. For example, information may be stored in the storage unit 71, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 71, etc., or information input via an input device may be stored in the storage unit 71, etc.

[0582] The reception unit 72 is preferably implemented by wireless or wired communication means, but may also be implemented by means of receiving broadcasts, device drivers for input means such as touch panels and keyboards, or control software for menu screens.

[0583] The processing unit 73, learning unit 731, score acquisition unit 732, long-term score acquisition unit 733, judgment unit 734, factor acquisition unit 735, recommendation acquisition unit 736, improvement degree acquisition unit 737, and recovery period acquisition unit 738 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 73, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0584] The output unit 74, which includes the behavior output unit 341, emotion output unit 342, score output unit 741, long-term score output unit 742, factor output unit 743, recommendation output unit 744, improvement level output unit 745, and recovery period output unit 746, is preferably implemented by wireless or wired communication means, but may also be implemented by driver software for an output device such as a display or speaker, or by driver software for an output device and an output device.

[0585] The terminal receiving unit 83 is usually implemented by wireless or wired communication means, but it may also be implemented by means of receiving broadcasts.

[0586] The terminal output unit 86 may or may not be considered to include output devices such as a display or speakers. The terminal output unit 86 can be implemented using driver software for an output device, or a driver software for an output device and an output device.

[0587] Next, an example of the operation of the behavior analysis device 7 will be explained using the flowchart in Figure 45. The flowchart in Figure 45 does not include the same processes as those of the behavior acquisition device 5. The processes that are the same as those of the behavior acquisition device 5 are the processes shown in the flowchart in Figure 40.

[0588] (Step S4501) The processing unit 73 determines whether the score acquisition conditions are met. If the score acquisition conditions are met, the process proceeds to step S4502; otherwise, the process proceeds to step S4519.

[0589] The score acquisition conditions are the conditions for acquiring the random score. These conditions include, for example, that a predetermined time (for example, 24:00 every day) has arrived, that the reception unit 72 has received a score acquisition instruction from the user, and that the reception unit 72 has received time-series information. The score acquisition instruction received by the reception unit 72 includes a user identifier. The time-series information received by the reception unit 72 is associated with the user identifier.

[0590] (Step S4502) The processing unit 73 assigns 1 to counter i.

[0591] (Step S4503) The processing unit 73 determines whether or not there is an i-th user from whom to obtain the disorder score. If there is an i-th user, the unit proceeds to step S4504; otherwise, it returns to step S4501.

[0592] (Step S4504) The score acquisition unit 732 acquires the disorder score of the i-th user. An example of such score acquisition process will be explained using the flowcharts in Figures 46, 47, and 48.

[0593] (Step S4505) The score output unit 741 associates the i-th user identifier with the time-series information used to acquire the disorder score and stores the disorder score acquired in step S4504 in the behavior management unit 313.

[0594] (Step S4506) The determination unit 734 determines whether the disorder score obtained in step S4504 matches the disorder condition. If it matches the disorder condition, the process proceeds to step S4507; otherwise, the process proceeds to step S4512.

[0595] (Step S4507) The factor acquisition unit 735 acquires the factors causing the disruption in the user's behavior. An example of this factor acquisition process will be explained using the flowchart in Figure 49.

[0596] (Step S4508) The recommendation acquisition unit 736 acquires recommendation information. An example of such recommendation acquisition process will be explained using the flowchart in Figure 50.

[0597] (Step S4509) The processing unit 73 or a component not shown constitutes output information. Output information is information that is output. The output information has a disturbance score. The output information here has, for example, one or more disturbance factors and one or more recommendation information.

[0598] (Step S4510) The output unit 74 outputs the output information configured in step S4509. The output unit 74 transmits the output information to, for example, the i-th user.

[0599] (Step S4511) The processing unit 73 increments counter i by 1. The process returns to step S4503.

[0600] (Step S4512) The improvement level acquisition unit 737 acquires the past disorder score of the i-th user. Preferably, the past disorder score is the most recent disorder score.

[0601] (Step S4513) The improvement level acquisition unit 737 determines whether the disorder score acquired in step S4504 and the disorder score acquired in step S4512 satisfy the improvement level output conditions. If the improvement level output conditions are met, the process proceeds to step S4514; otherwise, the process proceeds to step S4515.

[0602] (Step S4514) The improvement level acquisition unit 737 acquires the improvement level using the disorder score acquired in step S4504 and the disorder score acquired in step S4512. The improvement level output unit 745 associates the i-th user identifier with the time-series information used when acquiring the disorder score and stores the improvement level in the behavior management unit 313.

[0603] (Step S4515) The determination unit 734 determines whether the disorder score obtained in step S4504 and the disorder score obtained in step S4512 satisfy the recovery condition. If the recovery condition is satisfied, the process proceeds to step S4516; otherwise, the process proceeds to step S4517.

[0604] (Step S4516) The recovery period acquisition unit 738 acquires the recovery period, which is the difference between the time information corresponding to the disorder score acquired in step S4504 and the time information corresponding to the disorder score acquired in step S4512. The recovery period output unit 746 associates the i-th user identifier with the time-series information used when acquiring the disorder score and stores the recovery period in the behavior management unit 313.

[0605] (Step S4517) The processing unit 73 or a component not shown constitutes output information. The output information includes a disturbance score. The output information here includes, for example, the degree of improvement and the recovery period.

[0606] (Step S4518) The output unit 74 outputs the output information configured in step S4517. The output unit 74 transmits the output information to, for example, the i-th user.

[0607] (Step S4519) The processing unit 73 determines whether the conditions for acquiring a long-term score are met. If the conditions for acquiring a long-term score are met, the process proceeds to step S4520; otherwise, the process proceeds to step S4525. The conditions for acquiring a long-term score include, for example, the accumulation of disorder scores over a predetermined period, and the acceptance of instructions from the user.

[0608] (Step S4520) The processing unit 73 assigns 1 to counter i.

[0609] (Step S4521) The processing unit 73 determines whether or not there is an i-th user from whom to obtain a long-term disorder score. If there is an i-th user, the process goes to step S4522; otherwise, it returns to step S4501.

[0610] (Step S4522) The long-term score acquisition unit 733 acquires the long-term disorder score of the i-th user. An example of such long-term score acquisition process will be explained using the flowchart in Figure 51.

[0611] (Step S4523) The long-term score output unit 742 stores the long-term disturbance score obtained in step S4522 in the behavior management unit 313, associating it with the i-th user.

[0612] (Step S4524) The processing unit 73 increments counter i by 1. The process returns to step S4503.

[0613] (Step S4525) The processing unit 73 determines whether the learning conditions are met. If the learning conditions are met, the process proceeds to step S4526; otherwise, it returns to step S4501. The learning conditions are the conditions for creating a learning model. Examples of learning conditions include the number of time-series information data for the user exceeding a threshold, and the acceptance of instructions from the user.

[0614] (Step S4526) The processing unit 73 assigns 1 to counter i.

[0615] (Step S4527) The processing unit 73 determines whether or not there is an i-th user to create the learning model. If there is an i-th user, the process goes to step S4528; otherwise, it returns to step S4521.

[0616] (Step S4528) The learning unit 731 performs a learning process using the time-series information of the i-th user and obtains a learning model. An example of such a learning process will be explained using the flowchart in Figure 52.

[0617] (Step S4529) The output unit 74 associates the learning model acquired in step S4528 with the i-th user and stores it in the reference management unit 711.

[0618] (Step S4530) The processing unit 73 increments counter i by 1. The process returns to step S4527.

[0619] In the flowchart shown in Figure 45, processing is terminated by power-off or processing termination interrupts.

[0620] Next, we will explain the first example of the score acquisition process in step S4504 using the flowchart in Figure 46. The first example of the score acquisition process is the case in which a disorder score is obtained based on the similarity between the inspection vector, which is based on the user's time-series information, and the reference information, which is a vector.

[0621] (Step S4601) The score acquisition unit 732 acquires time-series information of the target user. This time-series information includes, for example, time-series information paired with the user identifier of the target user, and time-series information received by the reception unit 72.

[0622] (Step S4602) The score acquisition unit 732 determines whether or not to use environmental information to acquire the disordered score. If environmental information is to be used, the unit proceeds to step S4603; otherwise, the unit proceeds to step S4604.

[0623] (Step S4603) The score acquisition unit 732 acquires environmental information that corresponds to the time-series information of the target user.

[0624] (Step S4604) The score acquisition unit 732 constructs a vector using the time-series information of the target user, or the time-series information of the target user and environmental information. This vector is called a check vector.

[0625] The test vector is, for example, (time of behavior information 1, start time of behavior information 1, end time of behavior information 1, time of behavior information 2, start time of behavior information 2, end time of behavior information 2, ..., time of behavior information n, start time of behavior information n, end time of behavior information n, environment information 1, ..., environment information m). For example, environment information 1 is the weather, and environment information m is the temperature. Note that m is a natural number greater than or equal to 1.

[0626] (Step S4605) The score acquisition unit 732 acquires the reference vector from the reference management unit 711.

[0627] (Step S4606) The score acquisition unit 732 calculates the distance (e.g., cosΘ) between the test vector and the reference vector.

[0628] (Step S4607) The score acquisition unit 732 acquires the disorder score using an increment function that takes the distance acquired in step S4606 as a parameter. It then returns to the higher-level processing.

[0629] Next, a second example of the score acquisition process in step S4504 will be explained using the flowchart in Figure 47. In the flowchart in Figure 47, the explanation of the same steps as in the flowchart in Figure 46 will be omitted. The second example of the score acquisition process is the case using machine learning.

[0630] (Step S4701) The score acquisition unit 732 acquires a learning model from the standard management unit 711. Here, it is preferable for the score acquisition unit 732 to acquire a learning model that is paired with the user identifier of the target user.

[0631] (Step S4702) The score acquisition unit 732 provides the test vector and the learning model acquired in step S4701 to a machine learning prediction processing module, executes the module, and obtains the prediction result. The prediction result is, for example, a disorder score. The prediction result is, for example, whether or not it is disordered and the score returned by the module.

[0632] (Step S4703) The score acquisition unit 732 acquires a disorder score based on the prediction result. It returns to the higher-level processing.

[0633] The score acquisition unit 732, for example, acquires a disorder score of "0" if the prediction result is "not disordered" (e.g., "0"), and acquires a disorder score calculated by an increasing function that takes the score returned by the module as a parameter if the prediction result is "disordered" (e.g., "1").

[0634] Next, we will explain a third example of the score acquisition process in step S4504 using the flowchart in Figure 48. In the flowchart in Figure 48, the explanation of the same steps as in the flowchart in Figure 46 will be omitted. The third example is one in which the difference between the element information of the time-series information and the element information of the reference information is determined for each element information of the time-series information.

[0635] (Step S4801) The score acquisition unit 732 acquires reference information, including time-series information, from the reference management unit 711. The reference information may include one or more environmental information items. One or more environmental information items may be, for example, "<Weather> Sunny <Temperature> 15°C to 25°C <Humidity> 40% to 60%".

[0636] (Step S4802) The score acquisition unit 732 assigns 1 to counter i.

[0637] (Step S4803) The score acquisition unit 732 determines whether or not the i-th element information exists in the acquired time-series information of the target user. If the i-th element information exists, the unit proceeds to step S4804; otherwise, the unit proceeds to step S4811.

[0638] (Step S4804) The score acquisition unit 732 acquires the i-th element information from the time-series information of the target user. The element information may be behavior information and time information, or environmental information.

[0639] (Step S4805) The score acquisition unit 732 determines whether the behavioral information or environmental information acquired in step S4804 exists in the reference information. If it exists in the reference information, the unit proceeds to step S4806; otherwise, the unit proceeds to step S4809.

[0640] (Step S4806) The score acquisition unit 732 acquires the difference between the time information that is paired with the action information of the i-th element information and the time information in the reference information that is paired with the action information, or the difference between the environmental information (e.g., temperature) of the i-th element information and the environmental information (e.g., temperature) in the reference information. The score acquisition unit 732 temporarily stores the difference in association with the i-th element information.

[0641] The differences are, for example, differences in time, start time, or end time, one or more of the following. The differences are, for example, differences in environmental information or whether the environmental information falls within the range of the environmental information conditions contained in the standard information.

[0642] (Step S4807) The score acquisition unit 732 determines whether the difference obtained in step S4806 satisfies the addition condition. If the addition condition is met, the unit proceeds to step S4808; otherwise, the unit proceeds to step S4809.

[0643] The additive conditions are those that increase the disorder score. For example, an additive condition is when the difference is equal to or greater than a threshold. Another additive condition is when the conditions of the element information within the reference information are not met. For example, the conditions of the element information are "sleep duration is between 6 and 9 hours".

[0644] (Step S4808) The score acquisition unit 732 adds α to the random score. Proceed to step S4810. Note that the initial value of the random score is, for example, "0". Also, α may be a fixed positive number, or it may be a different positive number depending on the magnitude of the difference or the element information.

[0645] (Step S4809) The score acquisition unit 732 adds β to the disordered score. Note that β may be a fixed positive number or a positive number that varies depending on the element information.

[0646] (Step S4810) The score acquisition unit 732 increments counter i by 1. Return to step S4803.

[0647] (Step S4811) The score acquisition unit 732 assigns 1 to counter j.

[0648] (Step S4812) The score acquisition unit 732 determines whether the j-th overall condition exists. If the j-th overall condition exists, the process proceeds to step S4813; otherwise, it returns to the higher-level process. An overall condition is a condition relating to one or more elemental information within the entire time-series information. Examples of overall conditions include: "The time-series information containing a set of activity information for one week contains four or more instances of the activity information 'drinking party'," and "The time-series information containing a set of activity information for one week contains three or more instances where the duration of the activity information 'sleep' is less than 6 hours."

[0649] (Step S4813) The score acquisition unit 732 determines whether the time series information satisfies the j-th overall condition. If the j-th overall condition is met, the unit proceeds to step S4814; otherwise, it proceeds to step S4815.

[0650] (Step S4814) The score acquisition unit 732 adds γ to the disordered score. Note that γ may be a fixed positive number or a positive number that varies depending on the element information.

[0651] (Step S4815) The score acquisition unit 732 increments counter j by 1. Return to step S4812.

[0652] Next, an example of the factor acquisition process in step S4507 will be explained using the flowchart in Figure 49.

[0653] (Step S4901) The factor acquisition unit 735 assigns 1 to counter i.

[0654] (Step S4902) The factor acquisition unit 735 determines whether or not the i-th element information exists in the time-series information. If the i-th element information exists, the process proceeds to step S4903; otherwise, it returns to the higher-level processing.

[0655] (Step S4903) The factor acquisition unit 735 acquires the element information contained in the reference information corresponding to the i-th element information. The factor acquisition unit 735 acquires the difference between the two pieces of element information.

[0656] (Step S4904) The factor acquisition unit 735 determines whether the difference acquired in step S4903 satisfies the acceptance criteria. If the criteria are met, the unit proceeds to step S4905; otherwise, it proceeds to step S4906.

[0657] (Step S4905) The factor acquisition unit 735 temporarily stores the i-th element information in a buffer (not shown). This i-th element information is information that constitutes a disturbance factor.

[0658] (Step S4906) The factor acquisition unit 735 increments the counter i by 1. Return to step S4902.

[0659] Next, an example of the recommendation acquisition process in step S4508 will be explained using the flowchart in Figure 50.

[0660] (Step S5001) The recommendation acquisition unit 736 assigns 1 to counter i.

[0661] (Step S5002) The recommendation acquisition unit 736 determines whether the i-th disturbance factor exists in the buffer (not shown) where the disturbance factors temporarily accumulated in step S4905 are stored. If the i-th disturbance factor exists, the process proceeds to step S5004; otherwise, it returns to the higher-level process.

[0662] (Step S5003) The recommendation acquisition unit 736 acquires element information corresponding to the i-th disorder factor.

[0663] (Step S5004) The recommendation acquisition unit 736 assigns 1 to counter j.

[0664] (Step S5005) The recommendation acquisition unit 736 determines whether the j-th factor condition exists in the recommendation management unit 712. If the j-th factor condition exists, the process goes to step S5006; otherwise, the process goes to step S5010.

[0665] (Step S5006) The recommendation acquisition unit 736 determines whether the element information acquired in step S5003 satisfies the j-th factor condition. If the j-th factor condition is met, the process proceeds to step S5007; otherwise, the process proceeds to step S5009.

[0666] (Step S5007) The recommendation acquisition unit 736 acquires the recommendation source information that corresponds to the j-th factor condition from the recommendation management unit 712.

[0667] (Step S5008) The recommendation acquisition unit 736 uses the recommendation source information and the element information acquired in step S5003 to construct recommendation information and temporarily stores the recommendation information in a buffer (not shown).

[0668] (Step S5009) The recommendation acquisition unit 736 increments counter j by 1. Return to step S5005.

[0669] (Step S5010) The recommendation acquisition unit 736 increments the counter i by 1. The process returns to step S5002.

[0670] Next, an example of the long-term score acquisition process in step S4522 will be explained using the flowchart in Figure 51.

[0671] (Step S5101) The long-term score acquisition unit 733 acquires all disruption scores for a predetermined period from the behavior management unit 313, which are paired with the user identifier of the target user.

[0672] (Step S5102) The long-term score acquisition unit 733 detects the disorder scores that match the disorder conditions from among the disorder scores acquired in step S5101 and obtains the number of such disorder scores. The disorder score is the number of times the user's behavior was disordered during a predetermined period.

[0673] (Step S5103) The long-term score acquisition unit 733 acquires a long-term disorder score using all disorder scores and the number of disorder scores that meet the disorder conditions. It then returns to the higher-level processing. The long-term score acquisition unit 733 acquires a larger long-term disorder score the larger the disorder score and the larger the number of disorder scores that meet the disorder conditions.

[0674] Next, an example of the learning process in step S4528 will be explained using the flowchart in Figure 52.

[0675] (Step S5201) The learning unit 731 assigns 1 to counter i.

[0676] (Step S5202) The learning unit 731 determines whether the i-th time-series information to be learned exists in the action management unit 313. If the i-th time-series information exists, the unit proceeds to step S5203; otherwise, it proceeds to step S5208.

[0677] (Step S5203) The learning unit 731 obtains the i-th time-series information from the action management unit 313.

[0678] (Step S5204) The learning unit 731 constructs a vector from the i-th time series information. This vector is called the explanatory variable vector.

[0679] (Step S5205) The learning unit 731 acquires disturbance information corresponding to the i-th time-series information. The disturbance information is stored in the behavior management unit 313, for example, in pairs with the time-series information.

[0680] (Step S5206) The learning unit 731 acquires training data having explanatory variable vectors and disorder information, and temporarily stores the training data in a buffer that is not shown.

[0681] (Step S5207) The learning unit 731 increments counter i by 1. Return to step S5202.

[0682] (Step S5208) The learning unit 731 provides two or more training data from a buffer (not shown) to a module that performs machine learning training, executes the module, and obtains a training model. It then returns to the higher-level processing.

[0683] The following describes a specific example of the operation of the information system F in this embodiment.

[0684] Let's assume that the behavior management unit 313 of the behavior analysis device 7 stores the time-of-day information management table shown in Figure 53. The time-of-day information management table is a table that manages the behavioral information of one or more users for each time period. Here, the time-of-day information management table has one or more records that have "ID", "Date", "Time Period", "Behavioral Information", and "Disruption Score". For convenience, Figure 53 is a table that manages the behavioral information of one user (User U). Also, let's assume here that the disruption score is obtained in units of time from waking up to before the next waking up (on a daily basis).

[0685] Each record in Figure 53 represents information stored in the behavior management unit 313 by the behavior acquisition device 5. Alternatively, each record in Figure 53 may also represent information entered by user U.

[0686] Furthermore, the standard management unit 711 of the behavioral analysis device 7 stores a standard vector. The standard vector may be a self-standard vector, an external standard vector, or a recommended behavior vector.

[0687] Furthermore, the recommendation management unit 712 stores the recommendation management table shown in Figure 54. The recommendation management table is a table that contains one or more factor conditions and recommendation source information that is paired with the factor conditions.

[0688] In this situation, two specific examples are described below. Specific example 1 is the case where the disorder score and recommendation information are output. Specific example 2 is the case where the degree of improvement and the recovery period are output.

[0689] (Specific example 1) The reception unit 72 of the behavioral analysis device 7 receives time-series information, which includes the date, time zone, and behavioral information pairs "ID=538~548" in Figure 53, from waking up to going to sleep (until before the next waking up), paired with the user identifier of user U. The processing unit 73 then determines that the score acquisition condition (receiving time-series information for the time period from waking up to before the next waking up) is met.

[0690] Next, the score acquisition unit 732 uses the time-series information received from user U to acquire user U's disorder score of "6" through the score acquisition process described above (see Figures 46, 47, or 48). Here, the disorder score can take any rank from "0" to "10". Also, here, the disorder condition is "disorder score >= 5". Next, the score output unit 741 associates the disorder score of "6" with the relevant behavior information and stores it as the attribute value of "disorder score" in Figure 53.

[0691] Next, the determination unit 734 determines that the acquired disorder score "6" matches the disorder condition "disorder score >= 5".

[0692] Next, the factor acquisition unit 735 acquires the time period "0:30-5:00", the behavioral information "sleep", and the standard information corresponding to "sleep" in the standard information of the standard management unit 711 (for example, "<time> 6 hours to 10 hours <start time> until 23:00"), totaling "3 hours" (1.5 hours difference in time and 1.5 hours difference in start time). Then, the factor output unit 743 temporarily stores the time period "0:30-5:00", the behavioral information "sleep", and the difference "3 hours" in a buffer (not shown). In other words, the factor acquisition unit 735 acquires the disturbance factor "sleep".

[0693] Next, the recommendation acquisition unit 736 acquires recommendation information. The factor output unit 743 determines that the information temporarily stored in a buffer (not shown) satisfies the factor condition for "ID=1" in Figure 54. The recommendation acquisition unit 736 acquires the recommendation source information for "ID=1" in Figure 54. The recommendation acquisition unit 736 also acquires the element information "<behavioral information>Sleep <time period>0:30-5:00" which is paired with the disturbance factor "Sleep". The recommendation acquisition unit 736 assigns the acquired element information "<behavioral information>Sleep <time period>0:30-5:00" to the variable <element information> of the recommendation source information, and constructs the recommendation information.

[0694] Next, the processing unit 73 configures output information that includes a disorder score of "6" and recommendation information.

[0695] Next, the output unit 74 transmits the configured output information to user U. The output unit 74 transmits the output information, for example, to user U's email address. The output unit 74 outputs the output information, for example, on the mobile application of the user's terminal device 8.

[0696] Then, the user U's terminal device 8 outputs, for example, a disorder score and recommendation information as shown in Figure 55.

[0697] (Specific example 2) Next, let's assume that one week has passed since the disorder score "6". Then, let's assume that on each day of that week, the score acquisition unit 732 uses the accumulated time-series information to acquire disorder scores of "5", "4", "3", "3", "2", "2", and "0" respectively. Then, let's assume that the score output unit 741 has accumulated disorder scores of "5", "4", "3", "3", "2", "2", and "0" in association with user U, following the disorder score "6".

[0698] The judgment unit 734 then determines that the improvement output condition "the disorder score became 0 after the disorder condition was met" has been met.

[0699] Next, the improvement degree acquisition unit 737 acquires a disturbance score of "6" which matches the disturbance condition, and a disturbance score of "0" which satisfies the improvement degree output condition, from a situation that does not meet the disturbance condition, and obtains an improvement degree of "6-0=6".

[0700] Furthermore, the determination unit 734 determines that the disturbance score "6" obtained when the disturbance condition is met and the recent disturbance score "0" satisfy the recovery condition (the disturbance score became "0" after the disturbance condition was met).

[0701] Next, the recovery period acquisition unit 738 acquires a recovery period of "7 days," which is the difference between the date "10 / 20" included in the time information corresponding to the disturbance score "6" and the date "10 / 27" included in the time information corresponding to the disturbance score.

[0702] Next, the processing unit 73 acquires a disturbance score of "0", an improvement level of "6", and a recovery period of "7 days", and constructs output information containing this information. The output information includes the disturbance score, the improvement level, and the recovery period.

[0703] Next, the output unit 74 transmits the configured output information to user U. The output unit 74 transmits the output information, for example, to user U's email address. The output unit 74 outputs the output information, for example, on the mobile application of the user's terminal device 8.

[0704] Then, the user U's terminal device 8 outputs, for example, a disturbance score, an improvement degree, and a recovery period, as shown in Figure 56.

[0705] As described above, according to this embodiment, a disorder score, which represents the degree of disorder in the user's behavior, can be obtained. Furthermore, according to this embodiment, the disorder score can be obtained using, for example, self-reference information. Also, according to this embodiment, the disorder score can be obtained using, for example, machine learning prediction processing. Also, according to this embodiment, the disorder score can be obtained using, for example, one or more of the following: recommended behavior information, other-person reference information. Also, according to this embodiment, the disorder score can be obtained using, for example, environmental information.

[0706] Furthermore, according to this embodiment, it is possible to obtain the factors that disrupt the user's behavior.

[0707] Furthermore, according to this embodiment, it is possible to provide recommendations for improving irregular user behavior.

[0708] Furthermore, according to this embodiment, the degree of improvement in the user's disruptive behavior can be obtained. In addition, according to this embodiment, the degree of improvement in the user's disruptive behavior can be output at an appropriate timing when the conditions for outputting the degree of improvement are met.

[0709] As mentioned above, the behavior analysis device 7 in this embodiment may also be a terminal. Figure 57 shows a block diagram when the behavior analysis device 7 is a terminal. This behavior analysis device becomes the behavior analysis device 9. In other words, the behavior analysis device 9 has the functions of the behavior acquisition device 3.

[0710] Furthermore, the software that realizes the behavior analysis device 7 in this embodiment is a program as follows. In other words, this program is a program that causes a computer that can access a reference management unit, which stores reference information that serves as the basis for obtaining the degree of disorder in a user's behavior, to function as a receiving unit that receives time-series information including two or more time-series behavioral information that is associated with time information that identifies when a user took an action and is information that identifies the user's action, a score acquisition unit that uses the time-series information received by the receiving unit and the reference information of the reference management unit to acquire a disorder score, which is the degree of disorder in the user's behavior, and a score output unit that outputs the disorder score acquired by the score acquisition unit.

[0711] (Embodiment 6) This embodiment describes a behavioral analysis device that acquires user behavioral information, obtains a physical fitness score and a mental score for said behavioral information, and outputs them.

[0712] In this embodiment, we will describe an action analysis device that acquires physical strength scores and mental strength scores for acquired action information, using cumulative physical strength scores and cumulative mental strength scores acquired based on time-series information, which is two or more past action information of the user.

[0713] In this embodiment, we will describe a behavioral analysis device that acquires a disturbance score using time-series information and uses the disturbance score to acquire a physical strength score and a mental strength score for the acquired behavioral information. Note that the disturbance score is the degree to which behavior is disturbed relative to a steady state, so the disturbance score may also be called a stationarity score. A stationarity score is a score that indicates the degree to which behavior is steady or to which behavior is non-steady.

[0714] In this embodiment, we will describe an action analysis device that acquires and outputs cumulative physical fitness scores and cumulative mental scores using physical fitness scores and mental scores for action information, and an internal state estimation model.

[0715] In this embodiment, we will describe a behavioral analysis device that acquires cumulative physical fitness scores and cumulative mental scores using external environmental data.

[0716] This embodiment describes a behavioral analysis device that acquires and outputs a cumulative physical fitness score and a cumulative mental score using one or two types of information from vital data and sensing data.

[0717] This embodiment describes a behavioral analysis device that estimates a cumulative physical fitness score and a cumulative mental score using one or two types of information from vital data and sensing data, and then acquires and outputs a corrected cumulative physical fitness score and a corrected cumulative mental score based on the estimated cumulative physical fitness score and the estimated cumulative mental score.

[0718] In this embodiment, we will describe a behavioral analysis device that modifies the internal state estimation model according to the updated cumulative physical strength score and the updated cumulative mental strength score.

[0719] Figure 58 is a conceptual diagram of the information system G in this embodiment. The information system G comprises a behavioral analysis device 10, one or more terminal devices 11, and two or more communication devices B.

[0720] Figure 59 is a block diagram of the information system G in this embodiment. Figure 60 is a block diagram of the behavioral analysis device 10.

[0721] The behavior analysis device 10 comprises a storage unit 101, a reception unit 102, a processing unit 103, and an output unit 104. The behavior analysis device 10 may have all or part of the configuration of the behavior analysis device 7 or the behavior analysis device 9. In other words, the behavior analysis device 10 may implement all or part of the functions of the behavior analysis device 7 or the behavior analysis device 9. The storage unit 101 comprises a knowledge storage unit 1011, a reference management unit 711, a model storage unit 1013, and a learning management unit 1014. The processing unit 103 comprises a score acquisition unit 1031, an external environment acquisition unit 1032, a cumulative score acquisition unit 1033, a data acquisition unit 1034, a prediction unit 1035, and a model update unit 1036. The output unit 104 comprises a score output unit 1041, a cumulative score output unit 1042, and a predicted value output unit 1043.

[0722] The terminal device 11 includes a terminal storage unit 61, a terminal receiving unit 62, a terminal receiving unit 83, a terminal processing unit 114, a terminal transmission unit 65, and a terminal output unit 86. The terminal storage unit 61 includes a map management unit 312. The terminal processing unit 114 includes an intensity acquisition unit 231, a type determination unit 232, a position acquisition unit 332, an activity acquisition unit 333, a vital sign acquisition unit 334, a location acquisition unit 337, and a sensor information acquisition unit 1141.

[0723] The storage unit 101, which constitutes the behavioral analysis device 10, stores various types of information. These types of information include, for example, knowledge information, reference information, an internal state estimation model, learning information, behavioral information, and various conditions, which will be described later. It goes without saying that these conditions may also be embedded in the program.

[0724] These various conditions include, for example, model update conditions and recommendation conditions.

[0725] Model update conditions are the conditions for updating the internal state estimation model. For example, model update conditions include: after constructing the internal state estimation model, the newly stored training data has exceeded a threshold; and the difference between the cumulative score obtained using the internal state estimation model and the predicted cumulative score obtained by the prediction unit 1035 (described later) is equal to or greater than a threshold. The cumulative score consists of the cumulative score physical strength and the cumulative score mental strength. The predicted cumulative score consists of the predicted cumulative physical strength score and the predicted cumulative score mental strength.

[0726] Recommendation criteria are the conditions for recommending an action based on a candidate's actions. Examples of recommendation criteria include a score being above or greater than a threshold, or a score ranking being above or higher than a threshold. The scores refer to physical and mental scores.

[0727] The knowledge storage unit 1011 stores two or more pieces of knowledge information. These pieces of knowledge information are either a physical strength score corresponding to an action information, or a mental score corresponding to an action information.

[0728] Activity information includes, for example, "work," "watching TV," "walking," "running," "gym," "bathing," and "sleeping." Activity information may also include the time the activity was performed and the time period during which the activity occurred. The time period is the start time and / or end time.

[0729] The stamina score represents the degree to which a user's stamina recovers or is depleted when performing the action specified by the action information. For example, the stamina score can be "+3" or "-2". A stamina score of "+3" indicates that 3 units of stamina will be recovered when performing the action specified by the action information. A stamina score of "-2" indicates that 2 units of stamina will be depleted when performing the action specified by the action information. There is no limit to the range of values ​​that the stamina score can take or the number of levels in the stamina score.

[0730] The mental score represents the degree to which a user's mental fatigue recovers or is depleted after performing the action specified by the behavioral information. For example, a mental score could be "+3" or "-2". A mental score of "+3" indicates that performing the action specified by the behavioral information will recover "3" units of mental fatigue. A mental score of "-2" indicates that performing the action specified by the behavioral information will deplete "2" units of mental fatigue. The range of possible values ​​for the mental score and the number of levels in the mental score are not considered.

[0731] As mentioned above, the standards management unit 711 stores the standard information that serves as the basis for obtaining the degree of disruption in user behavior corresponding to time-series information.

[0732] The model storage unit 1013 stores one or more internal state estimation models. These internal state estimation models may be different for each user, or they may be common to two or more users. For example, a user identifier is associated with each internal state estimation model in the model storage unit 1013.

[0733] The internal state estimation model is a model for obtaining updated cumulative physical strength scores and updated cumulative mental scores using newly acquired physical strength scores, newly acquired mental scores, cumulative physical strength scores based on past actions, and cumulative mental scores based on past actions. Preferably, the cumulative physical strength score based on past actions is the cumulative physical strength score immediately before the latest action information is received. Similarly, preferably, the cumulative mental score based on past actions is the cumulative mental score immediately before the latest action information is received.

[0734] An internal state estimation model for obtaining only the cumulative physical strength score may be appropriately referred to as the first internal state estimation model. An internal state estimation model for obtaining only the cumulative mental strength score may be appropriately referred to as the second internal state estimation model. An internal state estimation model for obtaining both the cumulative physical strength score and the cumulative mental strength score may be appropriately referred to as the integrated internal state estimation model. It is preferable that the model storage unit 1013 contains both the first internal state estimation model and the second internal state estimation model. In other words, it is preferable that the internal state estimation model for obtaining the cumulative physical strength score and the internal state estimation model for obtaining the cumulative mental strength score are different models. An internal state estimation model may be, for example, a calculation formula, a learning model, or a correspondence table. (1) When the internal state estimation model is an equation (1-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0735] The first internal state estimation model is a first calculation formula that takes the newly acquired physical strength score and the cumulative physical strength score based on past actions as parameters and outputs an updated cumulative physical strength score. The first calculation formula is an increasing function with the physical strength score and the cumulative physical strength score as parameters.

[0736] The second internal state estimation model is a second calculation formula that takes the newly acquired mental score and the cumulative mental score based on past actions as parameters and outputs an updated cumulative mental score. The second calculation formula is an increasing function with the mental score and the cumulative mental score as parameters. (1-2) When obtaining cumulative physical score and cumulative mental score using one model

[0737] The integrated internal state estimation model is a comprehensive calculation formula that uses newly acquired physical strength scores, newly acquired mental strength scores, cumulative physical strength scores based on past actions, and cumulative mental strength scores based on past actions as parameters, and outputs updated cumulative physical strength scores and updated cumulative mental strength scores.

[0738] The above-mentioned calculation formulas are obtained using two or more training data sets, for example, through multiple regression analysis, polynomial regression, Bayesian regression, etc. (2) When the internal state estimation model is a learning model (2-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0739] The first internal state estimation model is a first learned model obtained through machine learning training using two or more training datasets, with newly acquired physical fitness scores and cumulative physical fitness scores based on past actions as explanatory variables, and the updated cumulative physical fitness score as the dependent variable. The dependent variable, the cumulative physical fitness score, is, for example, a value entered by the user. The method of creating the training dataset is not specified.

[0740] The second internal state estimation model is a second learned model obtained through machine learning training using two or more training datasets, with newly acquired mental scores and cumulative mental scores based on past behavior as explanatory variables, and the updated cumulative mental score as the dependent variable. The dependent variable, the cumulative mental score, is, for example, a value entered by the user. The method of creating the training dataset is not restricted.

[0741] A learning model is information constructed through the learning process of machine learning, and is used in the prediction process of machine learning. A learning model can also be called a learner, classifier, or classification model. The machine learning algorithm can be deep learning, random forest, decision tree, SVR, etc. Furthermore, various machine learning functions and various existing libraries can be used in machine learning, such as the TensorFlow® library, the R language's random forest module, and TinySVM. (2-2) When obtaining cumulative physical score and cumulative mental score using one model

[0742] The integrated internal state estimation model is a learned model obtained through machine learning training using two or more training data sets, with newly acquired physical strength scores, newly acquired mental strength scores, cumulative physical strength scores based on past actions, and cumulative mental strength scores based on past actions as explanatory variables, and the pair of updated cumulative physical strength scores and updated cumulative mental strength scores as the dependent variable. The dependent variables, cumulative physical strength scores and cumulative mental strength scores, are, for example, values ​​entered by the user. The method of creating the training data is not specified. (3) When the internal state estimation model is a correspondence table (3-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0743] The first internal state estimation model is a first correspondence table having two or more correspondence pieces of information, which include a newly acquired physical fitness score, a cumulative physical fitness score based on past actions, and an updated cumulative physical fitness score.

[0744] The second internal state estimation model is a second correspondence table that has two or more correspondence pieces of information, which include a newly acquired mental score, a cumulative mental score based on past behavior, and an updated cumulative mental score. (3-2) When obtaining cumulative physical score and cumulative mental score using one model

[0745] The integrated internal state estimation model is an integrated correspondence table having two or more pairs of correspondence information, each containing a newly acquired physical strength score, a newly acquired mental strength score, a cumulative physical strength score based on past actions, a cumulative mental strength score based on past actions, and an updated cumulative physical strength score and an updated cumulative mental strength score.

[0746] The learning management unit 1014 stores one or more learning information. The learning information may be different for each user, or it may be common to two or more users. The learning information in the learning management unit 1014 may also be associated with a user identifier.

[0747] The learning information in the learning management unit 1014 includes, for example, learning information categorized by action, or measured learning information.

[0748] Action-specific learning information refers to the learning information used by the score acquisition unit 1031 when acquiring a score for a single action. Actual measurement learning information refers to the learning information used by the prediction unit 1035 when acquiring a cumulative score.

[0749] Action-specific learning information is information created using two or more training data sets. Such training data sets have two or more explanatory variables and an objective variable. Each of the two or more explanatory variables may include, for example, a physical strength score and a mental score obtained using knowledge information. Preferably, each of the two or more explanatory variables may include past cumulative physical strength scores, past cumulative mental scores, and disorder scores. Preferably, the past cumulative physical strength scores and past cumulative mental scores are the cumulative physical strength scores and cumulative mental scores immediately before acquiring the action. The objective variables are the cumulative physical strength scores and cumulative mental scores.

[0750] Action-specific learning information includes, for example, a learning model, a correspondence table, and a calculation formula. The learning management unit 1014 may contain action-specific learning information that outputs a physical strength score and action-specific learning information that outputs a mental strength score, or it may contain action-specific learning information that outputs both the physical strength score and the mental strength score together. (1) When behavior-specific learning information is used as a learning model

[0751] Behavioral learning information is a learning model obtained through machine learning training using two or more training datasets. The behavioral learning model can be the first behavioral learning model, the second behavioral learning model, or a combined behavioral learning model. (1-1) When physical and mental scores are obtained using different models.

[0752] The first behavior-based learning model is a learning model obtained by performing machine learning training using two or more training data sets with physical fitness scores as the dependent variable, and two or more pieces of information, including physical fitness scores obtained using knowledge information, as explanatory variables. Preferably, the explanatory variables include past cumulative physical fitness scores and disorder scores.

[0753] The second behavior-based learning model is a learning model obtained by performing machine learning training using two or more training data sets with mental scores as the dependent variable, and two or more pieces of information, including mental scores obtained using knowledge information, as explanatory variables. Preferably, the explanatory variables include past cumulative mental scores and disorder scores.

[0754] The training data used to construct the first behavior-based learning model and the second behavior-based learning model may be the same or different. (1-2) When obtaining physical and mental scores using one model

[0755] The integrated behavioral learning model is a learning model obtained by performing machine learning training using two or more training datasets with physical fitness scores and mental scores as dependent variables, and with two or more pieces of information, including physical fitness scores and mental scores, obtained using knowledge information, as explanatory variables. Preferably, the explanatory variables include past cumulative physical fitness scores, past cumulative mental scores, and disorder scores. (2) When the learning information is a correspondence table (2-1) When physical and mental scores are obtained using different models.

[0756] The first behavior-specific correspondence table is a correspondence table in which two or more pieces of information, including physical fitness scores obtained using knowledge information, are used as explanatory variables, and two or more training data sets, with the physical fitness score as the dependent variable, are used as corresponding information. It is preferable that the explanatory variables include past cumulative physical fitness scores and disorder scores.

[0757] The second behavior-based correspondence table is a correspondence table in which two or more pieces of information, including mental scores obtained using knowledge information, are used as explanatory variables, and two or more training data sets, with the mental score as the dependent variable, are used as corresponding information. It is preferable that the explanatory variables include past cumulative mental scores and disorder scores.

[0758] The training data used to construct the first and second prediction correspondence tables may be the same or different. (2-2) When obtaining physical and mental scores using one model

[0759] The comprehensive correspondence table is a correspondence table in which two or more pieces of information, including physical strength scores and mental strength scores obtained using knowledge information, are used as explanatory variables, and two or more training data sets, with physical strength scores and mental strength scores as dependent variables, are used as correspondence information. Preferably, the explanatory variables include past cumulative physical strength scores, past cumulative mental strength scores, and disorder scores. (3) When the learning information is an arithmetic expression

[0760] The learning information is a mathematical formula that takes two or more explanatory variables as parameters and outputs the target variable. The formula can be a first action-specific formula, a second action-specific formula, or a combined action-specific formula. (3-1) When physical and mental scores are obtained using different models.

[0761] The first action-based calculation formula takes one or more pieces of information, including the physical strength score obtained using knowledge information, as parameters and returns the physical strength score. Preferably, the parameters include past cumulative physical strength scores and disorder scores.

[0762] The second action-based calculation formula is a formula that takes one or more pieces of information, including a mental score obtained using knowledge information, as parameters and returns a mental score. Preferably, the parameters include past cumulative mental scores and disturbance scores.

[0763] The comprehensive action-based calculation formula is a formula that takes one or more pieces of information as parameters, including a physical strength score obtained using knowledge information and a mental strength score obtained using knowledge information, and returns a physical strength score and a mental strength score. Preferably, the parameters include past cumulative physical strength scores, past cumulative mental strength scores, and disorder scores.

[0764] The learning information used by the prediction unit 1035 is referred to as "measured learning information" as appropriate. Measured learning information is information created using two or more training data sets. Such training data sets have two or more explanatory variables and an objective variable. Here, each of the two or more explanatory variables is one or more types of data from one or more vital data points of the user or one or more sensing data points of the user. The objective variable is one or two types of scores from a cumulative physical fitness score and a cumulative mental score. The cumulative physical fitness score acquired by the prediction unit 1035 is referred to as the predicted cumulative physical fitness score. The cumulative mental score acquired by the prediction unit 1035 is referred to as the predicted cumulative mental score.

[0765] Vital data refers to information obtained from a user's body. It can also be called biometric information. Examples of vital data include walking speed, walking acceleration, heart rate variability, changes in respiratory rate, heart rate per unit time (e.g., 1 minute or 30 seconds), heart rate variability, blood pressure (systolic and / or diastolic), respiratory rate per unit time, and body temperature.

[0766] Sensing data refers to information obtained as a result of sensing a user, or information obtained from such sensing data. Examples of sensing data include images of the user, voice information of the user, the user's temperature, numerical information of the user's smell, and numerical information of the user's taste. Examples of sensing data include emotional information of the user obtained from images of the user, emotional information of the user obtained from voice information of the user, and one or more features of voice information.

[0767] The learning information in the learning management unit 1014 may include, for example, a learning model, a correspondence table, and a calculation formula. The learning management unit 1014 may also contain learning information that outputs a cumulative physical strength score and learning information that outputs a cumulative mental strength score, or it may contain learning information that outputs both the cumulative physical strength score and the cumulative mental strength score together. (1) When the learning information is a learning model

[0768] The training information is a training model obtained through machine learning training using two or more training datasets. The training model is either a first predictive training model, a second predictive training model, or a combined predictive training model. (1-1) When predicting cumulative physical strength score and predicted cumulative mental strength score are obtained using different models.

[0769] The first predictive learning model is a learning model obtained by performing machine learning training using two or more training data sets, where each of the two or more data acquired by the data acquisition unit 1034 (described later) is used as an explanatory variable and the cumulative physical fitness score is used as the dependent variable.

[0770] The second predictive learning model is a learning model obtained by performing machine learning training using two or more training data sets, where each of the two or more data acquired by the data acquisition unit 1034 (described later) is used as an explanatory variable and the cumulative mental score is used as the dependent variable.

[0771] The training data used to construct the first and second predictive learning models may be the same or different. (1-2) When predicting cumulative physical strength score and predicting cumulative mental strength score are obtained using one model.

[0772] The comprehensive predictive learning model is a learning model obtained by performing machine learning training using two or more training data sets, where each of the two or more data acquired by the data acquisition unit 1034 (described later) is used as an explanatory variable, and the pair of cumulative physical strength score and cumulative mental score is used as the objective variable. (2) When the learning information is a correspondence table

[0773] The learning information is a correspondence table containing two or more correspondence tables, each containing two or more explanatory variables and one or more dependent variables. The correspondence table can be a first prediction correspondence table, a second prediction correspondence table, or a combined prediction correspondence table. (2-1) When predicting cumulative physical strength score and predicted cumulative mental strength score are obtained using different models.

[0774] The first prediction correspondence table is a first correspondence table that has two or more training data sets, each with two or more data points acquired by the data acquisition unit 1034 (described later) as explanatory variables and the cumulative physical fitness score as the dependent variable, as correspondence information.

[0775] The second prediction correspondence table is a second correspondence table that has two or more training data sets, each with the cumulative mental score as the dependent variable, and two or more data sets acquired by the data acquisition unit 1034 (described later) as explanatory variables.

[0776] The training data used to construct the first and second prediction correspondence tables may be the same or different. (2-2) When predicting cumulative physical strength score and predicting cumulative mental strength score are obtained using one model.

[0777] The comprehensive correspondence table is a correspondence table that contains correspondence information for each of the two or more data and each training data acquired by the data acquisition unit 1034, which will be described later. (3) When the learning information is an arithmetic expression

[0778] The training information is a formula that takes two or more explanatory variables as parameters and outputs the target variable. The formula can be a first prediction formula, a second prediction formula, or a combined prediction formula. (3-1) When predicting cumulative physical strength score and predicted cumulative mental strength score are obtained using different models.

[0779] The first prediction calculation formula is a formula that takes two or more data points acquired by the data acquisition unit 1034 (described later) as parameters and returns a predicted cumulative physical strength score.

[0780] The second prediction calculation formula is a formula that takes two or more data points acquired by the data acquisition unit 1034 (described later) as parameters and returns a predicted cumulative mental score. (3-2) When predicting cumulative physical strength score and predicting cumulative mental strength score are obtained using one model.

[0781] The comprehensive prediction calculation formula is a calculation formula that takes two or more data acquired by the data acquisition unit 1034 (described later) as parameters and returns a predicted cumulative physical strength score and a predicted cumulative mental score.

[0782] The reception unit 102 receives various instructions and information. These instructions and information include, for example, behavioral information, time-series information, and behavioral recommendation instructions.

[0783] An action recommendation directive is a directive that recommends one or more actions from among two or more actions. Typically, an action recommendation directive recommends one or more actions from among two or more actions that could be taken in the future. For example, an action recommendation directive may have two or more action pieces of information that identify two or more candidate actions.

[0784] The reception unit 102, for example, receives behavioral information. This behavioral information identifies actions that the user has taken or actions that the user may take in the future.

[0785] The reception unit 102 receives, for example, time-series information. Time-series information is information that identifies two or more action information in a time series. Preferably, such action information corresponds to time information that identifies when the user took the action.

[0786] Here, "reception" typically refers to the reception of information transmitted via wired or wireless communication lines, but it may also be a concept that includes the reception of information input from input devices such as keyboards, mice, and touch panels, as well as the reception of information read from recording media such as optical discs, magnetic discs, and semiconductor memory.

[0787] The processing unit 103 performs various processes. These processes include, for example, those performed by the score acquisition unit 1031, the external environment acquisition unit 1032, the cumulative score acquisition unit 1033, the data acquisition unit 1034, the prediction unit 1035, or the model update unit 1036.

[0788] The score acquisition unit 1031 acquires the physical strength score and mental strength score associated with the action information received by the reception unit 102. For example, the score acquisition unit 1031 acquires the physical strength score and mental strength score associated with the action information received by the reception unit 102 from the knowledge storage unit 1011.

[0789] The score acquisition unit 1031 preferably acquires a total score using the acquired physical strength score and the acquired mental strength score. Typically, the score acquisition unit 1031 acquires a larger total score the greater the physical strength score and the greater the mental strength score. For example, the score acquisition unit 1031 calculates the total score using an increasing function with physical strength score and mental strength score as parameters.

[0790] The score acquisition unit 1031 acquires, for example, a physical strength score and a mental strength score for each of the two or more action information contained in the time-series information received by the reception unit 102.

[0791] The score acquisition unit 1031, for example, uses the time-series information received by the reception unit 102 and the standard information from the standard management unit 711 to acquire a disorder score, which is the degree of disorder in the user's behavior, and also uses the disorder score to acquire a physical strength score and a mental strength score.

[0792] The score acquisition unit 1031 acquires physical strength and mental strength scores for a given action information, for example, using action-specific learning information. An example of the processing by the score acquisition unit 1031 when the action-specific learning information is a learning model, a correspondence table, or a calculation formula will be described below. First, the score acquisition unit 1031 acquires the physical strength and mental strength scores associated with the action information received by the reception unit 102. Furthermore, if past cumulative scores exist in the storage unit 101, it is preferable for the score acquisition unit 1031 to acquire physical strength and mental strength scores using those past cumulative scores. Furthermore, if disordered scores exist in the storage unit 101, it is preferable for the score acquisition unit 1031 to acquire physical strength and mental strength scores using those disordered scores. (1) When behavior-specific learning information is used as a learning model (1-1) When physical and mental scores are obtained using different models.

[0793] The score acquisition unit 1031 obtains, for example, a physical fitness score corresponding to the behavioral information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 then performs machine learning prediction processing using, for example, the physical fitness score, or the physical fitness score and cumulative physical fitness score, or the physical fitness score and disorder score, or the physical fitness score, cumulative physical fitness score and disorder score, and the first behavior-specific learning model to obtain the physical fitness score.

[0794] Furthermore, the score acquisition unit 1031 obtains, for example, a mental score corresponding to the behavioral information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 then performs machine learning prediction processing using, for example, the mental score, or the mental score and cumulative mental score, or the mental score and disorder score, or the mental score, cumulative mental score and disorder score, and a second behavior-specific learning model, to obtain a mental score. (1-2) When obtaining physical and mental scores using one model

[0795] The score acquisition unit 1031 obtains, for example, the physical strength score and mental strength score associated with the behavioral information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 performs machine learning prediction processing using, for example, the physical strength score and mental strength score, or the physical strength score and mental strength score and cumulative physical strength score and cumulative mental strength score, or the physical strength score and mental strength score and disorder score, or the physical strength score and mental strength score and cumulative physical strength score and cumulative mental strength score and disorder score, and a comprehensive behavioral learning model, to obtain the physical strength score and mental strength score. (2) When the learning information is a correspondence table (2-1) When physical and mental scores are obtained using different models.

[0796] The score acquisition unit 1031 obtains, for example, a physical strength score from the knowledge storage unit 1011 that corresponds to the action information received by the reception unit 102. The score acquisition unit 1031 constructs a vector from, for example, the physical strength score, or the physical strength score and the cumulative physical strength score, or the physical strength score and the disorder score, or the physical strength score, the cumulative physical strength score, and the disorder score. Next, the score acquisition unit 1031 obtains the physical strength score that is paired with the vector that most closely approximates the said vector from the first action-specific correspondence table.

[0797] The score acquisition unit 1031 obtains, for example, a mental score from the knowledge storage unit 1011 that corresponds to the action information received by the reception unit 102. The score acquisition unit 1031 constructs a vector from, for example, the mental score, or the mental score and the cumulative mental score, or the mental score and the disorder score, or the mental score, the cumulative mental score, and the disorder score. Next, the score acquisition unit 1031 obtains the mental score that is paired with the vector that most closely approximates the said vector from a second action-specific correspondence table. (2-2) When obtaining physical and mental scores using one model

[0798] The score acquisition unit 1031 obtains, for example, the physical strength score and mental strength score corresponding to the action information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 constructs a vector from, for example, the physical strength score and the mental strength score, or the physical strength score and the mental strength score and the cumulative physical strength score and the cumulative mental strength score, or the physical strength score and the mental strength score and the disorder score, or the physical strength score and the mental strength score and the cumulative physical strength score and the cumulative mental strength score and the disorder score. Next, the score acquisition unit 1031 obtains the physical strength score and mental strength score that are paired with the vector that most closely approximates the said vector from the comprehensive action-specific correspondence table. (3) When the learning information is an arithmetic expression (3-1) When physical and mental scores are obtained using different models.

[0799] The score acquisition unit 1031 obtains, for example, the physical strength score corresponding to the action information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 obtains, for example, the physical strength score, or the physical strength score and the cumulative physical strength score, or the physical strength score and the disorder score, or the physical strength score, the cumulative physical strength score, and the disorder score. Next, the score acquisition unit 1031 substitutes each of the acquired pieces of information as parameters into the first action-specific calculation formula, executes the first action-specific calculation formula, and obtains the physical strength score.

[0800] Furthermore, the score acquisition unit 1031 acquires, for example, a mental score corresponding to the action information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 acquires, for example, the mental score, or the mental score and the cumulative mental score, or the mental score and the disorder score, or the mental score, the cumulative mental score, and the disorder score. Next, the score acquisition unit 1031 substitutes each acquired piece of information as a parameter into a second action-specific calculation formula, executes the second action-specific calculation formula, and acquires the mental score. (3-2) When obtaining physical and mental scores using one model

[0801] The score acquisition unit 1031 obtains, for example, the physical strength score and mental strength score associated with the action information received by the reception unit 102 from the knowledge storage unit 1011. The score acquisition unit 1031 obtains, for example, the physical strength score and the mental strength score, or the physical strength score and the mental strength score and the cumulative physical strength score and the cumulative mental strength score, or the physical strength score and the mental strength score and the disorder score, or the physical strength score and the mental strength score and the cumulative physical strength score and the cumulative mental strength score and the disorder score. Next, the score acquisition unit 1031 substitutes each of the acquired pieces of information as parameters into the comprehensive action-specific calculation formula, executes the comprehensive action-specific calculation formula, and obtains the physical strength score and the mental strength score.

[0802] The external environment acquisition unit 1032 acquires one or more external environment data. The external environment acquisition unit 1032 receives one or more external environment data from, for example, a server (not shown). The external environment acquisition unit 1032 receives one or more external environment data from, for example, a terminal device 11. The external environment acquisition unit 1032 acquires one or more external environment data using sensors.

[0803] External environmental data includes, for example, the temperature, atmospheric pressure, weather, and humidity of the user's location, communication information, and search logs. Communication information refers to information about communications conducted by the user. This includes, for example, user input information or user received information. User input information is information entered by the user. This includes, for example, emails sent by the user, information posted by the user on social media, and information entered by the user in chat. User received information includes, for example, emails received by the user and information entered by others in channels of groups to which the user belongs.

[0804] The cumulative score acquisition unit 1033 acquires a cumulative physical strength score using the physical strength score for each of the two or more action information contained in the time-series information. The cumulative score acquisition unit 1033 acquires a cumulative mental strength score using the mental strength score for each of the two or more action information contained in the time-series information. The cumulative score acquisition unit 1033 may store the cumulative physical strength score and the cumulative mental strength score in the storage unit 101 or the like. The cumulative score acquisition unit 1033 may store the cumulative physical strength score and the cumulative mental strength score in the storage unit 101 or the like, associating them with a user identifier.

[0805] The cumulative score acquisition unit 1033 preferably acquires a total cumulative score using the acquired cumulative physical strength score and cumulative mental strength score. The cumulative score acquisition unit 1033 usually acquires a larger total cumulative score the larger the cumulative physical strength score and the larger the cumulative mental strength score. For example, the cumulative score acquisition unit 1033 calculates the total cumulative score using an increasing function with the cumulative physical strength score and cumulative mental strength score as parameters. The total cumulative score can also be called the internal state score.

[0806] If there is no cumulative physical strength score or cumulative mental strength score, the cumulative score acquisition unit 1033 may use the physical strength score acquired by the score acquisition unit 1031 as the cumulative physical strength score, and the mental strength score acquired by the score acquisition unit 1031 as the cumulative mental strength score.

[0807] The cumulative score acquisition unit 1033 acquires the cumulative physical strength score, for example, by an increasing function that takes as parameters the physical strength scores for each of the two or more action information contained in the time-series information. The cumulative score acquisition unit 1033 acquires the cumulative physical strength score, for example, by adding the physical strength scores for each of the two or more action information contained in the time-series information. It is preferable for the cumulative score acquisition unit 1033 to acquire a cumulative physical strength score that does not exceed the maximum value of the cumulative physical strength score.

[0808] The cumulative score acquisition unit 1033 acquires the cumulative mental score, for example, by an increasing function that takes as parameters the mental scores for each of the two or more action information contained in the time-series information. The cumulative score acquisition unit 1033 acquires the cumulative mental score by adding the mental scores for each of the two or more action information contained in the time-series information. It is preferable for the cumulative score acquisition unit 1033 to acquire a cumulative mental score that does not exceed the maximum value of the cumulative mental score.

[0809] Cumulative physical strength score represents the user's accumulated physical strength recovery or depletion level. Cumulative mental strength score represents the user's accumulated mental fatigue recovery or depletion level.

[0810] The cumulative score acquisition unit 1033 acquires, for example, the physical strength score acquired by the score acquisition unit 1031, the mental score acquired by the score acquisition unit 1031, the accumulated cumulative physical strength score, and the accumulated cumulative mental score. It then applies the physical strength score, the mental score, the accumulated physical strength score, and the accumulated mental score to an internal state estimation model to acquire and store the accumulated physical strength score and the accumulated mental score.

[0811] The cumulative score acquisition unit 1033 applies, for example, the physical strength score, mental strength score, cumulative physical strength score, cumulative mental strength score, and one or more external environmental data to an internal state estimation model to acquire and store the cumulative physical strength score and cumulative mental strength score.

[0812] The following describes the processing of the cumulative score acquisition unit 1033 in each case where the internal state estimation model is a calculation formula, a learning model, or a correspondence table. (1) When the internal state estimation model is an equation (1-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0813] The cumulative score acquisition unit 1033 acquires the physical strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0814] Next, the cumulative score acquisition unit 1033 substitutes the physical strength score and the cumulative physical strength score into the first calculation formula. The cumulative score acquisition unit 1033 may also substitute one or more external environmental data into the first calculation formula. Next, the cumulative score acquisition unit 1033 executes the first calculation formula and obtains the updated cumulative physical strength score.

[0815] The cumulative score acquisition unit 1033 acquires the mental score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative mental score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0816] Next, the cumulative score acquisition unit 1033 substitutes the mental score and the cumulative mental score into the second calculation formula. The cumulative score acquisition unit 1033 may also substitute one or more external environment data into the second calculation formula. Next, the cumulative score acquisition unit 1033 executes the second calculation formula and obtains the updated cumulative mental score. (1-2) When obtaining cumulative physical score and cumulative mental score using one model

[0817] The cumulative score acquisition unit 1033 acquires the physical strength score and mental strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score and accumulated mental strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data points acquired by the external environment acquisition unit 1032.

[0818] Next, the cumulative score acquisition unit 1033 substitutes the physical score, mental score, cumulative physical score, and cumulative mental score into the comprehensive calculation formula. The cumulative score acquisition unit 1033 may also substitute one or more external environment data into the comprehensive calculation formula.

[0819] Next, the cumulative score acquisition unit 1033 executes the comprehensive calculation formula and acquires the updated cumulative physical strength score and the updated cumulative mental score. (2) When the internal state estimation model is a learning model (2-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0820] The cumulative score acquisition unit 1033 acquires the physical strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0821] Next, the cumulative score acquisition unit 1033 uses the first learning model, the physical fitness score, and the cumulative physical fitness score to perform machine learning prediction processing and acquire the updated cumulative physical fitness score. The cumulative score acquisition unit 1033 may also use one or more external environmental data points to perform machine learning prediction processing and acquire the updated cumulative physical fitness score.

[0822] The cumulative score acquisition unit 1033 acquires the mental score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative mental score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0823] Next, the cumulative score acquisition unit 1033 uses the second learning model, the mental score, and the cumulative mental score to perform machine learning prediction processing and obtain an updated cumulative mental score. The cumulative score acquisition unit 1033 may also use one or more external environmental data points to perform machine learning prediction processing and obtain an updated cumulative mental score. (2-2) When obtaining cumulative physical score and cumulative mental score using one model

[0824] The cumulative score acquisition unit 1033 acquires the physical strength score and mental strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score and accumulated mental strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data points acquired by the external environment acquisition unit 1032.

[0825] Next, the cumulative score acquisition unit 1033 uses the learning model, physical strength score, mental score, cumulative physical strength score, and cumulative mental score to perform machine learning prediction processing and acquire the updated cumulative mental score. The cumulative score acquisition unit 1033 may also use one or more external environmental data to perform machine learning prediction processing and acquire the updated cumulative physical strength score and updated cumulative mental score. (3) When the internal state estimation model is a correspondence table (3-1) When cumulative physical score and cumulative mental score are obtained using different models.

[0826] The cumulative score acquisition unit 1033 acquires the physical strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0827] Next, the cumulative score acquisition unit 1033 refers to the first correspondence table and obtains the cumulative physical strength score from the first correspondence table as the updated cumulative physical strength score, which is paired with the vector that most closely approximates the vector whose elements are the physical strength score and the cumulative physical strength score. Alternatively, the cumulative score acquisition unit 1033 may obtain the cumulative physical strength score from the first correspondence table as the updated cumulative physical strength score, which is paired with the vector that most closely approximates the vector whose elements are the physical strength score, the cumulative physical strength score and the one or more external environmental data.

[0828] The cumulative score acquisition unit 1033 acquires the mental score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative mental score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data acquired by the external environment acquisition unit 1032.

[0829] Next, the cumulative score acquisition unit 1033 refers to the second correspondence table and obtains the cumulative mental score from the second correspondence table as the updated cumulative mental score, which is paired with the vector that most closely approximates the vector whose elements are the mental score and the cumulative mental strength score. Alternatively, the cumulative score acquisition unit 1033 may obtain the cumulative mental score from the second correspondence table as the updated cumulative mental score, which is paired with the vector that most closely approximates the vector whose elements are the mental score, the cumulative mental score, and the one or more external environment data. (3-2) When obtaining cumulative physical score and cumulative mental score using one model

[0830] The cumulative score acquisition unit 1033 acquires the physical strength score and mental strength score acquired by the score acquisition unit 1031. The cumulative score acquisition unit 1033 also acquires the cumulative physical strength score and accumulated mental strength score stored in the storage unit 101. The cumulative score acquisition unit 1033 may also acquire one or more external environment data points acquired by the external environment acquisition unit 1032.

[0831] Next, the cumulative score acquisition unit 1033 refers to the comprehensive correspondence table and obtains the cumulative physical score and cumulative mental score from the comprehensive correspondence table as the updated cumulative physical score and updated cumulative mental score, which are the vectors that best approximate the vector having physical score, mental score, cumulative physical score, and cumulative mental score as elements. Alternatively, the cumulative score acquisition unit 1033 may obtain the cumulative mental score from the comprehensive correspondence table as the updated cumulative mental score, which is the vectors that best approximate the vector having physical score, mental score, cumulative physical score, cumulative mental score, and one or more external environment data as elements, which are the vectors that best approximate the vector.

[0832] The cumulative score acquisition unit 1033, for example, uses the predicted cumulative physical strength score acquired by the prediction unit 1035 (described later) to acquire a corrected cumulative physical strength score by correcting the acquired cumulative physical strength score. The cumulative score acquisition unit 1033 also acquires a corrected cumulative physical strength score by correcting the cumulative physical strength score based on the difference between the cumulative physical strength score and the predicted cumulative physical strength score. The cumulative score acquisition unit 1033 also acquires representative values ​​(e.g., mean, weighted mean) of the cumulative physical strength score and the predicted cumulative physical strength score.

[0833] The cumulative score acquisition unit 1033, for example, uses the predicted cumulative mental score acquired by the prediction unit 1035 (described later) to acquire a corrected cumulative mental score by correcting the acquired cumulative mental score. The cumulative score acquisition unit 1033 also acquires a corrected cumulative mental score by correcting the cumulative mental score based on the difference between the cumulative mental score and the predicted cumulative mental score. The cumulative score acquisition unit 1033 also acquires representative values ​​(e.g., mean, weighted mean) of the cumulative mental score and the predicted cumulative mental score.

[0834] The data acquisition unit 1034 acquires two or more data, which are one or more types of data from the user's vital data or one or more sensing data from the user. The means by which the data acquisition unit 1034 acquires vital data or sensing data are not limited. For example, the data acquisition unit 1034 receives one or more types of data from the terminal device 11 or a device not shown, which are either one or more types of vital data or one or more types of sensing data. The data acquisition unit 1034 acquires one or more types of data from the user's sensors, which are either one or more types of vital data or one or more types of sensing data. The vital data acquired by the data acquisition unit 10...

Claims

1. A receiving unit that receives time-series information including one or more pieces of user action information in a time series, The speech information identifies the utterance directed to the user, and the speech acquisition unit acquires the utterance information corresponding to the time-series information received by the reception unit, A speech output device comprising: a speech output unit that outputs the speech information acquired by the speech acquisition unit.

2. The system further comprises a feedback acquisition unit that acquires feedback information for the output of the aforementioned speech information. The aforementioned speech acquisition unit, The speech output device according to claim 1, which also uses the feedback information acquired by the feedback acquisition unit to acquire the speech information.

3. The aforementioned feedback acquisition unit, The speech output device according to claim 2, wherein the speech output unit acquires the user's face image when it outputs the speech information, and acquires the feedback information based on the face image.

4. The aforementioned feedback acquisition unit, The speech output device according to claim 2, which acquires the user's voice information when the speech output unit outputs the speech information, and acquires the feedback information based on the voice information.

5. The aforementioned feedback acquisition unit, The speech output device according to claim 2, wherein after the speech output unit outputs the speech information, it acquires information input by the user and acquires the feedback information based on that information.

6. The aforementioned speech acquisition unit, The speech output device according to claim 1, which acquires input information based on the aforementioned time-series information, provides the input information to the LLM, and acquires the speech information from the LLM.

7. The speech output device according to claim 1, which is a conversational robot.

8. A speech output method comprising all the processing performed by the speech output device described in any one of claims 1 to 6.

9. Computers, A program for causing a speech output device to function as described in any one of claims 1 to 6.

Images