Information processing device, information processing method, program, recording medium, and data structure

Abstract

To provide an information processing device capable of reminding a user of information to be notified at appropriate timing in accordance with a situation, an information processing method, a program, a recording medium and a data structure.SOLUTION: An information processing device includes a user input information acquisition part for acquiring input information from a user including scheduled action information showing action scheduled by a user, a specification part for specifying corresponding movement means being movement means corresponding to the action scheduled by the user on the basis of the input information, a determination part for determining whether the user moves by the corresponding movement means when the user moves, and a control part for notifying the user of notification information including the scheduled action information through a notification part while moving by the corresponding movement means in the case that the user is determined to move by the corresponding movement means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, a recording medium, and a data structure. [Background technology]

[0002] There is known an interactive navigation device that recognizes a voice input from a user and responds to the recognized content. For example, Patent Document 1 discloses a navigation device that, when a user makes a voice input, acquires information about an item to be purchased and a store that sells the item from the voice, and reminds the user about the purchase when the user approaches the store within a predetermined period. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-51590 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems with the navigation device described in Patent Document 1 is that, for example, if the user never approaches the store while traveling during a specified period, the user will not receive a reminder to purchase an item.

[0005] Another problem is that if the item a user wants to purchase is one that cannot be carried on foot, and the user happens to be approaching a store on foot, the user will receive a reminder to purchase the item that cannot be carried on foot, which the user will find annoying.

[0006] The present invention has been made in consideration of the above points, and aims to provide an information processing device, an information processing method, a program, and a recording medium that are capable of reminding a user of information that should be notified to the user at an appropriate timing depending on the situation. [Means for solving the problem]

[0007] The information processing device described in claim 1 is characterized by having a user input information acquisition unit that acquires input information from the user including planned action information indicating actions planned by the user; an identification unit that identifies a corresponding means of transportation that is a means of transportation corresponding to the action planned by the user based on the input information; a determination unit that determines whether the user is traveling by the corresponding means of transportation while the user is traveling; and a control unit that performs notification processing to notify the user via a notification unit of notification information including the planned action information while the user is traveling by the corresponding means of transportation when it is determined that the user is traveling by the corresponding means of transportation.

[0008] The information processing method described in claim 10 is an information processing method executed by an information processing device, and is characterized by having: a user input information acquisition step of acquiring input information from the user including planned action information indicating the action planned by the user; an identification step of identifying a corresponding means of transportation that is a means of transportation corresponding to the action planned by the user based on the input information; a determination step of determining whether the user is traveling by the corresponding means of transportation while the user is traveling; and a control step of performing notification processing to notify the user of notification information including the planned action information via a notification unit while traveling by the corresponding means of transportation when it is determined that the user is traveling by the corresponding means of transportation.

[0009] The program described in claim 11 is a program to be executed by a computer, and is characterized in that it causes the computer to execute the following steps: a user input information acquisition step of acquiring input information from the user including planned action information indicating the action planned by the user; an identification step of identifying a corresponding means of transportation that is a means of transportation corresponding to the action planned by the user based on the input information; a determination step of determining whether the user is traveling by the corresponding means of transportation while the user is traveling; and a control step of performing notification processing to notify the user of notification information including the planned action information via a notification unit while traveling by the corresponding means of transportation when it is determined that the user is traveling by the corresponding means of transportation.

[0010] A recording medium according to claim 12 is a recording medium on which the above program is recorded.

[0011] The data structure described in claim 13 is a data structure used in an information processing device having an identification unit that identifies the user's means of transportation while the user is moving and a control unit that notifies the user of notification information via a notification unit, and includes planned action information that indicates an action planned by the user, and a corresponding means of transportation that indicates a means of transportation corresponding to the action planned by the user that is indicated by the planned action information, characterized in that the identification unit is used for a process in which the identification unit identifies the user's current means of transportation while the user is moving, and a process in which the control unit notifies the user of the notification information including the planned action information via the notification unit while the user is moving by the corresponding means of transportation when it is determined that the means of transportation identified by the identification unit corresponds to the corresponding means of transportation. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a configuration of an information processing device according to a first embodiment and a sound input to the information processing device. [Figure 2] 1 is a block diagram illustrating an example of a configuration of an information processing device according to a first embodiment. [Figure 3]4 is a table illustrating an example of a scheduled event list stored in the information processing device according to the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating a situation in which notification information is notified to a user from the information processing device according to the first embodiment. [Figure 5] 4 is a flowchart showing a control routine of the information processing device according to the first embodiment. [Figure 6] 4 is a flowchart showing a control routine of the information processing device according to the first embodiment. [Figure 7] 10A and 10B are diagrams illustrating sounds input to an information processing device according to a modified example. [Figure 8] 10 is a table showing an example of attribute information of an object stored in an information processing device according to a modified example. [Figure 9] 10 is a table illustrating an example of a scheduled event list stored in an information processing device according to a modified example. [Figure 10] 10 is a flowchart showing a control routine of an information processing device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and the description of the same components will be omitted. [Example]

[0014] 1 is a diagram illustrating a configuration of a terminal device 10 as an information processing device according to a first embodiment and a voice input to the terminal device 10. The terminal device 10 is a communication device including a microphone 12, a touch panel display 13, a speaker 14, a GPS receiver built into the terminal device 10, and a control unit that controls these components, all of which are housed in a frame F. In this embodiment, the terminal device 10 is a smartphone capable of communicating with other devices.

[0015] The microphone 12 is a voice input device that receives voice uttered by, for example, an operator (hereinafter also referred to as a user) of the terminal device 10 and converts the voice into an electrical signal. In this embodiment, the microphone 12 is provided on one end side of the terminal device 10.

[0016] The touch panel display 13 is a display device that combines a display that displays a screen based on the control of the control unit and a touch panel that accepts input operations from the user of the terminal device 10. In this embodiment, the touch panel display 13 is disposed approximately in the center of the terminal device 10.

[0017] The speaker 14 is an audio output device that outputs audio under the control of the control unit. In this embodiment, the speaker 14 is provided on the other end side of the terminal device 10.

[0018] The above-described positions of the microphone 12, the touch panel display 13, and the speaker 14 of the terminal device 10 are merely examples, and they may be arranged in other positions.

[0019] 1 shows a situation in which a user of a terminal device 10 is making a speech input to the terminal device 10. In FIG. 1, the user, for example, realizes that he or she is out of beer while at home and inputs a speech into the microphone 12 saying, "Next time I go out in my car, I'll buy a case of beer."

[0020] In this embodiment, the control unit of the terminal device 10 is configured to be able to recognize speech from the user of the terminal device 10. That is, the control unit 17 converts the speech input by the user's speech into text consisting of character strings.

[0021] In this embodiment, the control unit of the terminal device 10 identifies and extracts words indicating an action planned by the user (hereinafter also referred to as a planned action) as planned action information from the text generated by speech recognition of the input speech. The words indicating an action planned by the user are, for example, words indicating a movement by the user to a destination or words indicating a purchase of an item.

[0022] In this embodiment, the control unit of the terminal device 10 identifies and extracts, from the text generated by speech recognition of the input speech, a term indicating a means of transportation (hereinafter also referred to as a corresponding means of transportation) corresponding to the scheduled action indicated by the scheduled action information. That is, the control unit of the terminal device 10 identifies what means of transportation the user will use to perform the scheduled action.

[0023] In this embodiment, the control unit of the terminal device 10 associates words indicating the user's planned action as planned action information with the user's corresponding means of transportation, and stores them as an information set.

[0024] In this embodiment, the control unit of the terminal device 10 identifies the user's current means of transportation when the user starts traveling with the terminal device 10. If the information set contains a corresponding means of transportation that matches the identified current means of transportation of the user, the control unit of the terminal device 10 notifies the user of notification information including words indicating the planned action that corresponds to the corresponding means of transportation.

[0025] 2 is a block diagram showing an example of the configuration of a terminal device 10 according to Example 1. In this example, the terminal device 10 is a device in which a control unit 17, a communication unit 18, an input unit 19, an output unit 21, and a large-capacity storage device 22 cooperate with each other via a system bus 16.

[0026] The GPS receiver 15 is a receiver that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 15 transmits the received GPS signals to the control unit 17 via the input unit 19. The control unit 17 acquires location information of the terminal device 10 based on the GPS signals.

[0027] The control unit 17 is configured with a CPU (Central Processing Unit) 17A, a ROM (Read Only Memory) 17B, a RAM (Random Access Memory) 17C, etc., and functions as a computer. The CPU 17A reads and executes various programs stored in the ROM 17B and the large-capacity storage device 22, thereby realizing various functions.

[0028] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs stored in the mass storage device 22 (including a program for executing processing in the terminal device 10, which will be described later) can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0029] The communication unit 18 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 17. The communication unit 18 is, for example, a network interface card (NIC) for connecting to a network. The connection between the terminal device 10 and the network can be established, for example, by mobile communication such as 3G (3rd Generation), 4G (4th Generation), or 5G (5th Generation), or wireless communication such as Wi-Fi (registered trademark).

[0030] The input unit 19 is an interface unit that communicatively connects the control unit 17 with the microphone 12, the GPS receiver 15, and the touch panel display 13. In this embodiment, the control unit 17 acquires voice information indicating voice input from the user via the input unit 19. In other words, the control unit 17 functions as a user input information acquisition unit that acquires input information from the user.

[0031] The output unit 21 is an interface unit that is communicatively connected to the touch panel display 13 and the speaker 14. The control unit 17 can transmit a video or image signal to the touch panel display 13 via the output unit 21 to cause it to display, or transmit an audio signal to the speaker 14 to cause it to output sound.

[0032] The mass storage device 22 is configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc., and is a storage device that stores various programs such as an operating system and software for terminals.

[0033] A database for speech recognition (hereinafter referred to as a speech recognition DB) 22A of the mass storage device 22 is a database that stores language models, pronunciation dictionaries, etc. to be used for speech recognition of input speech. When a speech is input by the user, the control unit 17 performs speech recognition by referring to the speech recognition DB 22A, and converts the speech into text consisting of character strings.

[0034] [Extraction process of planned actions and corresponding means of transportation] An example of the process executed by the control unit 17 in this embodiment to extract words indicating the user's planned action and the corresponding means of transportation from the input voice will be described below. In the following description, it is assumed that the user has input a voice utterance such as "I'll buy a case of beer the next time I go out by car," as described above.

[0035] 3 is a table TB1 (hereinafter referred to as a scheduled activity list TB1) of an information set that associates the user's scheduled activity with words indicating corresponding means of transportation. The scheduled activity list TB1 is stored in the mass storage device 22 by the control unit 17, for example.

[0036] In this embodiment, the control unit 17 converts the speech input by the user into text consisting of a character string using the above-mentioned speech recognition DB 22 A. Specifically, the control unit 17 recognizes the speech input by the user, "I'll buy a case of beer the next time I go out by car," and converts it into text.

[0037] Then, the control unit 17 extracts, from the text generated by the speech recognition, words indicating the user's planned action and words indicating the corresponding means of transportation when performing the planned action. Specifically, the control unit 17 extracts the words "buy a case of beer" which are words indicating the user's planned action. The control unit 17 also extracts the words "by car" which are words indicating the user's corresponding means of transportation when performing the planned action.

[0038] The control unit 17 associates the extracted words indicating the user's planned behavior with words indicating the corresponding means of transportation, and stores them as an information set. Specifically, the control unit 17 identifies the words "by car," which are the words indicating the corresponding means of transportation, as indicating an automobile, and stores "buy a case of beer" and "automobile" in association with each other and with an information set ID, as shown in the scheduled behavior list TB1.

[0039] [Scheduled event notification processing] An example of a process of notifying a scheduled action executed by the control unit 17 in this embodiment will be described below. In the following description, it is assumed that after the user has performed the voice input, the user is traveling from home to the car together with the terminal device 10.

[0040] The control unit 17 identifies the user's current means of transportation when it is determined that the user has started moving, for example, based on the location information of the terminal device 10. That is, in this embodiment, the control unit 17 functions as an identification unit that identifies the user's current means of transportation.

[0041] In this embodiment, the control unit 17 acquires the current movement status of the user based on the position information of the terminal device 10. Specifically, the control unit 17 acquires, for example, the distance between two points, from a reference position during the movement of the terminal device 10 to a position a predetermined distance away, and identifies the user's current means of transportation based on the speed calculated from the distance and the time required to move between the two points, i.e., the user's movement speed.

[0042] In this embodiment, the control unit 17 determines that the user is traveling by car when the speed calculated when traveling several hundred meters from the user's home is equal to or greater than a predetermined speed, for example, with the user's home as the reference position. In other words, the control unit 17 determines that the user's current means of transportation is "automobile."

[0043] The control unit 17 refers to the scheduled action list TB1, and if there is a scheduled action of the user that corresponds to the identified user's current means of transportation, identifies the scheduled action as information that should be notified to the user. Specifically, the control unit 17 refers to the scheduled action list TB1, and identifies the information set indicating the identified "automobile," here the information set with information set ID "1," and identifies the scheduled action of the information set, "buy a case of beer," as information that should be notified to the user.

[0044] Here, the output of notification information in the terminal device 10 will be described with reference to Fig. 4. Fig. 4 is a diagram showing a situation in which the control unit 17 of the terminal device 10 notifies the user of a voice indicating the user's planned action. In this embodiment, the control unit 17 notifies the user of notification information including the planned action by using the words "buy a case of beer," which is the planned action identified above.

[0045] 4, the control unit 17 outputs a voice message such as "You were going to buy a case of beer when you went out in your car, right?" via the speaker 14 which is communicably connected to the output unit 21. That is, in this embodiment, the speaker 14 is a notification unit which performs notification processing in which the control unit 17 notifies the user of notification information.

[0046] As a result, while the user is traveling by car from home, the user can obtain notification information such as, "You were supposed to buy a case of beer when you left by car," etc. Then, the user can receive the obtained notification information and, for example, stop by a supermarket or the like before returning home from going out and remember to buy a case of beer.

[0047] An example of a specific operation of the terminal device 10 in this embodiment will be described below with reference to FIGS.

[0048] 5 is a flowchart showing the scheduled behavior storage routine RT1 executed by the control unit 17 of the terminal device 10. In the following, similar to the above description, it is assumed that the user inputs a voice saying, "Next time I go out in my car, I'll buy a case of beer."

[0049] The control unit 17 starts the scheduled behavior storage routine RT1, for example, when the power is turned on to the terminal device 10. Note that the scheduled behavior storage routine RT1 is repeatedly executed by the control unit 17 as long as the power of the terminal device 10 is turned on.

[0050] First, the control unit 17 determines whether or not a scheduled behavior storage start operation for processing to store the user's scheduled behavior has been received (step S101). The control unit 17 determines that an operation for the scheduled behavior storage processing has been performed, for example, when the user calls the wake word of the terminal device 10 or makes a predetermined voice input such as "Make a note!". When the control unit 17 determines that a scheduled behavior storage start operation for the scheduled behavior storage processing has not been received (step S101: NO), the control unit 17 ends the scheduled behavior storage routine RT1.

[0051] When the control unit 17 determines that the scheduled behavior storage start operation has been received (step S101: YES), it receives a voice input by speech from the user (step S102). For example, as described above, the control unit 17 acquires a voice input by speech from the user via the microphone 12 of the terminal device 10. Note that, in step S102, if there is no voice input from the user within a certain period of time, the scheduled behavior storage routine RT1 may be terminated.

[0052] After step S102, the control unit 17 performs speech recognition on the input speech using the speech recognition DB 22A of the mass storage device 22, and converts the speech into text (step S103). In this embodiment, the control unit 17 converts the speech input by the user, "I'll buy a case of beer the next time I go out in my car," into text consisting of a character string.

[0053] After step S103, the control unit 17 extracts, from the text as the speech recognition result, words indicating the user's planned action and words indicating the corresponding means of transportation that is the means of transportation when performing the planned action (step S104). Specifically, the control unit 17 extracts, from the text "Next time I go out by car, I will buy a case of beer", which is the words indicating the user's planned action, and "by car", which is the words indicating the corresponding means of transportation.

[0054] After step S104, the control unit 17 associates the words indicating the planned action extracted in step S104 with the words indicating the corresponding means of transportation, and stores them as an information set (step S105).

[0055] Specifically, the control unit 17 identifies the user's corresponding means of transportation as "automobile" from the phrase "by car" indicating the corresponding means of transportation. Then, as shown in the scheduled action list TB1, the control unit 17 associates the phrase "buy a case of beer" indicating the scheduled action with the phrase "automobile" indicating the corresponding means of transportation, assigns an information set ID to the phrase, and stores the association in the mass storage device 22. The control unit 17 ends the scheduled action storage routine RT1 after step S105.

[0056] Next, a description will be given of the scheduled action notification process executed by the control unit 17. Fig. 6 is a flowchart showing a scheduled action notification routine RT2 executed by the control unit 17 of the terminal device 10. In the following, similar to the above description, it is assumed that after the voice input, the user is traveling from home to his / her car with the terminal device 10.

[0057] The control unit 17 starts the scheduled behavior notification routine RT2, for example, when the power is turned on to the terminal device 10. Note that the scheduled behavior notification routine RT2 is repeatedly executed by the control unit 17 as long as the power of the terminal device 10 is on.

[0058] First, the control unit 17 determines whether the user has started moving based on the location information of the terminal device 10 obtained by, for example, a GPS signal (step S201). If the control unit 17 determines that the user has not started moving (step S201: NO), the control unit 17 ends the planned behavior notification routine RT2.

[0059] When the control unit 17 determines that the user has started moving (step S201: YES), the control unit 17 acquires the user's current movement status (step S202). In this embodiment, the control unit 17 acquires the user's movement speed based on the position information of the terminal device 10.

[0060] Specifically, as described above, the control unit 17 obtains the distance between two points, which is a predetermined reference point and a position several hundred meters away, based on the position information of the terminal device 10, and calculates and obtains the user's movement speed from the distance and the time required to move between the two points.

[0061] After step S202, the control unit 17 identifies the user's current means of transportation based on the current movement status of the terminal device 10, i.e., the user's current movement speed (step S203). In this embodiment, the control unit 17 identifies the user's current means of transportation as "automobile" when the user is moving along the movement route at a predetermined speed or faster.

[0062] After step S203, the control unit 17 refers to the scheduled action list TB1 and determines whether there is a corresponding means of transportation that matches the user's current means of transportation, i.e., whether there is a scheduled action corresponding to the user's current means of transportation (step S204). In other words, the control unit 17 is a determination unit that determines whether the user is traveling by the corresponding means of transportation. When the control unit 17 determines that there is no scheduled action corresponding to the current means of transportation (step S204: NO), it ends the scheduled action notification routine RT2.

[0063] When the control unit 17 determines that there is a scheduled action corresponding to the user's current means of transportation (step S204: YES), it notifies the user of the scheduled action corresponding to the current means of transportation via the notification unit while traveling by that means of transportation (step S205).

[0064] Specifically, the control unit 17 refers to the above-mentioned scheduled action list TB1, and identifies the information set indicating "car" identified as the corresponding means of transportation, in this case the information set with information set ID "1", and identifies the scheduled action of that information set, "buy a case of beer", as information that should be notified to the user.

[0065] Then, while traveling by "car," which is the corresponding means of transportation, the control unit 17 outputs notification information including the above-mentioned scheduled action from the speaker 14. Specifically, the control unit 17 outputs a voice message to the user via the speaker 14, such as "You were going to buy a case of beer when you went out by car, right?" The control unit 17 ends the scheduled action notification routine RT2 after step S205.

[0066] According to this embodiment, the above-mentioned scheduled action storage routine RT1 and scheduled action notification routine RT2 associate the user's scheduled action with the corresponding means of transportation from the input voice and store them as an information set, and also notify the user of the scheduled action to be notified, i.e., the information to be reminded, in a timely manner based on the user's current travel situation.

[0067] According to this embodiment, when a user is traveling with the terminal device 10 by a specific means of transportation, if there is something that can only be done by that means of transportation, such as transporting a heavy object by car, the user can input this information by voice and the system will notify the user of the planned action corresponding to the means of transportation at the appropriate time.

[0068] Therefore, for example, even if a user is traveling by a specific means of transportation, it is possible to prevent a situation in which the user is unable to perform the scheduled action because the scheduled action corresponding to the means of transportation cannot be acquired, resulting in a loss of time. Therefore, according to this embodiment, it is possible to remind the user of the scheduled action at an appropriate timing depending on the situation.

[0069] In this embodiment, the control unit 17 calculates the user's movement speed as the user's movement status based on the location information of the terminal device 10, and identifies the user's current means of transportation based on the movement speed, but this is not limiting. For example, the terminal device 10 may include a sensor such as an acceleration sensor and acquire speed information from the sensor.

[0070] Furthermore, the control unit 17 may be configured to be able to acquire map information, and may identify the user's current means of transportation based on the map information and the above-mentioned location information in addition to or instead of the above-mentioned traveling speed. Specifically, for example, the control unit 17 identifies a train as the means of transportation if the user is traveling along a railroad track, and identifies a car as the means of transportation if the user is traveling along a road at a predetermined speed or faster.

[0071] Furthermore, the control unit 17 may identify the user's current means of transportation based on an acceleration pattern detected by an acceleration sensor included in the terminal device 10, in addition to or instead of the above-mentioned traveling speed.

[0072] Specifically, the terminal device 10 may be configured to store in advance a plurality of reference acceleration patterns specific to each mode of transportation, which are acceleration patterns detected when traveling by various modes of transportation, in the mass storage device 22. The control unit 17 then compares the acceleration pattern detected by the acceleration sensor during the user's current travel with the reference acceleration patterns stored in the mass storage device 22, and identifies the mode of transportation as a car when it determines that the acceleration pattern detected by the acceleration sensor during the user's current travel has the highest similarity to the reference acceleration pattern corresponding to a car.

[0073] In the present embodiment, the control unit 17 calculates the user's movement speed as the user's movement status based on the location information of the terminal device 10, and identifies the user's current means of transportation based on the movement speed, but this is not limited to this. For example, the control unit 17 may identify the user's current means of transportation based on the user of the terminal device 10 and a companion, in addition to the movement speed.

[0074] Specifically, for example, when the terminal device 10 continuously detects one or more other terminal devices by receiving Bluetooth (registered trademark) radio waves emitted by other terminal devices having Bluetooth (registered trademark) functionality while the user is moving, it can be assumed that the owner of the one or more other terminal devices is moving together with the user of the terminal device 10.

[0075] For example, if the ID of another terminal device detected by the terminal device 10 through reception of Bluetooth (registered trademark) radio waves has already been registered in the terminal device 10, the control unit 17 considers the owner of the other terminal device to be a family member or acquaintance of the user of the terminal device 10, and determines that the owner is a companion of the user of the terminal device 10. If there are about 1 to 4 other registered terminal devices traveling together with the terminal device 10, it may be estimated that the number of people accompanying the user is about 1 to 4, and may also be determined that the user's current means of transportation is a car based on the travel route of the terminal device 10.

[0076] Furthermore, if the ID of another terminal device detected by the terminal device 10 by receiving Bluetooth (registered trademark) radio waves is not registered in the terminal device 10, the control unit 17 may consider the owner of the other terminal device to be not an acquaintance of the user of the terminal device 10, presume that the user of the terminal device 10 is traveling by public transportation, and may determine that the user's current means of transportation is a bus or a train based on the travel route of the terminal device 10.

[0077] In this embodiment, the control unit 17 outputs the notification information including the user's scheduled action as sound via the speaker 14, but this is not limited to this. For example, the control unit 17 may display the notification information including the user's scheduled action on the touch panel display 13 in addition to or instead of the speaker 14.

[0078] In this embodiment, the control unit 17 outputs the voice "You said you were going to buy a case of beer when you go out by car," via the speaker 14, but this is not limiting. For example, the control unit 17 may notify the user directly of the input voice "Next time you go out by car, I'll buy a case of beer," as notification information.

[0079] In this embodiment, the control unit 17 extracts the user's planned actions and corresponding means of transportation from the voice input by the user, but the voice may contain other information, and the control unit 17 may extract the other information.

[0080] For example, the voice input from the user may include words indicating the planned date and time of the action indicated by the user's planned action. For example, when the voice input from the user includes a travel date and time such as "At the supermarket on September 20th..." or "At the department store next Sunday...", the control unit 17 may output the notification information at a timing determined based on the date and time.

[0081] For example, the control unit 17 may be configured to output notification information when the user travels by the supported transportation means at a timing determined based on the date and time. In other words, the control unit 17 can prevent the output of notification information even if the user travels by the supported transportation means until the timing determined based on the date and time.

[0082] In this embodiment, the notification information notified to the user may have a predetermined notification period. For example, the control unit 17 may repeatedly notify the user of the notification information for a predetermined period when the user is performing a predictive behavior. Also, for example, the control unit 17 may not output the notification information after the predetermined period has elapsed.

[0083] In this embodiment, the speech recognition of the input speech performed by the control unit 17 may be performed by AI. For example, the speech input by the user's speech may be input to AI having a learning model that has trained a speech dictionary and a language model, and text consisting of character strings may be obtained as the output.

[0084] In this embodiment, the above-described scheduled behavior storage process and scheduled behavior notification process are performed by a smartphone as the terminal device 10. However, the present invention is not limited to this configuration as long as the device configuration is capable of performing the processes. For example, each configuration of the terminal device 10 may be applied to a terminal device that can be carried by a user, such as a tablet terminal.

[0085] Furthermore, for example, the above-mentioned scheduled behavior storage process and scheduled behavior notification process may be executed by a server device capable of communicating with the terminal device 10 instead of the terminal device 10, and the terminal device 10 may receive the results of the process from the server device.

[0086] Specifically, for example, the terminal device 10 may transmit a voice input from the user of the terminal device 10 to the microphone 12 to a server device, and the server device may perform a scheduled action storage process and a scheduled action notification process. The terminal device 10 may then notify the user of the notification information including the user's scheduled action transmitted from the server device via the speaker 14. The notification information including the user's scheduled action transmitted from the server device may also be configured to be viewable on other terminal devices such as smartphones and tablet terminals.

[0087] [Variations] Next, a modified example of Example 1 will be described. The modified example of Example 1 differs from Example 1 in the process of storing the user's planned behavior based on the difference in the content of the input voice, but other points, such as the configuration of the terminal device 10, are the same as those of Example 1. Note that in this modified example, a case will be described in which the above-mentioned planned behavior of the user is a transportation behavior of purchasing and transporting an item.

[0088] Fig. 7 is a diagram showing a voice input to the terminal device 10 serving as an information processing device. As in the first embodiment, Fig. 7 shows a state in which a user of the terminal device 10 inputs a voice by speaking to the terminal device 10. In Fig. 1, for example, the user realizes that he or she is out of beer while at home and inputs a voice saying "I will buy a case of beer by the end of this week" into the microphone 12.

[0089] [Processing to identify compatible means of transportation] An example of a process executed by the control unit 17 in this modification to identify words indicating corresponding means of transportation from the user's planned action from the input voice will be described below. In the following description, it is assumed that the user has input a voice by uttering "I will buy a case of beer by the end of this week," as described above. Note that the process of extracting the planned action is the same as in the first embodiment, and therefore will not be described again.

[0090] 8 is a table TB2 (hereinafter referred to as attribute information TB2) showing corresponding means of transportation determined based on the attributes of an item to be transported when the user's scheduled behavior is a transportation behavior of purchasing and transporting the item. The attribute information TB2 is stored in the mass storage device 22 by the control unit 17, for example.

[0091] In this modified example, when the input voice does not include a corresponding means of transportation corresponding to the user's planned action and the planned action is a transportation action of purchasing and transporting an item, the control unit 17 refers to the attribute information TB2 and identifies the corresponding means of transportation based on the attributes of the item to be transported.

[0092] In the attribute information TB2, as an example of an attribute of an object, the corresponding means of transportation is determined based on whether the object to be transported is small or light, or large or heavy, that is, based on the ease with which the user can transport it by themselves.

[0093] For example, if the items to be transported are small amounts of food or daily necessities, walking or cycling are considered to be suitable modes of transportation as they are easy for the user to transport on their own. On the other hand, if the items to be transported are furniture such as a desk or food items such as beverages purchased in cases, it is considered difficult for the user to transport them on their own, so a car is considered to be suitable modes of transportation.

[0094] Here, the identification of the corresponding transportation means will be described with reference to Fig. 9. Fig. 9 is a table TB3 showing an example of a scheduled event list (hereinafter referred to as scheduled event list TB3).

[0095] In this modified example, the user's scheduled action is a transportation action of purchasing and transporting an item, "buying a case of beer." Therefore, the control unit 17 identifies the item to be transported as "a case of beer," and identifies the corresponding transportation means as an automobile by referring to the attribute information TB2.

[0096] Then, as in Example 1, the control unit 17 associates the words "buy a case of beer" indicating the user's scheduled action with the words "car" indicating the identified corresponding means of transportation, and stores them with an information set ID, as shown in the scheduled action list TB3.

[0097] The following describes the scheduled action storage process executed by the control unit 17 in this modified example. Note that the scheduled action notification process is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0098] 10 is a flowchart showing the scheduled behavior storage routine RT3 executed by the control unit 17 of the terminal device 10. In the following, similar to the above description, it is assumed that the user inputs a voice saying, "I will buy one case of beer this week." Note that in the scheduled behavior storage routine RT3, steps S101 to S103 are the same as those in the first embodiment, and therefore description thereof will be omitted.

[0099] After step S103, the control unit 17 extracts words indicating the user's planned behavior from the text as the speech recognition result (step S301). Specifically, the control unit 17 extracts the words "buy one case of beer" indicating the user's planned behavior from the text "buy one case of beer by the end of this week."

[0100] In this modified example, as described above, if the input voice does not include a corresponding means of transportation corresponding to the user's planned action, and the planned action is a transportation action of purchasing and transporting an item, the corresponding means of transportation is identified based on the attributes of the item to be transported by referring to attribute information TB2.

[0101] Therefore, after step S301, the control unit 17 identifies the corresponding means of transportation from the words indicating the user's planned behavior (step S302). Specifically, the control unit 17 identifies the item to be transported as "one case of beer" from the words "buy one case of beer" indicating the user's planned behavior, and identifies the corresponding means of transportation as an automobile by referring to the attribute information TB2.

[0102] After step S302, the control unit 17 associates the term indicating the planned action extracted in step S301 with the term indicating the corresponding means of transportation, and stores them as an information set (step S105). After step S105, the control unit 17 ends the planned action storage routine RT3.

[0103] According to this modification, when the user's planned action included in the input voice from the user is a transportation action of purchasing and transporting an item, even if the input voice does not include a word indicating a corresponding transportation means, it is possible to identify the corresponding transportation means based on the attributes of the item to be transported. Therefore, according to this modification, when a transportation action of purchasing and transporting an item is performed, the corresponding transportation means is automatically identified without the need to input a specific transportation means.

[0104] As described above, the control unit 17 notifies the user of the scheduled action corresponding to the means of transportation at an appropriate timing. Therefore, according to this modification, it is possible to remind the user of the scheduled action at an appropriate timing depending on the situation.

[0105] The control routines shown in the above-described embodiment and modified examples are merely examples, and can be appropriately selected and modified depending on the application or conditions of use. [Explanation of symbols]

[0106] 10 Terminal Equipment 12. Microphone 13 Touch panel display 14 speakers 15 GPS receiver 16 System Bus 17 Control Unit 18 Communications Department 19 Input section 21 Output section 22 Mass storage

Claims

1. a user input information acquisition unit that acquires input information from the user, the input information including planned action information indicating an action that the user plans to take; an identification unit that identifies a corresponding means of transportation that is a means of transportation that corresponds to the action planned by the user based on the input information; a determination unit that determines whether the user is traveling by the corresponding transportation means while the user is traveling; a control unit that performs a notification process to notify the user of notification information including the planned action information via a notification unit while the user is traveling by the corresponding transportation means when it is determined that the user is traveling by the corresponding transportation means, The information processing device is characterized in that, when the action planned by the user is a transportation action to transport an object, the identification unit identifies one or more means of transportation determined based on the attributes of the object to be transported as the corresponding means of transportation.

2. The information processing device according to claim 1, characterized in that the control unit identifies the date and time when the user plans to perform the action indicated by the planned action information from the input information, and performs a notification process to notify the user of the notification information via the notification unit when travel by the corresponding transportation means is performed at a timing determined based on the date and time.

3. 3. The information processing apparatus according to claim 1, wherein the input information is a voice input by the user's speech.

4. 4. The information processing apparatus according to claim 1, wherein the determination unit determines whether the user is traveling by the supported transportation means based on a companion traveling with the user.

5. 5. The information processing apparatus according to claim 1, wherein the control unit notifies the user of the notification information through a voice in the notification process.

6. The information processing apparatus according to claim 5 , wherein the control unit notifies the user of the input information in the notification process.

7. 7. The information processing apparatus according to claim 1, wherein the control unit visually notifies the user of the notification information in the notification process.

8. An information processing method executed by an information processing device, a user input information acquisition step of acquiring input information from the user including planned action information indicating an action planned by the user; a step of identifying a corresponding transportation means that is a transportation means that corresponds to the action planned by the user based on the input information; a determination step of determining whether the user is traveling by the corresponding transportation means while the user is traveling; a control step of performing a notification process in which, when it is determined that the user is traveling by the corresponding transportation means, notification information including the planned action information is notified to the user via a notification unit while the user is traveling by the corresponding transportation means; An information processing method characterized in that, in the identification step, when the action planned by the user is a transportation action to transport an object, one or more means of transportation determined based on the attributes of the object to be transported are identified as the corresponding means of transportation.

9. A program to be executed by a computer, a user input information acquisition step of acquiring input information from the user including planned action information indicating an action planned by the user; a step of identifying a corresponding transportation means that is a transportation means that corresponds to the action planned by the user based on the input information; a determination step of determining whether the user is traveling by the corresponding transportation means while the user is traveling; a control step of performing a notification process in which, when it is determined that the user is traveling by the corresponding transportation means, notification information including the planned action information is notified to the user via a notification unit while the user is traveling by the corresponding transportation means; A program to be executed by the computer, characterized in that in the identification step, when the action planned by the user is a transportation action to transport an object, one or more means of transportation determined based on the attributes of the object to be transported are identified as the corresponding means of transportation.

10. A recording medium storing the program according to claim 9.

Images