Information processing device, information processing method, and information processing program

Abstract

It accurately detects and notifies users of items they have left behind. [Solution] The information processing device includes: an acquisition unit that acquires item carrying information, which associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area in real space or virtual space; a setting unit that, based on the acquired item carrying information, sets as target objects items that the user has carried when leaving the target area a predetermined number of times or more within a predetermined time period; and a notification unit that determines whether the user is carrying the target object when leaving the target area, and if it is determined that the user is not carrying the target object, notifies the user of the lost item information.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] There is disclosed a technique for photographing a user entering a facility or the like by an imaging device installed in the facility or the like, detecting an article worn by the user from the photographed image, and preventing the user from forgetting something when the user exits or the like based on the detection result (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-described technology, it is required to accurately detect a user's forgotten item.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of accurately detecting and notifying a user's forgotten item.

Means for Solving the Problems

[0006] The information processing device according to the present invention includes: an acquisition unit that acquires item carrying information, which associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries the item in a target area in real space or virtual space; a setting unit that, based on the acquired item carrying information, sets the item as a target object if the user has left the target area with the item for a predetermined number of times or more during a predetermined time period; and a notification unit that determines whether the user is carrying the target object when leaving the target area, and if it is determined that the user is not carrying the target object, notifies the user of lost item information.

[0007] The information processing method according to the present invention includes the steps of: acquiring item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area in real space or virtual space; setting the item as a target object based on the acquired item carrying information, and determining whether the user is carrying the item when leaving the target area a predetermined number of times or more during a predetermined time period; and determining whether the user is carrying the target object when leaving the target area, and if it is determined that the user is not carrying the target object, notifying the user of lost item information.

[0008] The information processing program according to the present invention causes a computer to perform the following steps when a user carries an item in a target area in real space or virtual space: a process to acquire item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried; a process to set as an object an item that the user has carried when leaving the target area a predetermined number of times or more during a predetermined time period, based on the acquired item carrying information; and a process to determine whether the user is carrying the object when leaving the target area, and if it is determined that the user is not carrying the object, a process to notify the user of lost item information. [Effects of the Invention]

[0009] According to the present invention, it is possible to accurately detect and notify users of items they have left behind. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic functional block diagram showing an example of an information processing device according to the embodiment. [Figure 2] Figure 2 shows an example of an image captured in real space. [Figure 3] Figure 3 shows an example of an image captured in real space. [Figure 4] Figure 4 shows an example of an image captured in real space. [Figure 5] Figure 5 shows an example of information stored in the memory unit. [Figure 6] Figure 6 shows an example of an image captured in real space. [Figure 7] Figure 7 shows an example of how to notify a user of lost items. [Figure 8] Figure 8 is a flowchart showing an example of the process of setting an object in the information processing device according to the embodiment. [Figure 9] Figure 9 is a flowchart showing an example of the process used when an information processing device according to the embodiment notifies a user of lost item information. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the information processing apparatus, information processing method, and information processing program according to the present invention will be described with reference to the drawings. However, the present invention is not limited by these embodiments. Furthermore, the components in the following embodiments include those that are easily substituted or substantially identical to those that are easily substituted by those skilled in the art.

[0012] FIG. 1 is a functional block diagram schematically showing an example of an information processing apparatus 100 according to the present embodiment. When a user forgets something in a real space such as home or workplace, or in a virtual space such as a game, the information processing apparatus 100 shown in FIG. 1 notifies the user of the forgotten item information.

[0013] As shown in FIG. 1, the information processing apparatus 100 includes a communication unit 10, a control unit 20, and a storage unit 30. The information processing apparatus 100 may be a portable information terminal such as a smartphone, a tablet, or a notebook personal computer, or may be a wearable device such as a glasses type or a watch type, or may be a stationary terminal such as a desktop personal computer.

[0014] The communication unit 10 performs wired or wireless communication with an external device. The communication unit 10 includes an interface such as a network interface card.

[0015] The control unit 20 performs various information processes. The control unit 20 includes a processor such as a CPU (Central Processing Unit) and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 20 has an acquisition unit 21, a setting unit 22, and a notification unit 23.

[0016] The acquisition unit 21 acquires article carrying information. The article carrying information is information in which the name of an article carried by a user, the carrying date and time when the article was carried, and the carrying position where the article was carried are associated with each other when the user carries an article in a target area in a real space or a virtual space.

[0017] Based on the acquired article carrying information, when the setting unit 22 detects an article that has left the target area while being carried by the user for a predetermined number of times or more in a predetermined time period, the setting unit 22 sets the article as a target object. The setting unit 22 can set an importance level according to the number of times the target object is carried. For example, the setting unit 22 can be set so that the importance level is higher for a target object carried more times.

[0018] When the user leaves the target area, the notification unit 23 determines whether the user is carrying an object. If the notification unit 23 determines that the user is not carrying an object in this determination, it notifies the user of the forgotten item information. The notification unit 23 determines that the user is not carrying an object when the time when the user leaves the target area does not correspond to the carrying time zone calculated based on the carrying date and time, and the user is not wearing the object. The carrying time zone can be, for example, a time zone calculated to include the plurality of carrying dates and times when a plurality of carrying dates and times are detected for each item. Also, when an importance level is set for the object, the notification unit 23 can notify the forgotten item information based on the importance level of the object.

[0019] The storage unit 30 stores various programs, information such as data, applications, etc. for performing the processing in the control unit 20. The storage unit 30 includes a storage such as a HDD (Hard Disk Drive) or a SSD (Solid State Drive). The storage unit 30 stores the above-described item carrying information.

[0020] The storage unit 30 stores an information processing program that causes a computer to execute a process of acquiring item carrying information in which the name of the item carried by the user, the carrying date and time when the item was carried, and the carrying position where the item was carried are associated with each other when the user carries an item in the target area of the real space or the virtual space, a process of setting, as an object, an item that the user has left the target area while carrying it a predetermined number of times or more in a predetermined time zone based on the acquired item carrying information, and a process of determining whether the user is carrying the object when the user leaves the target area and notifying the user of the forgotten item information if it is determined that the user is not carrying the object.

[0021] In the information processing device 100, the processor in the control unit 20 reads various programs and loads them into memory, thereby executing information processing corresponding to the functions of each of the above-mentioned units. Examples of various programs include programs received by the communication unit 10, programs stored in the storage unit 30, and programs recorded on an external recording medium. The information processing device 100 functions as an information processing device (computer) that executes various information processing. Note that other information processing devices different from the information processing device 100 may execute various programs, or the information processing device 100 and other information processing devices may cooperate to execute various programs.

[0022] Next, an example of the operation of the information processing device 100 configured as described above will be explained. First, an example in real space will be explained. The target area in real space can be, for example, a place that the user visits on a daily basis, such as their home or workplace. In this case, the user is photographed by a shooting device 40, such as a camera fixed inside the home or workplace, or a camera worn by the user, and the shooting results are input to the information processing device 100. In this embodiment, the shooting device 40 can generate shooting results that associate the captured image with the date and time of shooting.

[0023] In the information processing device 100, the acquisition unit 21 acquires, from the shooting results of the shooting device 40, a captured image of the user's hands, and the shooting date and time and shooting location associated with the captured image, as captured image information.

[0024] Figures 2 to 4 show examples of images captured in real space. The acquisition unit 21 can identify an item M1 held by the user in their hand, as shown in Figure 2, using, for example, known image recognition processing, and can detect the name of the identified item M1.

[0025] Furthermore, as shown in Figure 3, the acquisition unit 21 can identify the item M1 as the item M1 carried by the user through image processing or the like, and detect its name, if it detects that the user has picked up the item M1 and then placed it in a storage compartment B1 such as a bag or clothing pocket, or if it detects that the user has picked up the item M1 and then put it on their person.

[0026] Furthermore, the acquisition unit 21 acquires the date and time of carrying and the location of carrying the item M1 based on the captured image. The date and time of carrying can be detected based on the date and time of capture of the captured image information. The location of carrying can be determined, for example, by calculating the distance between the shooting device 40 and the item being photographed based on the captured image, and then determining the location based on the calculation result and the shooting location of the shooting device 40. In addition, if the user is carrying the information processing device 100 and the information processing device 100 is equipped with a location detection unit that receives GPS signals and determines its position, the location of carrying may be acquired based on the detection result of the location detection unit.

[0027] In this way, the acquisition unit 21 can acquire item carrying information based on the captured image information.

[0028] The setting unit 22 detects, based on the acquired item carrying information, whether the user has left the target area R with the item for a predetermined number of times or more within a predetermined time period. As shown in Figure 4, the setting unit 22 can detect that the user has left the target area R, for example, when the user fades out of the captured image. Alternatively, the setting unit 22 may detect that the user has left the target area R based on the position detected by the position detection unit described above. In the example in Figure 4, the user has left the target area R without carrying item M2.

[0029] Figure 5 shows an example of information stored in the memory unit 30. The setting unit 22 can store in the memory unit 30, for example, the name of the item carried by the user, the date and time of carrying, the time period of carrying, the location of carrying, and the number of times the item was left the target area R, in association with each other, as shown in Figure 5. The items M1 and M2 shown in Figure 5 are assumed to be the same items M1 and M2 shown in Figures 2 to 4. In Figure 5, items with "〇" in the target item column are indicated as having been set as a target item, while items with "-" in the target item column are indicated as not having been set as a target item.

[0030] The setting unit 22 detects that the user has left the shooting location carrying a specific item more than a predetermined number of times within a predetermined time period, and sets that item as the target object. When setting an object, the setting unit 22 can set an importance level according to the number of times the item has been carried. For example, the setting unit 22 can set the importance level to be higher for objects that have been carried more often. The setting unit 22 can store the object and its importance level in the storage unit 30. Figure 5 shows an example where items M1 and M2 are set as target objects. In the following example, the importance level of item M1, "Importance Level 1," is assumed to be higher than the importance level of item M2, "Importance Level 2."

[0031] Next, we will explain the operation of notifying the user of lost item information. In this case, as described above, the shooting result of an external shooting device 40, whose shooting position is predetermined, is input to the information processing device 100. The acquisition unit 21 acquires the captured image of the user's hands, the shooting date and time and shooting position associated with the captured image, as captured image information from the shooting result of the shooting device 40.

[0032] The notification unit 23 acquires user movement information indicating whether the user has left the target area R based on the acquired captured image information. Based on the user movement information, if the notification unit 23 detects that the user has left the target area R, it determines whether the user is carrying an object. The notification unit 23 can recognize whether the user is carrying an item by performing known image recognition processing on the captured image, similar to the above processing of the setting unit 22. The notification unit 23 compares the item recognized as being carried by the user with the object stored in the storage unit 30 and detects whether the item is an object. If the notification unit 23 detects that the item is an object, it determines that the user is carrying an object.

[0033] Furthermore, the notification unit 23 can determine that the user is not carrying an object if the user is not wearing the object corresponding to the time the user leaves the target area R and the time zone calculated based on the carrying date and time. Figure 6 is a diagram showing an example of a captured image in real space. Figure 6 shows the case where the user leaves the target area R without carrying items M1 and M2 at a time within the range of "Time Zone 1". As shown in Figure 5, the time zones for carrying items M1 and M2 are both "Time Zone 1". In this case, the notification unit 23 can determine that the user is not carrying an object because the user is not wearing items M1 and M2 corresponding to "Time Zone 1".

[0034] The notification unit 23 notifies the user of lost item information if it determines that the user is not carrying the target item when leaving the target area R. Lost item information is, for example, information that makes the user aware that they have forgotten to take the target item with them when leaving the shooting location. Figure 7 shows an example of how lost item information is notified to the user. In Figure 7, the lost item information is notified to the display device 50 as text information. In this case, for item M1, which is of high importance, "You have forgotten item M1" is displayed, and for item M2, which is of lower importance than item M1, "(Please also check item M2)" is displayed in smaller letters than the lost item information for item M1, in parentheses. In this way, the way lost item information is notified may be changed according to the importance of the target item. The notification unit 23 may also notify the user of lost item information by, for example, outputting it as image information to the display device 50, outputting it as audio information to the audio output device 60, or outputting it as vibration to the vibration device 70. The display device 50, the audio output device 60, and the vibration device 70 may be mounted on the information processing device 100, or they may be provided externally to the information processing device 100.

[0035] Next, an example of a virtual space will be described. In this embodiment, the virtual space will be described using, for example, a virtual space set up in an application such as a game. In this case, the target area can be any area in which an avatar corresponding to a user can move. The virtual space can be displayed by, for example, a display device 50. In this embodiment, for example, by recording the video of the virtual space displayed on the display device 50, captured image information of the hands of the user or the avatar corresponding to the user can be obtained. In this case, the acquisition unit 21 can acquire item carrying information based on the captured image information, just as in the case of real space, and perform the same processing as described above.

[0036] Furthermore, in the virtual space, the acquisition unit 21 can acquire item carrying information and user movement information without using captured image information. That is, in the virtual space, date and time information is set in advance, and the names of the items placed in the virtual space and the positions of the items in the virtual space are set. Also, in the virtual space, the position of the user's avatar is set according to the user's operations, etc. Therefore, when the user's avatar carries an item in the virtual space, the acquisition unit 21 can acquire item carrying information by acquiring the setting information of the item's name, date and time of carrying, and carrying position, even without recording the video of the virtual space. Also, when the user leaves the target area, the acquisition unit 21 can acquire user movement information based on the setting information regarding the user's position in the virtual space, even without recording the video of the virtual space.

[0037] Examples of such virtual spaces include, for instance, when an avatar corresponding to a user carries an item (object) in a game space (virtual space), or when the avatar leaves the target area. When acquiring item carrying information, the acquisition unit 21 can acquire the carrying date and time based on the date and time information set in the game, and can acquire the name and carrying location of the item based on the item's setting information in the game. Furthermore, when acquiring user movement information, the acquisition unit 21 can detect that the user has moved from the target area based on the setting information regarding the avatar's position in the game. In the example of a virtual space, when item carrying information and user movement information are acquired, processing similar to that in the real world can be performed.

[0038] Figure 8 is a flowchart showing an example of the process for setting an object in the information processing device 100 according to this embodiment. As shown in Figure 8, when a user carries an item in the target area R in real space or virtual space, the acquisition unit 21 acquires item carrying information that associates the name of the item carried by the user with the date and time the item was carried and the location where the item was carried (step S10).

[0039] The setting unit 22 detects, based on the acquired item carrying information, whether or not there is an item that the user has left the target area R with while carrying it more than a predetermined number of times during a predetermined time period (step S20). If it is detected in step S20 that the user has left the target area R with an item more than a predetermined number of times during a predetermined time period (Yes in step S20), the item is set as the target object (step S30) and the process ends. If it is not detected in step S20 that the user has left the target area R with an item more than a predetermined number of times during a predetermined time period (No in step S20), step S30 is skipped and the process ends.

[0040] Figure 9 is a flowchart showing an example of the process used when the information processing device 100 according to this embodiment notifies the user of lost item information. As shown in Figure 9, the notification unit 23 acquires user movement information indicating whether or not the user has left the target area R (step S110).

[0041] The notification unit 23 determines, based on the acquired user movement information, whether the user took the object with them when leaving the target area R (step S120). If, in step S120, it is determined that the user did not take the object with them when leaving the target area R (No in step S120), the notification unit 23 notifies the user of the lost item information (step S130) and terminates the process. On the other hand, if, in step S120, it is determined that the user took the object with them when leaving the target area R (Yes in step S120), step S130 is skipped and the process terminates.

[0042] As described above, the information processing device 100 according to this embodiment includes: an acquisition unit 21 that acquires item carrying information, which associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area R in real space or virtual space; a setting unit 22 that sets as an object an item an item the user has carried when the user has left the target area R a predetermined number of times or more during a predetermined time period, based on the acquired item carrying information; and a notification unit 23 that determines whether the user is carrying an object when leaving the target area R, and if it is determined that the user is not carrying an object, notifies the user of the lost item information.

[0043] The information processing method according to this embodiment includes the steps of: acquiring item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area R in real space or virtual space; setting an item as a target object if the user has left the target area R with it for a predetermined number of times or more during a predetermined time period, based on the acquired item carrying information; and determining whether the user is carrying the target object when leaving the target area R, and if it is determined that the user is not carrying the target object, notifying the user of lost item information.

[0044] The information processing program according to this embodiment causes the computer to perform the following processes when a user carries an item in a target area R in real space or virtual space: acquiring item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried; setting an item as a target object if the user has left the target area R with it for a predetermined number of times or more during a predetermined time period, based on the acquired item carrying information; and determining whether the user is carrying the target object when leaving the target area R, and if it is determined that the user is not carrying the target object, notifying the user of the lost item information.

[0045] According to this configuration, based on item carrying information, items that have been carried and left the target area more than a predetermined number of times within a predetermined time period are set as target items. When a user leaves the target area, it is determined whether or not they are carrying the target item, and if it is determined that they are not carrying it, the user is notified of the lost item. This allows for accurate detection and notification of lost items by users.

[0046] In the information processing device 100 according to this embodiment, the notification unit 23 determines that the user is not carrying the object if the user is not wearing the object that corresponds to the time the user leaves the target area R and the time period of carrying the object calculated based on the carrying date and time.

[0047] This configuration allows for accurate detection of whether the user is carrying the target object.

[0048] In the information processing device 100 according to this embodiment, the setting unit 22 sets an importance level for the object according to the number of times it has been carried, and the notification unit 23 notifies the user of lost item information based on the importance level of the object.

[0049] This configuration allows for priority notification of important lost item information to the user.

[0050] The technical scope of the present invention is not limited to the embodiments described above, and modifications can be made as appropriate without departing from the spirit of the invention. For example, in the above embodiment, the notification unit 23 was described as being able to determine that the user is not carrying an object if the user is not wearing the object corresponding to the time the user leaves the target area R and the time zone calculated based on the carrying date and time, but it is not limited to this. For example, the notification unit 23 may be configured to determine that the user is not carrying an object if the user is not wearing the object corresponding to the time or day of the week the user leaves the target area R and the time zone or day of the week calculated based on the carrying date and time. Similarly, in the above embodiment, the setting unit 22 can set an importance level according to the number of times an object has been carried when setting an object. For example, the setting unit 22 was described as being able to set an importance level such that the more times an object has been carried, the higher its importance level, but it is not limited to this. For example, the setting unit 22 can set an importance level according to the day of the week an object has been carried when setting an object. For example, the setting unit 22 may be configured to set an importance level such that the more times an object has been carried, the higher its importance level. [Explanation of symbols]

[0051] B1...Storage unit, M1, M2...Items, R...Target area, 10...Communication unit, 20...Control unit, 21...Acquisition unit, 22...Setting unit, 23...Notification unit, 30...Storage unit, 40...Photography device, 50...Display device, 60...Audio output device, 70...Vibration device, 100...Information processing device

Claims

1. An acquisition unit acquires item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area in real space or virtual space. A setting unit that, based on the acquired item carrying information, sets the item as the target object if the user has left the target area while carrying it a predetermined number of times or more during a predetermined time period, When the user leaves the area, a notification unit determines whether or not the user is carrying the object, and if it determines that the user is not carrying the object, the notification unit notifies the user of the lost item. An information processing device equipped with the following features.

2. The notification unit determines that the user is not carrying the object if the user is not wearing the object that corresponds to the time the user leaves the target area and the time period of the mobile phone calculated based on the mobile phone date and time. The information processing apparatus according to claim 1.

3. The setting unit sets an importance level for the object according to the number of times it has been carried. The notification unit notifies the lost item information based on the importance of the item. The information processing apparatus according to claim 1.

4. When a user carries an item in a target area in real space or virtual space, the steps include obtaining item carrying information that associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried; Based on the acquired information about the item being carried, the user sets the item as the target object if the user has left the target area while carrying it for a predetermined number of times or more during a predetermined time period. When the user leaves the area, the system determines whether or not they are carrying the object, and if it is determined that they are not carrying the object, it notifies the user of the lost item. Information processing methods including

5. A process to acquire item carrying information, which associates the name of the item carried by the user, the date and time the item was carried, and the location where the item was carried, when the user carries an item in a target area in real space or virtual space. Based on the acquired item carrying information, the process of setting the item as the target object if the user has left the target area while carrying it a predetermined number of times or more during a predetermined time period, When the user leaves the area, the system determines whether or not they are carrying the object, and if it determines that they are not carrying the object, it notifies the user of the lost item. An information processing program that causes a computer to execute something.

Images