Information processing method, information processing device, and information processing program

Abstract

This information processing device: acquires a current image captured by an imaging device in an entranceway of a building; acquires related information connected to the current image; acquires a past image, the related information of which matches or is similar to that of the current image; extracts differences between the current image and the past image; generates, on the basis of the extracted differences, a task request requesting a task for assisting a user included in the current image; and outputs the generated task request.

Description

[Technical Field]

[0001] Information processing method, information processing device, and information processing program [Background technology]

[0002] Patent document 1 discloses an entrance system that captures an image of a user standing on a landing at an entrance, identifies the user ID from the captured image, measures the user's body surface temperature using a thermometer installed at the entrance, obtains weather information taking the user's home into consideration from a weather forecast provider's server, and suggests the most suitable clothing items for the user based on the user ID, body surface temperature, and weather information.

[0003] However, the technology described in Patent Document 1 does not extract the differences between the current entrance image and the past entrance image, and therefore further improvement is needed to provide support that is suitable for the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-301973 Summary of the Invention

[0005] An object of the present disclosure is to provide a technology that provides appropriate assistance to a user based on the difference between a current image and a past image.

[0006] An information processing method in one aspect of the present disclosure is an information processing method in an information processing device, in which a processor of the information processing device acquires a current image captured by an imaging device at the entrance of a building, acquires related information related to the current image, acquires a past image whose related information matches or is similar to that of the current image, extracts differences between the current image and the past image, and generates a task request to request a task to assist a user included in the current image based on the extracted differences, and outputs the generated task request.

[0007] According to the present disclosure, it is possible to provide support suited to the user based on the difference between the current image and the past image. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of an overall configuration of an information processing system according to a first embodiment of the present disclosure. [Figure 2] 2 is a block diagram showing an example of the configuration of the information processing device shown in FIG. 1. FIG. [Figure 3] FIG. 10 is a diagram illustrating an example of the data configuration of an image information table. [Figure 4] FIG. 2 is a diagram illustrating an example of an image captured by a camera. [Figure 5] 4 is a flowchart showing an example of processing performed by the information processing device according to the first embodiment of the present disclosure. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of an information processing device according to a second embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a data configuration of a supporter table. [Figure 8] 10 is a flowchart showing an example of processing by an information processing device according to a second embodiment of the present disclosure. [Figure 9] FIG. 11 is a diagram illustrating an example of a configuration of an information processing device according to a third embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating an example of a data configuration of a completion time table. [Figure 11] FIG. 10 is a diagram illustrating an example of a data configuration of a required time table. [Figure 12] FIG. 10 is an explanatory diagram of a fourth embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a configuration of an information processing device according to a fifth embodiment of the present disclosure. [Figure 14] FIG. 10 is a diagram illustrating an example of a data configuration of a task determination table. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Background to one aspect of the present disclosure) A monitoring system is being considered that monitors children by notifying the parents of the images taken by a camera installed at the entrance of a home when it detects that the child has left the house from images captured by the camera.

[0010] Parents can check whether their child has left anything behind by looking at the image in the notification. For example, if the weather forecast for today indicates that it will rain when the child is walking home from school, but the image in the notification shows that the child does not have an umbrella, the parent can confirm that the child has forgotten the umbrella and can bring it to the child.

[0011] However, even if a parent notices from the image that their child has left something behind, they may be busy with work or other tasks, making it difficult for the parent to take action to return the item to their child. In this case, the child may return home soaked in the rain without an umbrella. Seeing their child soaked in the rain, the parent may feel guilty for not being able to return the umbrella to their child. To avoid such situations, it is desirable to realize a service that automatically detects children's forgotten items and returns them to the child based on the detection results.

[0012] For example, the aforementioned Patent Document 1 discloses that the body surface temperature of a user standing on a landing at a front door is measured with a thermometer, and if the body surface temperature exceeds a threshold, the outdoor temperature threshold for suggesting that the user wear a coat is increased. However, in Patent Document 1, although a camera captures an image of a user standing at a front door, the captured image is only used to identify the user's ID, and the current image is not compared with past images with matching or similar related information. Therefore, further improvement is needed to provide support appropriate for the user, such as returning a forgotten item to the user in the image.

[0013] The inventor then discovered that by comparing a current image taken at the entrance with a past image in which related information such as a weather forecast is the same or similar to the current image, it is possible to automatically detect items that a child should have with them today as lost items, and that such a service can be realized by generating a task for a third party to return the lost items to the child.

[0014] Furthermore, the inventor has discovered that by comparing such a current image with a past image having the same or similar related information, it is possible to provide not only a service for returning lost items to the person in the image, but also a service for providing various support to the user in the image. For example, by comparing a current image with a past image taken at the same time, it is possible to detect a change in the time of sunset from a change in outdoor brightness, and to realize a service for sending a message urging children to come home early.

[0015] The present disclosure has been made based on these findings.

[0016] An information processing method in one aspect of the present disclosure is an information processing method in an information processing device, in which a processor of the information processing device acquires a current image captured by an imaging device at the entrance of a building, acquires related information related to the current image, acquires a past image whose related information matches or is similar to that of the current image, extracts differences between the current image and the past image, and generates a task request to request a task to assist a user included in the current image based on the extracted differences, and outputs the generated task request.

[0017] With this configuration, a past image whose related information matches or is similar to that of the current image is acquired, and the differences between the acquired past image and the current image are extracted, so that the past image captured under conditions where the related information matches can be regarded as the correct answer and the differences from the current image can be extracted. As a result, information that should be used to support the user and is included in the current image can be accurately identified from these differences, and support appropriate for the user can be realized.

[0018] In the information processing method, the difference may be an item that the user should have, and the task may be a task of transporting the item that the user should have.

[0019] This configuration can provide assistance in returning lost items to users.

[0020] In the information processing method, the difference may be a change in outdoor brightness, and the task may be a task of notifying the user of a message regarding returning home.

[0021] According to this configuration, a message regarding returning home can be notified to the user based on a change in outdoor brightness, thereby realizing a notification urging the user to return home earlier because it is getting dark, for example.

[0022] In the information processing method, the difference may be a combination of users when they return home, and the task may be a task of ordering food or a task of controlling a device.

[0023] According to this configuration, a task for ordering food or a task for controlling a device is executed depending on the difference in the user combination when the user returns home. For example, a task that a user who usually returns home with the user would perform after returning home is automatically executed, thereby providing support to the user when they return home.

[0024] In the information processing method, the related information may include weather information.

[0025] According to this configuration, the differences between the current image and past images whose weather information matches or is similar to the current image are extracted, so that items that the user should have with them can be detected based on the current weather information.

[0026] In the above information processing method, the task request may be generated using a task decision rule in which the content of the difference and the content of the task are associated in advance.

[0027] According to this configuration, a task is determined using a task determination rule in which the content of the difference and the content of the task are associated in advance, so that an appropriate task can be quickly determined in accordance with the difference.

[0028] In the above information processing method, when outputting the task request, an acceptance request is output to encourage the assistant assisting the first user to accept the task, and the task request may be output when consent information indicating that the assistant has agreed to the acceptance request is obtained.

[0029] According to this configuration, a task request can be made while respecting the wishes of the support candidate.

[0030] In the above information processing method, when outputting the task request, a deadline for executing the task may be set based on the content of the difference and user information indicating the user, and the deadline may be included in the acceptance request.

[0031] According to this configuration, a deadline for executing a task is set according to the content of the difference and the user included in the photograph, and the set deadline is included in the acceptance request, so that the supporter can be presented with information to decide whether or not to accept the support.

[0032] In the above information processing method, the execution deadline may be determined to be a time before the required time for the task, which is predetermined depending on the supporter and the user, from the completion time of the task, which is predetermined depending on the user information and the content of the difference.

[0033] According to this configuration, the execution deadline is set to a time that is before the required time for the task from the completion time of the task, so that the task can be completed by the completion time taking into consideration the required time for the task.

[0034] In the above information processing method, the content of the difference may be belongings that the user should have with them, the completion time may be a time determined based on the time when the user starts using the belongings that the user should have with them, and the required time may be the time required to transport the belongings that the user should have with them.

[0035] According to this configuration, the belongings that the user should have can be delivered to the user by the time when the user starts using the belongings.

[0036] In the above information processing method, the task may be performed by a supporter other than the user.

[0037] According to this configuration, the supporter can provide support to the user.

[0038] In the above information processing method, the past image may be further associated with information indicating whether or not a task request has been made, and in extracting the differences, a past image for which no task request has been made may be identified based on the information indicating whether or not a task request has been made, and the differences between the identified past image and the current image may be extracted.

[0039] According to this configuration, the identified past images are images for which no task requests have been made, so that the identified past images are more likely to contain information that should support the user, and an appropriate task can be determined for the user.

[0040] In the above information processing method, when extracting the differences, if the past images whose related information matches or is similar to that of the current image consist of multiple past images, it may be determined whether or not there is a difference between each past image and the current image, and if the number of past images determined to have a difference exceeds the number of past images determined to have no difference, it may be determined that there is a difference.

[0041] According to this configuration, it is possible to determine whether or not to generate a task request, taking into consideration the needs of the user.

[0042] In another aspect of the present disclosure, an information processing device includes a first acquisition unit that acquires a current image captured by an imaging device at the entrance of a building, a second acquisition unit that acquires related information related to the current image, a third acquisition unit that acquires a past image whose related information matches or is similar to that of the current image, a generation unit that extracts differences between the current image and the past image and generates a task request to request a task to assist a user included in the current image based on the extracted differences, and an output unit that outputs the task request.

[0043] According to this configuration, it is possible to provide an information processing device that can obtain the same effects as the above-described information processing method.

[0044] An information processing program in another aspect of the present disclosure is an information processing program that causes a computer to execute an information processing method in an information processing device, and executes the following processes: acquire a current image captured by an imaging device at the entrance of a building, acquire related information related to the current image, acquire a past image whose related information matches or is similar to that of the current image, extract differences between the current image and the past image, generate a task request to request a task to assist a user included in the current image based on the extracted differences, and output the generated task request.

[0045] According to this configuration, it is possible to provide an information processing program that can achieve the same effects as the above-described information processing method.

[0046] The present disclosure can also be realized as an information processing system operated by such an information processing program. Needless to say, such a computer program can be distributed on a computer-readable non-transitory recording medium such as a CD-ROM or via a communication network such as the Internet.

[0047] Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and step orders shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all of the embodiments, the respective contents can be combined.

[0048] (Embodiment 1) 1 is a diagram illustrating an example of the overall configuration of an information processing system according to the first embodiment of the present disclosure. The information processing system includes an information processing device 1, a related information server 2, a camera 3 (imaging device), a supporter server 4, a first terminal 5, and a second terminal 6. The information processing system supports a user who is included in an image captured by the camera 3.

[0049] The information processing device 1, the related information server 2, and the supporter server 4 are, for example, cloud servers provided on a network. The information processing device 1, the related information server 2, the camera 3, the supporter server 4, the first terminal 5, and the second terminal 6 are connected to each other via the network so as to be able to communicate with each other.

[0050] Camera 3 is, for example, a fixed camera installed at the entrance of a building where the user to be assisted lives. In the following description, the entrance is assumed to be the front door. Camera 3 is installed at a position at the front door where the line of sight is directed from inside to outside. Therefore, the image captured by camera 3 is an image of the front door space captured from inside the building. Therefore, when the user opens the front door when going out, camera 3 can capture an image of the front door including the outdoor scenery. Therefore, the image captured by camera 3 can include outdoor environmental information. Furthermore, since camera 3 is installed at the front door, the image captured by camera 3 can include belongings carried by the user when going out.

[0051] The installation position of the camera 3 is not limited to the entrance, and may be any position (for example, a room or a hallway) as long as the entrance can be photographed.

[0052] Furthermore, camera 3 may be installed at an entrance or exit of a building other than the entrance, as long as it is in a position where a user leaving the building can be detected. For example, camera 3 may be installed at a location where a user passes through at the boundary between multiple spaces within a building. In this case, camera 3 can capture a user moving between rooms or a user passing through a door at the boundary between the living room and the entrance space. Furthermore, camera 3 may be installed at a gate, or in a hallway or room connected to the entrance.

[0053] The first terminal 5 may be, for example, a portable information terminal such as a smartphone or a tablet computer, or a desktop computer. The first terminal 5 is carried, for example, by a person related to a user included in an image captured by the camera 3. Hereinafter, the user included in the image will be referred to as the first user, and the person related to the first user will be referred to as the second user. The first user may be, for example, a child, and the second user may be, for example, the child's guardian (mother or father).

[0054] The second terminal 6 may be, for example, a portable information terminal such as a smartphone or a tablet computer, or may be a desktop computer. The second terminal 6 is a terminal carried by a supporter of the first user.

[0055] The related information server 2 is configured as a cloud server that provides weather information, and transmits weather information to the information processing device 1 in response to an acquisition request from the information processing device 1. The camera 3 transmits captured images to the information processing device 1. The information processing device 1 generates a task request and transmits the generated task request to the supporter server 4. The weather information is, for example, information indicating the weather forecast for the day. For example, the weather information is information such as sunny in the morning and rain in the afternoon.

[0056] The supporter server 4 transmits the task request transmitted from the information processing device 1 to the second terminal 6, thereby notifying the supporter of the task request.

[0057] 2 is a block diagram showing an example of the configuration of the information processing device 1 shown in FIG. The information processing device 1 includes a communication unit 11, a processor 12, and a memory 13. The communication unit 11 is a communication circuit that connects the information processing device 1 to a network. For example, the communication unit 11 receives an image transmitted from the camera 3 and receives related information transmitted from the related information server 2. For example, the communication unit 11 transmits a task request to the supporter server 4.

[0058] The processor 12 is configured by a processor such as a CPU, and includes a first acquisition unit 121, a second acquisition unit 122, a third acquisition unit 123, a generation unit 124, and an output unit 125. The first acquisition unit 121 to the output unit 125 may be realized by the processor executing an information processing program, or may be configured by dedicated hardware circuits.

[0059] The first acquisition unit 121 acquires images captured by the camera 3 using the communication unit 11. If the camera 3 is configured to capture images of the entrance at a predetermined frame rate, the first acquisition unit 121 acquires the images captured by the camera 3 at the predetermined frame rate. The first acquisition unit 121 may also acquire images of people entering and leaving the entrance. In this case, the camera 3 may capture images of the entrance when an opening / closing sensor provided on the entrance door detects that the door is open or closed, or when a human presence sensor provided on the entrance detects a person.

[0060] The second acquisition unit 122 acquires related information related to the current image captured by the camera 3, associates the acquired related information with the image, and records it in the image information table 131 of the memory 13. The related information includes, for example, weather information and date and time information indicating the current date and time. For example, when an image is acquired by the first acquisition unit 121, the second acquisition unit 122 transmits a weather information acquisition request to the related information server 2, and acquires, as related information, the weather information transmitted from the related information server 2 in response to the acquisition request. The date and time information includes, for example, the year, month, day, and time.

[0061] The third acquisition unit 123 acquires, from among the past images recorded in the image information table 131 (FIG. 3) of the memory 13, past images whose related information matches or is similar to that of the current image acquired by the first acquisition unit 121. For example, the third acquisition unit 123 may acquire from the image information table 131 past images associated with related information whose content matches or is similar to that of the weather information acquired by the second acquisition unit 122.

[0062] For example, if the weather information acquired by the second acquisition unit 122 indicates that it will be sunny in the morning and rainy in the afternoon, the third acquisition unit 123 acquires from the image information table 131 past images associated with related information including the weather information that it will be sunny in the morning and rainy in the afternoon. For example, the third acquisition unit 123 may determine that two pieces of weather information that show the same trend in weather changes from morning to afternoon are similar weather information, such as when the weather information associated with one of the current image and the past image is sunny in the morning and rainy in the afternoon, and the weather information associated with the other is cloudy in the morning and rainy in the afternoon. Examples of weather change trends include a trend in the weather worsening over time, a trend in the weather improving over time, etc.

[0063] The generation unit 124 extracts differences between the current image and the past image, and generates a task request for a task to assist the first user included in the current image based on the extracted differences. For example, if the differences satisfy a predetermined condition, the generation unit 124 generates a task request for a supporter to deliver a forgotten item to the first user. A task is an action of a supporter or computer processing to support the first user. An example of support by computer processing is processing to notify the first user of a message to support the first user. An example of the predetermined condition is a condition that an item that the first user was carrying in the past image but is not carrying in the current image is a condition that the first user is not carrying an item that should be carried.

[0064] For example, the generation unit 124 extracts people and objects included in each of the current image and the past image through image processing. At this time, the generation unit 124 identifies which user among the users residing in the building the extracted person is. As the image processing, for example, a process using a recognizer that recognizes objects and a recognizer that recognizes the face of a person can be adopted. As the process for identifying a person, for example, a face recognition process can be adopted. The generation unit 124 can identify who the person is by comparing registered feature amounts of the face of each user residing in the building, which feature amounts are associated with a user ID, with feature amounts of the face of the person included in the image.

[0065] The generation unit 124 compares the current image with the previous image and extracts, as differences, objects that are included in the previous image but not included in the current image. Then, if the object extracted as a difference is an item belonging to the first user, the generation unit 124 determines that the predetermined condition is satisfied. Examples of the item include a bag, umbrella, shoes, jacket, coat, documents, gym bag, swimming bag, and rainwear. For example, if the object extracted as a difference is located within a predetermined distance from the first user, the generation unit 124 may determine that the object is an item belonging to the first user.

[0066] The supporter is a person who supports the first user by performing a task. The supporter may be, for example, an employee of a support service provider who helps the first user with carrying a lost item, or may be a local volunteer.

[0067] The output unit 125 transmits the task request generated by the generation unit 124 to the supporter server 4 using the communication unit 11.

[0068] The memory 13 is a non-volatile rewritable storage device such as a hard disk drive or a solid state drive. The memory 13 may be provided in an external server connected to the information processing device 1 via a network. The memory 13 includes an image information table 131.

[0069] FIG. 3 is a diagram showing an example of the data configuration of the image information table 131. The image information table 131 stores an image ID, an image capture time, recognition information, related information, whether or not a task request has been made, and an image in association with each other. The image ID is an identifier for an image captured by the camera 3. The image capture time is the time when the image was captured by the camera 3. The recognition information is information recognized from an image by the generation unit 124 through image processing, and includes user information and information indicating the user's belongings. Furthermore, the recognition information includes environmental information such as morning, noon, and night. The generation unit 124 may identify the environmental information by performing image processing on the image, or may identify the environmental information from the image capture time.

[0070] The related information is related information that the second acquisition unit 122 has acquired from the related information server 2. Here, weather information indicating the weather forecast for the day including the image capturing time is registered as the related information.

[0071] The presence or absence of a task request is information indicating whether a task request has been made for the image. For example, for the image with ID 00003, a task of carrying an umbrella has been performed, so the presence or absence of a task request is "yes."

[0072] The image column records the image captured by the camera 3.

[0073] 4 is a diagram showing an example of an image 500 captured by the camera 3. The image 500 is an image captured when a first user U opens a door 501 and goes out. The camera 3 has its line of sight set in a direction from indoors to outdoors, so it can capture an image of such a scene of going out. An area 502 surrounded by a door frame 503 is an outdoor image, which will be described later.

[0074] FIG. 5 is a flowchart showing an example of processing by the information processing device 1 according to the first embodiment of the present disclosure.

[0075] (Step S1) The first acquisition unit 121 acquires an image captured by the camera 3. The image acquired here is a current image. When a mode in which images are constantly transmitted from the camera 3 is adopted, for example, the first acquisition unit 121 may acquire, as the current image, an image having an image capture time closest to the current time and including the first user. Alternatively, the first acquisition unit 121 may acquire, as the current image, an image having the most recent image capture time recorded in the image information table 131 and including the first user. Note that the first user may be a specific user who is predetermined to receive support.

[0076] (Step S2) The second acquisition unit 122 acquires related information related to the current image from the related information server 2. Here, weather information indicating the weather forecast for the current time is acquired as the related information. The weather forecast is the weather forecast for today. For example, the second acquisition unit 122 may acquire weather information based on information for acquiring related information. The information is, for example, location information of the first user and time information indicating the current time. The location information of the first user may be acquired from a mobile terminal carried by the first user, or may be location information of a building. The location information of the building may be acquired from the memory 13. The time information may be acquired from a clock (not shown) included in the information processing device 1, or may be acquired from an external server (not shown) connected to the network. The second acquisition unit 122 may identify the area where the first user resides from the location information of the first user or the location information of the building, and acquire current weather information for that area from the related information server 2.

[0077] (Step S3) The third acquisition unit 123 acquires past images whose related information matches or is similar to that of the current image from the image information table 131. The determination of whether the related information matches or is similar is as described above. If the related information for the current image is rain, in the example of Figure 3, the related information for "ID00001" and "ID00003" is rain, so both images are acquired as past images.

[0078] (Step S4) The generation unit 124 extracts the differences between the current image and the past image. Here, as described above, objects that are included in the past image but not included in the current image are extracted as differences.

[0079] The generation unit 124 may extract the difference between the current image and a past image for which the task execution status is "no" in the image information table 131. In this case, the identified past image is not requested to perform a task, and therefore is highly likely to include belongings that the first user should have, and by comparing this past image with the current image, the belongings that the first user should have can be extracted with high accuracy.

[0080] Furthermore, when the past images whose related information matches or is similar to that of the current image are composed of multiple past images, the generation unit 124 may calculate the number of past images whose differences from the current image satisfy a predetermined condition, and determine that the predetermined condition is satisfied if this number exceeds the number of past images whose differences do not satisfy the predetermined condition. For example, even if a user does not have an umbrella when it is raining, a task request to carry an umbrella may not necessarily be generated due to the user's needs. In this case, there is no need to deliver an umbrella to the first user. In this configuration, a task request to carry belongings will not be generated in such a case, so it is possible to determine whether or not to generate a task request taking the first user's needs into consideration.

[0081] (Step S5) The generation unit 124 generates a task request based on the difference. Here, as described above, if the object extracted as the difference is a belonging that the first user should have, a task request is generated to request a supporter to carry the belonging to the first user.

[0082] (Step S6) The output unit 125 transmits (outputs) the task request to the supporter server 4 using the communication unit 11. The supporter server 4, which receives the task request, transmits the task request to the supporter's mobile terminal (not shown). The task request includes the belongings to be transported, the address of the first user's home, and the destination. The supporter goes to the first user's home to pick up the belongings to be transported, travels to the destination, and delivers the belongings to the first user. This allows the first user to retrieve the lost items.

[0083] As described above, according to the first embodiment, a past image whose related information matches or is similar to that of a current image is acquired, and the differences between the acquired past image and the current image are extracted. Therefore, the past image captured under conditions where the related information matches can be regarded as the correct answer, and the differences from the current image can be extracted. As a result, information that should be used to support the user and is included in the current image can be accurately identified from the differences, and support suited to the user can be realized.

[0084] (Embodiment 2) In the second embodiment, the support candidate is asked whether or not to accept a task request. FIG. 6 is a block diagram showing an example of the configuration of an information processing device 1A in the second embodiment. The information processing device 1A differs from the information processing device 1 in the output unit 125A of the processor 12A. The difference is that the memory 13A further stores a supporter table 132.

[0085] The output unit 125A outputs an acceptance request (first acceptance request) to the supporter server 4 using the communication unit 11, encouraging the supporter supporting the first user to accept the task, and when consent information indicating that the support candidate has agreed to the first acceptance request is obtained using the communication unit 11, the output unit 125A outputs the task request to the supporter server 4 using the communication unit 11.

[0086] FIG. 7 is a diagram showing an example of the data configuration of the supporter table. The supporter table 132 is a table that associates user information with support candidate information. The first user information is an identifier of the first user. The support candidate information is an identifier of a support candidate who can support the first user. In the example of FIG. 7, support candidates P1, P2, and P3 are defined for the first users UA and UB, and support candidates P4, P5, and P6 are defined for the first user UC. A supporter is ultimately selected from these support candidates. By selecting a support candidate for each first user in this way, a final supporter can be selected from support candidates who can smoothly provide support to the first user.

[0087] The supporter table 132 may have a pre-set priority order for multiple candidates for each first user. Also, as shown by first users UA and UB, the same support candidate may be associated with different first users. Furthermore, the supporter table 132 may define support candidates according to the task content and the first user.

[0088] 8 is a flowchart showing an example of the process of information processing device 1A according to Embodiment 2 of the present disclosure. The processes from step S1 to step S5 and step S6 are the same as those in FIG. 5, and therefore description thereof will be omitted.

[0089] (Step S7) The output unit 125A acquires support candidate information from the supporter table 132. For example, when the first user UA is extracted from the current image, support candidate information of support candidates P1, P2, and P3 is acquired. Note that when support candidates are registered in the supporter table 132 in association with the content of the task and the first user, the output unit 125A only needs to acquire support candidate information associated with the content of the task and the first user.

[0090] (Step S8) The output unit 125A transmits a first acceptance request to the support candidate indicated by the acquired support candidate information, confirming their acceptance of the task, to the supporter server 4. The supporter server 4 that receives the first acceptance request may notify each support candidate of the first acceptance request by transmitting the first acceptance request to the second terminal 6 carried by each support candidate. The first acceptance request may include the content of the task and the conditions of the task request. The conditions of the task request may be, for example, a deadline for executing the task. The first acceptance request may also include a history of the dates and times when a certain supporter previously performed a task for the first user.

[0091] (Step S9) The output unit 125A determines whether consent information has been acquired from at least one support candidate. If consent information has been acquired (YES in step S9), the process proceeds to step S10; if consent information has not been acquired (NO in step S9), the process ends. For example, if a support candidate who has viewed the first acceptance request agrees with the first acceptance request, they can enter consent information into the second terminal 6, and if they do not agree with the first acceptance request, they can enter non-consent information into the second terminal 6. The entered consent information or non-consent information is input into the information processing device 1A via the supporter server 4. If the determination in step S9 is NO, the output unit 125A may send a message to the first terminal 5 indicating that the task was requested from support candidates but none accepted.

[0092] (Step S10) The output unit 125A uses the communication unit 11 to send a second acceptance request to the first terminal 5, which requests the second user to confirm acceptance of the task. This notifies the second user, who is the guardian of the first user, of the second acceptance request. The second acceptance request includes the content of the task and the name of the support candidate who entered the consent information. This provides the second user with information to help them decide whether or not to accept the task. The second acceptance request for the task may also be notified to the first user. In this case, the second acceptance request is sent to the mobile terminal (not shown) of the first user.

[0093] (Step S11) The output unit 125A determines whether or not consent information has been acquired from the first terminal 5. If consent information has been acquired (YES in step S11), the process proceeds to step S12, and if consent information has not been acquired (NO in step S11), the process ends. For example, the second user who viewed the second acceptance request inputs consent information into the first terminal 5 if he or she agrees with the second acceptance request, and inputs non-consent information into the first terminal 5 if he or she does not agree with the second acceptance request.

[0094] (Step S12) The output unit 125A determines a supporter from among the support candidates who agreed to the first acceptance request. For example, if there are multiple support candidates who agreed to the first acceptance request, one support candidate may be randomly determined from each support candidate. Furthermore, if a priority order is set in advance for each support candidate, the output unit 125A may determine the support candidate with the highest priority order from among the multiple support candidates who agreed to the first acceptance request as the supporter.

[0095] (Step S6) The output unit 125A transmits the task request to the supporter server 4 using the communication unit 11 to officially request the task from the determined supporter. The task request is transmitted from the supporter server 4 to the second terminal 6 of the determined supporter. This notifies the determined supporter of the task request.

[0096] In addition, if a priority order is set for the support candidates, in steps S9 to S12, the output unit 125A may send the first acceptance request in order starting from the support candidate with the highest priority, and determine the supporter once consent information is obtained.

[0097] As described above, according to the second embodiment, the first acceptance request is notified to the support candidate, and the task request is made when consent information is entered, so that the task request can be made while respecting the will of the support candidate. Also, the second acceptance request is notified to the second user, and the task request is made when consent information is entered, so that it is possible to prevent a task from being executed without the permission of the second user.

[0098] (Embodiment 3) The third embodiment sets a deadline for task execution. FIG. 9 is a diagram showing an example of the configuration of an information processing device 1B according to the third embodiment of the present disclosure. The information processing device 1B differs from the information processing device 1A in that an output unit 125B of a processor 12A is included. Another difference is that a memory 13B stores a completion time table 133 and a required time table 134.

[0099] The output unit 125B sets a deadline for the task based on the content of the difference extracted by the generation unit 124 and the user information indicating the first user. The deadline is, for example, a deadline by which the supporter must start the task. The deadline is determined from the completion time of the task, which is predetermined according to the user information of the first user and the content of the difference, to a time before the required time for the task, which is predetermined according to the supporter and the first user. The completion time is determined based on the start time of the first user's use of the belongings. The required time is the time required for the supporter to transport the belongings to the first user.

[0100] FIG. 10 is a diagram showing an example of the data configuration of the completion time table 133. The completion time table 133 specifies the completion time of a task corresponding to the first user and their belongings. The completion time table 133 stores first user information, belonging information, and completion time in association with each other. The first user information is the user ID of the first user. The belonging information indicates an object determined to be an belonging that the first user should have. The completion time is the completion time of a task in which the first user carries the belongings that the first user should have. In the example of FIG. 10, for the first user UA, the completion time of the task of carrying an umbrella is set to 3:30 p.m. This takes into consideration the time when the first user UA returns home from a destination (e.g., school). For the first user UA, the completion time of the task of carrying gym clothes is set to 10:00 a.m. This takes into consideration the time when the first user UA puts on the gym clothes while away from home. For the first user UB, the task of carrying an umbrella is set to 7:00 p.m. This is in consideration of the time when the first user UB returns home from a place where he or she has gone out (for example, a cram school).

[0101] FIG. 11 is a diagram showing an example of the data configuration of the required time table 134. The required time table 134 is a table that specifies the required time according to the supporter and the first user. The required time table 134 stores supporter information, first user information, and the required time in association with each other. The supporter information is the supporter's identifier. The supporter's identifier is the same as the support candidate's identifier. The first user information is the first user's identifier. In the example of FIG. 11, the required time is set to one hour for supporter Q1 and first user UA. This is because it takes one hour for supporter Q1 to transport belongings to the first user UA's destination (school). The required time is set to one hour for supporter Q2 and first user UA. This is because it takes one hour for supporter Q2 to transport belongings to the first user UA's destination (school).

[0102] The process by which the output unit 125B sets the execution deadline is as follows. The following explanation assumes that the first user receiving support is first user UA, the supporter is supporter Q1, and the belongings being transported are an umbrella. The output unit 125B obtains the completion time of first user UA and the umbrella, "15:30", from the completion time table 133. Next, the output unit 125B obtains the required time of supporter Q1 and first user UA, "1 hour", from the required time table 134. Next, the output unit 125B sets "14:30", which is the completion time "15:30" minus the required time "1 hour", as the execution deadline.

[0103] The output unit 125B includes the set execution deadline in the first acceptance request described in the second embodiment and transmits it to the supporter server 4. This first acceptance request is transmitted to the second terminal 6 of the supporter Q1. Therefore, the supporter Q1 can check the execution deadline included in the first acceptance request and determine whether or not to agree to the first support request.

[0104] In this way, according to the third embodiment, the execution deadline is set by subtracting the time required for the supporter to transport the belongings to the first user's destination from the completion time, which is the time when the first user will use the belongings, and the first support request including this execution deadline is notified to the supporter. Therefore, support by a supporter who can transport the belongings by the time when the first user will start using the belongings is realized.

[0105] (Fourth embodiment) In the fourth embodiment, a message regarding returning home to the user is adopted as a task. In the present embodiment, the configuration of the information processing device 1 is the same as that in FIG. 2, and therefore will be described using FIG. 2. The configuration of the information processing device 1 in the fourth embodiment may be the configuration in FIG. 6 or FIG. 9. Below, differences between the fourth embodiment and the first to third embodiments will be described, and a description of commonalities will be omitted.

[0106] 2, the first acquisition unit 121 acquires, as the current image, an image captured by the camera 3 when the first user returns home. For example, the first acquisition unit 121 may acquire, as the image when the first user returns home, an image captured during a predetermined time period when the first user returns home, showing a scene in which the first user opens the door and enters indoors from outdoors.

[0107] The second acquisition unit 122 acquires the imaging time and weather information of the current image captured by the camera 3 as related information, and records the acquired imaging time and weather information in association with the image in the image information table 131. The second acquisition unit 122 may acquire the imaging time included in the image transmitted from the camera 3 as related information.

[0108] The third acquisition unit 123 acquires from the image information table 131 past images whose capture times are the same or similar to those of the current image and whose weather information is the same or similar to those of the current image. Here, a past image whose capture time is the same or similar to that of the current image refers to a past image captured a predetermined number of days before the current image, i.e., a past image captured in the same time period as the capture time of the current image. A predetermined number of days before is, for example, 5 to 9 days before the capture time of the current image. The same time period refers to, for example, a time period within a certain period before and after the capture time of the current image. The certain period can be an appropriate time span, such as 5 minutes, 10 minutes, or 20 minutes. For example, if the capture time of the current image is 5:00 PM, a past image captured 7 days ago between 4:50 PM and 5:10 PM is acquired.

[0109] The third acquisition unit 123 may acquire, as a past image, an image in which the image capturing time and weather information match or are similar to those of the current image and in which the door is open.

[0110] The generation unit 124 extracts the difference between the decrease in outdoor brightness obtained from the past image and the current image. In this case, the generation unit 124 may calculate the outdoor brightness from the brightness of the image. Furthermore, the generation unit 124 may extract an image of the inside of a door frame included in the image as an outdoor image showing the outside of the building, and calculate the outdoor brightness based on the brightness of the outdoor image. Here, the outdoor brightness may be, for example, a continuous value or a step value in which brightness is evaluated in multiple steps (for example, five steps). The decrease in the outdoor brightness is the value obtained by subtracting the brightness of the previous image from the brightness of the previous image.

[0111] If the extracted decrease in brightness is greater than a predetermined value, the generation unit 124 generates a task request for executing a task of notifying the first user of a message about returning home. The predetermined value is, for example, a value that assumes that the sunset time will be earlier and it will be difficult for the first user to return home while it is still light outside at the current time of returning home.

[0112] The output unit 125 transmits the task request to the supporter server 4. The supporter server 4 that receives the task request generates notification information including a message notifying the user to come home earlier, and transmits the generated notification information to the first terminal 5 or a mobile terminal (not shown) carried by the first user. This notifies the guardian of the first user or the first user of the message to come home earlier.

[0113] The output unit 125 may transmit the task request, for example, when the user returns home this time, or when the user returns home next time. Note that in the fourth embodiment, there is no need to select a requester, and therefore the processes of S7 to S12 in the flowchart of FIG. 8 are unnecessary.

[0114] 12 is an explanatory diagram of the fourth embodiment. The diagram on the left shows an image of the first user when he / she returned home last week, and the diagram on the right shows an image of the first user when he / she returned home today. The image of the first user when he / she returned home today shows a darker outdoor brightness than the image of the first user when he / she returned home last week. This is because the seasons have changed and the daylight hours have become shorter.

[0115] In this case, it is preferable from a safety standpoint for the first user to come home earlier. Therefore, in the fourth embodiment, when such an event occurs, a message is sent to the first user instructing him or her to come home earlier. This encourages the first user to come home earlier next time. This ensures the safety of the first user and gives a sense of security to the guardian.

[0116] (Embodiment 5) In the fifth embodiment, a task is determined by referring to a task determination table. Fig. 13 is a diagram showing an example of the configuration of an information processing device 1C in the fifth embodiment of the present disclosure. The information processing device 1C differs from the information processing device 1 in that a memory 13C further stores a task determination table 135.

[0117] FIG. 14 is a diagram showing an example of the data configuration of the task determination table 135. The task determination table 135 is a table that is referenced when the generation unit 124 determines a task. The task determination table 135 defines a task according to the content of the difference and related information. If the difference between the current image and the past image is an item that the first user should have, a task to transport the item is executed. For example, if the difference is an item, weather information is used as related information, and the task is determined in the same manner as in the first embodiment.

[0118] Furthermore, if the difference is in belongings, the first user's schedule information may be used as the related information. In this case, the second acquisition unit 122 acquires the first user's schedule information as the related information. For example, the second acquisition unit 122 acquires the first user's schedule information from an external server (not shown) that stores schedule information, and records the first user's schedule information in the image information table 131 in association with the current image.

[0119] The third acquisition unit 123 acquires past images having matching or similar schedule information from among the past images recorded in the image information table 131. Matching or similar schedule information corresponds to, for example, a case where the same schedule is included in one schedule information and another schedule information. An example of a schedule is the same extracurricular activity of the first user. Examples of extracurricular activities include cram school, swimming lessons, etc.

[0120] If the belongings extracted as differences between the past image and the current image are items necessary for the schedule, i.e., if the past image contains items necessary for the lesson but the current image does not, the generation unit 124 determines a task to transport the items to the first user and generates a task request for the task. As a result, the first user can have the items necessary for the lesson transported by a supporter while out and receive the items.

[0121] 14, if the difference between the current image and the past image is the brightness of the outdoors when the first user returned home, a task of notifying a message about returning home is executed. The details of this process are the same as those in the fourth embodiment.

[0122] 14, if the difference between the current image and the past image is a combination of the user returning home, a task of ordering food and a task of controlling a device are determined. In this case, the first acquisition unit 121 acquires the image of the user returning home as the current image. The method of selecting the image of the user returning home has been described above.

[0123] The second acquisition unit 122 acquires the day of the week and time period when the current image was captured as related information, and records the acquired day of the week and time period in the image information table 131 in association with the current image.

[0124] The third acquisition unit 123 acquires past images recorded in the image information table 131 that have the same or similar day of the week and time period as the current image. The same or similar day of the week and time period means, for example, that the images were captured on the same day of the week and in the same time period. For example, if the current image was captured at 5:30 PM on Wednesday, the third acquisition unit 123 acquires past images that were captured in the 5:00 PM time period on Wednesday.

[0125] The generation unit 124 extracts the difference in the combination of users included between the current image and the past image as a difference. A difference in the combination of users is, for example, a case where the current image includes only a child, whereas the past image includes a child and a parent.

[0126] If the difference in the combination satisfies a predetermined condition, the generation unit 124 generates a task request for executing a task to order food or a task to control an appliance. An example of the predetermined condition is a condition in which the past image includes a child and a parent, while the current image includes only a child. An example of ordering food is ordering delivery or ingredients. An example of controlling an appliance is controlling a washing machine to operate, or controlling a hot water supply system to fill a bathtub with hot water. A task request for a task to order food may be an order request to order a predetermined dish or ingredients from a predetermined destination. A task request for a task to control an appliance may be, for example, a control signal to operate an appliance under predetermined conditions.

[0127] As a result, if a child who usually comes home with their guardian does not come home with their guardian today, food can be automatically ordered or the device can be controlled, and support for the child's daily life can be provided.

[0128] The generation unit 124 may switch between the three tasks shown in Fig. 14 depending on the difference. For example, if the difference is about belongings, the generation unit 124 may determine a task to transport the belongings; if the difference is about the brightness outside, the generation unit 124 may determine a task to notify the user of a message about returning home; and if the difference is about the combination of users returning home, the generation unit 124 may determine a task to order food and / or a task to control a device.

[0129] (Variation 1) The generation unit 124 may change the tasks to be referenced from the task determination table 135 depending on the time of day. For example, the generation unit 124 may determine only tasks related to belongings in the morning, and may determine tasks related to user combinations and / or outdoor brightness in the afternoon.

[0130] (Variation 2) The second acquisition unit 122 does not need to associate the acquired related information with the image and store it in the memory 13. For example, a mode may be adopted in which a server separate from the information processing device 1 has a database that stores related information and past images in association with each other. In this case, the third acquisition unit 123 only needs to acquire from this database the current image and past images whose related information matches or is similar to the current image. [Industrial Applicability]

[0131] The present disclosure is useful in the technical field of providing suitable support to users.

Claims

1. An information processing method in an information processing device, The processor of the information processing device Acquire a current image captured by an imaging device at the entrance of the building; obtaining relevant information related to the current image; Obtaining a past image whose related information matches or is similar to that of the current image; extracting differences between the current image and the past image, and generating a task request based on the extracted differences to request a task to assist the user included in the current image; outputting the generated task request; Information processing methods.

2. The difference is the possessions that the user should have; The task is a task of transporting belongings to the user. The information processing method according to claim 1.

3. the difference is a change in outdoor brightness, The task is a task of notifying the user of a message regarding returning home. The information processing method according to claim 1.

4. The difference is the combination of users at the time of returning home, The task is a task of ordering food or a task of controlling a device. The information processing method according to claim 1.

5. The related information includes weather information. The information processing method according to claim 1.

6. In generating the task request, the task request is generated using a task decision rule in which the content of the difference and the content of the task are previously associated with each other. The information processing method according to claim 1.

7. In the output of the task request, an acceptance request is output to an assistant assisting the first user to encourage the assistant to accept the task, and when consent information indicating that the assistant has consented to the acceptance request is acquired, the task request is output. The information processing method according to claim 1.

8. In the output of the task request, a deadline for the task is set based on the content of the difference and user information indicating the user, and the deadline is included in the acceptance request.

8. The information processing method according to claim 7.

9. the execution deadline is determined to be a time before a required time for the task, which is predetermined depending on the supporter and the user, from a completion time of the task, which is predetermined depending on the user information and the content of the difference; 9. The information processing method according to claim 8.

10. The difference is the belongings that the user should have, the completion time is a time determined based on a time when the user starts using the belongings that should be carried, The required time is the time required to transport the belongings to the user.

10. The information processing method according to claim 9.

11. The task is performed by a supporter other than the user. The information processing method according to claim 1.

12. The past image is further associated with information indicating whether or not a task request has been made, In the extraction of the difference, a past image for which the task request has not been made is identified based on the information indicating the presence or absence of the task request, and a difference between the identified past image and the current image is extracted. The information processing method according to claim 1.

13. In the extraction of the difference, if the past images having the same or similar related information as the current image are made up of a plurality of past images, the presence or absence of the difference between each of the past images and the current image is determined, and if the number of past images determined to have the difference exceeds the number of past images determined to have no difference, it is determined that the difference exists. The information processing method according to any one of claims 1 to 12.

14. a first acquisition unit that acquires a current image captured by an imaging device at an entrance of the building; a second acquisition unit that acquires related information related to the current image; a third acquisition unit that acquires a past image whose related information matches or is similar to the related information of the current image; a generation unit that extracts a difference between the current image and the past image, and generates a task request based on the extracted difference to request a task for supporting a user included in the current image; an output unit that outputs the task request, Information processing device.

15. An information processing program that causes a computer to execute an information processing method in an information processing device, Acquire a current image captured by an imaging device at the entrance of the building; obtaining relevant information related to the current image; Obtaining a past image whose related information matches or is similar to that of the current image; extracting differences between the current image and the past image, and generating a task request based on the extracted differences to request a task to assist the user included in the current image; Output the generated task request and execute the process. Information processing program.

Images