Information processing system, information processing method, and program

Abstract

To provide an information processing system or the like that enables a user to communicate with a close person as when the person were close to the user.SOLUTION: A system includes a partner generation unit that generates a digital partner on the basis of learning, a conversation generation unit that generates a conversation between the digital partner and a user, a conversation information acquisition unit that acquires first information related to the content of the conversation, and a behavioral information acquisition unit that acquires second information related to the user's behavior, and the conversation generation unit generates a conversation on the basis of the first information acquired by the conversation information acquisition unit in accordance with the second information acquired by the behavioral information acquisition unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a support system for elderly people and the like that uses an observational conversation robot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-014747 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the system disclosed in Patent Document 1, it is difficult for users such as elderly people to communicate with the observational conversation robot in the same way as if a close friend such as a spouse or child were nearby.

[0005] Therefore, in one aspect, an object of the present invention is to provide an information processing system and the like that allows a user to communicate with a person as if they were close to them. [Means for solving the problem]

[0006] (1) In one embodiment, a partner generation unit that generates a digital partner based on the learning; a conversation generation unit that generates a conversation of the digital partner for the user; a conversation information acquisition unit that acquires first information related to the content of the conversation; a behavior information acquisition unit that acquires second information related to the user's behavior; Equipped with An information processing system is provided in which the conversation generation unit generates the conversation based on the first information acquired by the conversation information acquisition unit in accordance with the second information acquired by the behavioral information acquisition unit.

[0007] (2) In the above aspect (1), preferably, The second information relates to a conversation of the user with the digital partner.

[0008] (3) In the above aspect (1), preferably, The digital partner communicates with the user via a terminal device selected by the user.

[0009] (4) In the above aspect (1), preferably, The first information is obtained via the Internet.

[0010] (5) In the above aspect (1), preferably, The first information or the second information is based on an image or sound acquired via a terminal device of the user.

[0011] (6) In the above aspect (1), preferably, The terminal device with which the digital partner communicates is selected based on the user's voice or the user's location.

[0012] (7) In another aspect, a partner generation step of generating a digital partner based on the learning; a conversation generation step of generating a conversation of the digital partner for the user; a conversation information acquisition step of acquiring first information related to the content of the conversation; a behavior information acquisition step of acquiring second information related to the user's behavior; Equipped with An information processing method is provided in which, in the conversation generation step, the conversation is generated based on the first information acquired by the conversation information acquisition step in accordance with the second information acquired by the behavioral information acquisition step.

[0013] (8) In another embodiment, a partner generation step of generating a digital partner based on the learning; a conversation generation step of generating a conversation of the digital partner for the user; a conversation information acquisition step of acquiring first information related to the content of the conversation; a behavior information acquisition step of acquiring second information related to the user's behavior; on the computer, In the conversation generation step, a program is provided that generates the conversation based on the first information acquired in the conversation information acquisition step in accordance with the second information acquired in the behavior information acquisition step. [Effects of the Invention]

[0014] In one aspect, the present invention allows users to communicate with each other as if they were close friends or family members nearby. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating a configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 1 illustrates a conversation between a user and a digital partner. [Figure 3] FIG. 10 is a diagram showing an example of a display screen of a tablet. [Figure 3A] FIG. 10 is a diagram showing an example of a display screen of a tablet. [Figure 3B] FIG. 10 is a diagram showing an example of a display screen of a tablet. [Figure 4] FIG. 10 is a diagram illustrating an example of a function that is realized by cooperation between a terminal device and another device. [Figure 5]10A and 10B are diagrams showing example images taken by a tablet during cooking. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0017] The information processing system of this embodiment provides a service that generates digital partners using AI (anthropomorphic AI) for users such as elderly people and enables communication with the digital partners. In this service, the digital partners are generated by having the AI ​​learn the faces and voices of, for example, a spouse or child. The generated digital partners can communicate with the user via a terminal device. By communicating with the digital partners, the user can converse with their spouse or child as if they were actually nearby. This can, for example, help prevent dementia in elderly people living alone. Furthermore, for example, by incorporating information on improving dietary habits and the latest crime trends into the conversation, it is possible to prevent the user's nutritional status from deteriorating and crime victimization can be prevented.

[0018] FIG. 1 is a diagram showing the configuration of an information processing system according to this embodiment.

[0019] The information processing system of this embodiment includes a server 10 managed by a service provider, a tablet 20 equipped with a camera, a microphone, and a speaker as a terminal device for a user 50 (FIG. 2), and an earphone microphone 30 as another terminal device for the user 50. The tablet 20 and the earphone microphone 30 are each capable of communicating with the server 10. The server 10, the tablet 20, and the earphone microphone 30 each have a predetermined program installed therein for executing the processing required to provide the service.

[0020] As shown in FIG. 1, the server 10 includes a partner generation unit 11 that generates a digital partner based on learning, and a conversation information acquisition unit 12 that acquires first information related to the content of the conversation between the digital partners.

[0021] The tablet 20 is also provided with a conversation generation unit 21 that generates a conversation between the partner and the user, and a behavioral information acquisition unit 22 that acquires second information related to the user's behavior. The earphone microphone 30 is also provided with a conversation generation unit 31 that generates a conversation between the partner and the user, and a behavioral information acquisition unit 32 that acquires second information related to the user's behavior. In this embodiment, the digital partner communicates with the user via the tablet 20 or the earphone microphone 30.

[0022] 1, for the sake of convenience, the functions of the partner generation unit 11 and the conversation information acquisition unit 12 are assigned to the server 10. Also, the functions of the conversation generation unit 21 and the behavioral information acquisition unit 22 are assigned to the tablet 20, and the functions of the conversation generation unit 31 and the behavioral information acquisition unit 32 are assigned to the earphone microphone 30. However, these functions constituting the information processing system can be assigned to any of the server 10, the tablet 20, and the earphone microphone 30. Furthermore, these functions may be achieved by cooperation between any two or more of the server 10, the tablet 20, and the earphone microphone 30.

[0023] Furthermore, the terminal device used for communication with a digital partner is not limited to the tablet 20 and the earphone microphone 30, but may be a smartphone or any other device. For example, instead of or in addition to the earphone microphone 30, glasses or a hat that function as a terminal device with earphones and a microphone may be used.

[0024] Next, the operation of the information processing system of this embodiment will be described.

[0025] The partner generation unit 11 generates a digital partner based on the learning. The attributes, face information, and voice information of the digital partner are registered in advance.

[0026] As shown in FIG. 1, for example, a camera 61 installed in the home of a real-life relative 60 (such as a child or grandchild) captures a video of the relative 60, and the partner generation unit 11 acquires the video and audio captured by the camera 61 as facial information and voice information via communication and registers the information in the server 10. Next, the partner generation unit 11 generates a digital partner that resembles the relative 60 based on learning from the acquired facial information and voice information. By taking videos as needed, the partner generation unit 11 can update the digital partner, and changes in the relative 60 (such as the growth of a grandchild) are reflected in the digital partner.

[0027] For example, the partner generation unit 11 can generate a digital partner based on the user's spouse before death. In this case, the partner generation unit 11 can generate a digital partner that resembles the spouse before death by acquiring face information and voice information from videos of the spouse recorded before death.

[0028] The conversation information acquisition unit 12 acquires various information (first information) via the Internet or other means. The first information includes, for example, the latest information on crimes, including fraud methods, and information on events near the home of the user 50.

[0029] The behavior information acquisition unit 22 and the behavior information acquisition unit 32 acquire a wide range of information (second information) related to the behavior of the user 50, including the conversation of the user 50.

[0030] The conversation generation unit 21 and the conversation generation unit 31 generate a conversation of predetermined content for the user 50 at a predetermined timing based on the first information acquired by the conversation information acquisition unit 12 and the second information acquired by the behavioral information acquisition unit 22 and the behavioral information acquisition unit 32.

[0031] FIG. 2 is a diagram illustrating a conversation between a user and a digital partner.

[0032] In the example of FIG. 2, a relative 60 (FIG. 1) is displayed as a digital partner on the display screen of the tablet 20, and the user 50 can converse face-to-face with the relative 60 displayed on the display screen. The user 50's voice (second information) is captured by the microphone of the tablet 20, and the user 50's movements and facial expressions (second information) are captured by the camera of the tablet 20, and the behavioral information acquisition unit 22 analyzes and recognizes (acquires) the content of the conversation of the user 50 and the state of the user 50. The conversation generation unit 21 generates a conversation according to the content of the conversation of the user 50 and the state of the user 50 acquired by the behavioral information acquisition unit 22, and outputs the conversation via the speaker or earphones of the tablet 20. The conversation generation unit 21 also controls the facial movements and facial expressions of the relative 60 displayed on the tablet 20 to be generated in accordance with the conversation.

[0033] The content of the conversation uttered by the relative 60 can be generated based not only on the content of the conversation of the user 50 and the state of the user 50, but also on various information (first information) acquired by the conversation information acquisition unit 12 via the Internet or other means. For example, in the example of FIG. 2 , the conversation generation unit 21 can acquire, via the conversation information acquisition unit 12, information that the relative 60 is flying a kite in a park near the location where the tablet 20 is installed (e.g., the home of the user 50). The conversation generation unit 21 engages the user 50 in a conversation based on the acquired information. Note that the location where the tablet 20 is installed can be recognized based on, for example, location information indicated by a sensor provided in the tablet 20 or an image (e.g., an image of the room) captured by a camera of the tablet 20. Furthermore, the tablet 20 can be moved, for example, inside and outside the home, but can also be placed in a specific location, such as a specific seat at the dining table, while the user 50 is at home.

[0034] Next, an example of the operation of the information processing system of this embodiment when the user 50 goes out will be described.

[0035] 3 to 3B are diagrams showing examples of display screens on a tablet.

[0036] FIG. 3 shows a state in which a relative 60 (FIG. 1) is displayed on the display screen of the tablet 20. In this state, as described above, the user 50 can communicate with a digital partner via the tablet 20. Meanwhile, FIG. 3A shows a state in which the go out button 51 displayed on the display screen of the tablet 20 has been operated. When the go out button 51 is operated, the relative 60 disappears from the display screen and the earphone microphone 30 switches to an operating state. Furthermore, as shown in FIG. 3B, a come home button 52 is displayed on the display screen of the tablet 20 in place of the go out button 51.

[0037] In this state, the user 50 wearing the earphone microphone 30 can go out with their digital partner. The user's 50 voice (second information) is captured by the microphone of the earphone microphone 30, and the behavioral information acquisition unit 32 analyzes and acquires the content of the user's 50 conversation. The conversation generation unit 31 also generates a conversation according to the content of the user's 50 conversation acquired by the behavioral information acquisition unit 32, and outputs the conversation via the earphone of the earphone microphone 30.

[0038] The content of the conversation uttered by the relative 60 can be generated based not only on the content of the conversation of the user 50, but also on various information (first information) acquired by the conversation information acquisition unit 12 via the Internet or other means, and information (second information) acquired via the behavioral information acquisition unit 22. The second information also includes environmental sounds captured by the microphone of the earphone microphone 30. When the behavioral information acquisition unit 22 recognizes the environment in which the user 50 is placed based on the environmental sounds, the recognition result is reflected in communication with the user 50 via the generation of a conversation by the conversation generation unit 31.

[0039] The earphone microphone 30 is also provided with a sensor that detects the position of the earphone microphone 30, and the behavioral information acquisition unit 32 acquires position information (second information) from the sensor in real time. Therefore, the conversation generation unit 31 can, for example, throw a conversation to the user 50 at an appropriate time informing the user 50 that the user 50 is flying a kite in a park that the user 50 passes by. It is also possible to provide audio guidance to the destination to the user 50 who has told the user their destination. In this case, by providing the earphone microphone 30 with a sensor that detects the orientation of the earphone microphone 30 (the orientation of the head of the user 50 wearing the earphone microphone 30), it is also possible to inform the user 50 of the direction they should go by audio. Furthermore, when the user 50 uses transportation, the conversation generation unit 31 can acquire information about the transportation via the conversation information acquisition unit 12 and provide audio guidance to the user 50.

[0040] When the user 50 returns from going out and operates the return home button 52 (FIG. 3B) displayed on the display screen of the tablet 20, the display screen of the tablet 20 returns to the state in which the relatives 60 are displayed (FIG. 3).

[0041] In this way, in this embodiment, the digital partner can communicate via the terminal device (the tablet 20 or the earphone microphone 30) selected by the user 50. Note that such switching of the terminal device may be performed, for example, via communication between the tablet 20 and the earphone microphone 30. Alternatively, it may be performed via communication between each terminal device (the tablet 20 and the earphone microphone 30) and the server 10.

[0042] In this embodiment, a terminal device with which the digital partner will communicate may be selected based on the voice of the user 50 or the location of the user 50. In this case, for example, when the user 50 calls out to the tablet 20 or the earphone microphone 30, the tablet 20 or the earphone microphone 30 that is called may be selected as the terminal device with which to communicate. Also, when the user 50 is photographed with the camera of the tablet 20, or when a predetermined pose of the user 50 is photographed with the camera of the tablet 20, this may trigger the tablet 20 to be selected as the terminal device with which to communicate.

[0043] Next, other functions using digital partners will be described.

[0044] FIG. 4 is a diagram showing an example of a function that is realized by cooperation between a terminal device and other devices.

[0045] The example shown in Fig. 4 illustrates an example of cooperation between the tablet 20 and the intercom 71. In this example, images and audio acquired by the intercom 71 are transmitted to the tablet 20, and the tablet 20 then determines whether or not the visitor is a registered person. In this case, by registering images and the like of people who are expected to visit in advance in the tablet 20, the tablet 20 (behavioral information acquisition unit 22) can determine whether or not the visitor is a registered person. The conversation generation unit 21 can generate a conversation according to the attributes of the recognized visitor.

[0046] For example, if the visitor is not a registered person, the digital partner can warn the user 50 through the conversation generated by the conversation generation unit 21. The digital partner can also encourage the user 50 to take a predetermined action. In this case, the conversation generated by the conversation generation unit 21 can reflect various information (first information) acquired via the conversation information acquisition unit 12. As described above, the first information includes the latest information on crimes, including fraud methods.

[0047] The example shown in FIG. 4 also illustrates cooperation between the tablet 20 and a telephone 72. The telephone 72 may be either a fixed telephone or a mobile phone. In this example, the voice of a call on the telephone 72 (the voice of the other party) and the telephone number of the other party are acquired by the tablet 20 via communication or via a microphone. In this case, too, by registering the voice, telephone number, etc. of an expected caller in the tablet 20 in advance, it becomes possible for the tablet 20 (behavioral information acquisition unit 22) to determine whether the caller is a registered person. The conversation generation unit 21 can generate a conversation according to the attributes of the recognized caller.

[0048] For example, if the caller is not a registered person, the digital partner can warn the user 50 through the conversation generated by the conversation generation unit 21. In this case, too, the conversation generated by the conversation generation unit 21 can reflect various pieces of information (first information) acquired via the conversation information acquisition unit 12.

[0049] In addition, through digital partners, meal recipe suggestions can be made.

[0050] For example, by taking a picture of the dining table with the camera of the tablet 20, the dietary contents of the user 50 can be analyzed based on the image. In this case, the dietary contents may be accurately identified through a dialogue between the user 50 and the digital partner. Based on the identified dietary contents for a predetermined period, recipes that take into account necessary nutrients can be communicated to the user 50 via the digital partner. This makes it possible to support the diet of the user 50 from a nutritional perspective, and to improve the eating habits of the user 50.

[0051] Furthermore, for example, ingredients can be analyzed by taking a picture of the inside of the refrigerator with the camera of the tablet 20. In this case, the ingredients may be accurately identified through a dialogue between the user 50 and the digital partner. Based on the identified ingredients, meal recipes using the ingredients can be suggested to the user 50 via the digital partner.

[0052] Cooking assistance can also be provided via the digital partner. For example, when a meal recipe is selected via the tablet 20, the cooking steps according to the recipe can be communicated to the user 50 via the digital partner. In this case, by taking a picture of the hands of the user 50 who is cooking with the camera of the tablet 20, it is possible to communicate the cooking steps step by step in detail. The cooking steps may be communicated, for example, via the display screen of the tablet 20, using an image of the digital partner demonstrating the cooking process in addition to the voice of the digital partner.

[0053] 5 is a diagram showing an example of an image captured by a tablet while cooking. In this case, the behavioral information acquisition unit 22 analyzes the progress of cooking based on the images captured by the tablet 20, and can communicate the cooking steps according to the recipe to the user 50 through the digital partner conversation generated by the conversation generation unit 21. Furthermore, if the progress of cooking differs from the expected recipe, the recipe can be updated at that time, and the steps to be communicated to the user 50 can be changed.

[0054] In this embodiment, an example of a relative 60 is shown as a digital partner, but the attributes of a digital partner are arbitrary and are not limited to a real person or a person who existed in the past, and may be, for example, a specific character or animal (e.g., a pet that user 50 loved).

[0055] Furthermore, multiple digital partners can be registered for the user 50. In this case, the user 50 may be able to select a digital partner with which to communicate. Furthermore, the conversation generation unit 21 may select a digital partner whose attributes match the content of the conversation. For example, for an urgent conversation, the conversation generation unit 21 may select a digital partner whose attributes match the urgency, thereby ensuring effective communication to avoid an emergency. Furthermore, the conversation generation unit 21 can enhance the realism of the digital partner by selecting the content and expressions of the conversation based on the digital partner's attributes, such as age and gender.

[0056] Communications made in the information processing system of this embodiment can be notified to relevant parties as appropriate. For example, if the digital partner is a real person, the content of the exchange between the user 50 and the digital partner and a video of the user 50 taken during the conversation can be sent to the real person. This facilitates communication between the real person and the user 50. Furthermore, by having a common understanding of the conversation with the digital partner between the user 50 and the real person, it becomes possible to enjoy communication including the digital partner.

[0057] Furthermore, when an emergency situation is recognized via the tablet 20 (behavioral information acquisition unit 22) or the earphone microphone 30 (behavioral information acquisition unit 32), a notification can be sent to a predetermined notification destination.

[0058] As described above, the information processing system of this embodiment allows the user 50 to have a conversation with a digital partner. Therefore, the user 50 can communicate with the digital partner as if a close friend were nearby. This can, for example, prevent dementia in the user 50 who is an elderly person living alone. Furthermore, for example, by incorporating information about improving eating habits and the latest crime trends into the conversation, it can prevent the user 50 from becoming a victim of crime and from losing their nutritional status.

[0059] Although the embodiments have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of components of the above-described embodiments. [Explanation of symbols]

[0060] 10 Servers 11 Partner Generation Unit 12 Conversation information acquisition unit 20 tablets 21 Conversation Generation Unit 22 Behavioral information acquisition unit 30 Earphone Microphone 31 Conversation Generation Unit 32 Behavioral information acquisition unit 50 users

Claims

1. a partner generation unit that generates a digital partner based on the learning; a conversation generation unit that generates a conversation of the digital partner for the user; a conversation information acquisition unit that acquires first information related to the content of the conversation; a behavior information acquisition unit that acquires second information related to the user's behavior; Equipped with An information processing system in which the conversation generation unit generates the conversation based on the first information acquired by the conversation information acquisition unit in accordance with the second information acquired by the behavioral information acquisition unit.

2. The information processing system of claim 1 , wherein the second information relates to a conversation of the user with the digital partner.

3. The information processing system according to claim 1 , wherein the digital partner communicates via a terminal device selected by the user.

4. The information processing system according to claim 1 , wherein the first information is acquired via the Internet.

5. The information processing system according to claim 1 , wherein the first information or the second information is based on an image or a sound acquired via a terminal device of the user.

6. The information processing system according to claim 1 , wherein a terminal device with which the digital partner communicates is selected based on the user's voice or the user's location.

7. a partner generation step of generating a digital partner based on the learning; a conversation generation step of generating a conversation of the digital partner for a user; a conversation information acquisition step of acquiring first information related to the content of the conversation; a behavior information acquisition step of acquiring second information related to the user's behavior; Equipped with An information processing method in which, in the conversation generation step, the conversation is generated based on the first information acquired by the conversation information acquisition step in accordance with the second information acquired by the behavioral information acquisition step.

8. a partner generation step of generating a digital partner based on the learning; a conversation generation step of generating a conversation of the digital partner for a user; a conversation information acquisition step of acquiring first information related to the content of the conversation; a behavior information acquisition step of acquiring second information related to the user's behavior; on the computer, In the conversation generation step, a program generates the conversation based on the first information acquired in the conversation information acquisition step in accordance with the second information acquired in the behavior information acquisition step.

Images