Conversational ai-robot friend apparatus using hyperscale ai, and conversational method using the same

The conversational AI-robot friend device uses a super-large AI for early dementia diagnosis and prevention by engaging in daily life conversations, performing personal authentication and cognitive assessment, and providing cognitive training.

KR103046491B1Active Publication Date: 2026-11-10RES & BUSINESS FOUND SUNGKYUNKWAN UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
KR1020230057484
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-05-03
Publication Date
2026-11-10
Estimated Expiration
2043-05-03

AI Technical Summary

Technical Problem

Existing technologies fail to accurately diagnose dementia early and prevent its progression due to reliance on conventional medical examinations, which do not adequately capture daily life activities, and lack effective methods for personal authentication and cognitive assessment in daily interactions.

Method used

A conversational AI-robot friend device utilizing a super-large AI for offline and online conversations, equipped with a conversation generation unit and memory diagnosis unit, which engages in personal authentication and assesses cognitive abilities through past memory questions, providing cognitive function enhancement training and early dementia diagnosis.

Benefits of technology

Enables early dementia diagnosis and cognitive function enhancement through continuous daily life interaction, reducing the risk of dementia progression and alleviating symptoms by analyzing conversation data and memory abilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112023049293310-PAT00010_ABST
    Figure 112023049293310-PAT00010_ABST
Patent Text Reader

Abstract

The present invention relates to a conversational AI-robot friend device using a super-large AI capable of conversing with a person like a friend, and a conversational method using the same, which can determine whether a user has symptoms of dementia through conversation with the user in daily life. In particular, the invention provides a conversational AI-robot friend device using a super-large AI that performs early diagnosis of dementia by testing the user's memory and cognitive abilities over a predetermined period in daily life, as well as automatically performing cognitive function enhancement training during conversation, through a conversational AI module comprising a conversation generation unit that creates topics for conversation with the user and a memory diagnosis unit that asks about past memories, by means of a conversational AI module that performs personal authentication through the user's voice, and automatically performs cognitive function enhancement training during conversation.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a conversational AI-robot friend device using a super-large AI and a conversation method using the same, which not only enables conversation with a user (e.g., a user registered as a master to the robot) like a friend in offline and online situations, but also determines whether the user has symptoms of dementia through conversation with the user in daily life. In particular, the invention provides a conversational AI-robot friend device using a super-large AI and a conversation method using the same, which includes a conversation generation unit that creates topics for conversation with the user and a memory diagnosis unit that asks about past memories, thereby enabling personal authentication through the user's voice, testing the user's memory and cognitive abilities over a predetermined period in daily life to diagnose dementia early, and automatically performing cognitive function enhancement training during conversation. Background Technology

[0002] Dementia is the greatest fear for those entering old age. According to one survey, nearly 40% of people selected dementia as the disease they fear most in their later years, surpassing cancer, which accounted for 20.09% of the responses. As dementia causes memory loss and renders basic daily activities impossible, it leads to significant suffering and conflict within families. Consequently, it is one of the diseases that incurs the highest social costs. Therefore, rather than being a problem for individuals to solve alone, society must address it systematically or by leveraging its technological capabilities.

[0003] Korea's aging rate is among the highest in the world. Consequently, it is also one of the countries with the fastest-growing dementia rate globally. According to the Korean Society of Biotechnology and Bioengineering, the elderly population increased by 60 percent from 6.13 million in 2013 to approximately 9.84 million in 2023, while the number of dementia patients rose by 77 percent from 570,000 to 1.01 million during the same period. Furthermore, in 2022, the number of dementia patients among the elderly population aged 80 to 84 reached 20.89%, and for those aged 85 and older, it reached 38.96%. In summary, looking at those aged 80 and older, the figure stands at 59.85%, meaning that one in two people suffers from dementia. As we face an era of centenarians and super-aging, the early diagnosis and prevention of dementia are areas where we must make urgent efforts, not only for the promotion of individual health but also for the soundness and development of society.

[0004] Fortunately, the consensus among modern medical experts is that dementia is not an incurable disease, is preventable, and can be fully overcome through management and intensified medication if diagnosed early. To lower the prevalence of dementia, early diagnosis and prevention must be implemented first and foremost.

[0005] Dementia is not diagnosed simply because of memory impairment. Dementia is a multiple cognitive disorder; for instance, it is said that a diagnosis can only be made when it impairs the ability to perform daily activities—such as making phone calls, using public transportation, or washing—which individuals previously performed well on their own. Therefore, to make such a diagnosis, it is crucial to carefully observe the individual in their daily life and collect information, rather than relying solely on conventional medical examinations. However, realistically, with the aging population, the rise of nuclear families, and an increasing number of people living alone, the number of individuals falling into these blind spots of observation and care is expected to continue growing.

[0006] To address these issues, it is expected that robots will play a significant role in accurately assessing the user's condition, diagnosing dementia early, and ultimately preventing it by engaging in conversations with users in daily life to promote brain development and activity, and by observing, collecting, and analyzing data on the user's daily living and conversational abilities. Prior art literature

[0007] Korean Patent Publication No. 2059903 (Registered Dec. 20, 2019) The problem to be solved

[0008] The present invention aims to provide a conversational AI-robot friend device utilizing a super-large AI and a conversational method using the same, which solves the problems of the aforementioned prior art by utilizing a robot capable of offline and online conversations like a friend with a user (e.g., a user registered as the master of the robot), enabling personal authentication in daily life without a separate user authentication procedure, and determining the presence of dementia symptoms and alleviating dementia symptoms through offline and online conversations with the user. More specifically, the invention aims to provide a conversational AI-robot friend device utilizing a super-large AI and a conversational method using the same, and a method for diagnosing and alleviating dementia through the same, which analyzes the user's conversation during a conversation by means of a conversational AI module comprising an interest topic search unit for finding story materials to proceed with a conversation with the user, a conversation generation unit that creates a conversation from the story materials, and a memory diagnosis unit that asks about past memories, and which determines the risk factors for dementia in the user and simultaneously performs user authentication based on the analyzed results.

[0009] The above-mentioned interest topic search unit can find story materials or conversation topics that the user might be interested in from past conversations, news on the internet, the user's TV viewing history, radio eavesdropping history, and eavesdropping history of conversations between the user and others.

[0010] In addition, to solve the problems of the aforementioned prior art, the present invention aims to provide a conversational AI-robot friend device using a super-large AI that can slow down or prevent the progression of dementia by equipping the conversational AI module with a memory diagnostic unit that helps restore the user's memory by providing the correct answer or hint for questions regarding past memories that cannot be answered, thereby solving the problems of the prior art described above, and a method for diagnosing and alleviating dementia through the same.

[0011] In addition, the present invention aims to provide a conversational AI-robot friend device using a super-large AI and a conversation method using the same, in order to solve the problems of the aforementioned prior art, wherein the robot can perform online conversations such as telephone conversations, SNS conversations, remote video conversations, and messenger conversations with a user via a wireless communication connection means, in addition to offline conversations with a user (e.g., a user registered as the master of the robot).

[0012] In addition, the present invention aims to provide a conversational AI-robot friend device using a super-large AI capable of performing online conversations between pre-registered virtual friends and a user (e.g., a user registered as a master to the robot) and a conversation method using the same, in order to solve the problems of the aforementioned prior art.

[0013] In addition, the robot in the present invention may be a chatbot, a PC (personal computer), a laptop, a mobile phone, a TV (Television), or a smart mirror.

[0014] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0015] Hereinafter, the person using the robot shall be referred to as "User," "Robot User," or "Master." For example, "Master" or "Robot User" may be a user of an AI-Robot Friend device who has been approved in advance through the user registration process provided by the App.

[0016] In addition, the "master's terminal" may be the terminal of the above-mentioned user.

[0017] Additionally, "Master's lines" may be the lines of the aforementioned user or the lines of the robot user.

[0018] As a technical means for achieving the above-mentioned technical problem, a conversational AI-robot friend device using a super-large AI according to one embodiment of the present invention comprises a robot and a conversational AI (artificial intelligence) module, wherein the robot includes: a line of sight determination means for determining whether it is within a line of sight (a predetermined distance) capable of conversing with a master; a microphone unit for eavesdropping on sound information within the line of sight (including the master's voice, telephone conversation, TV sound, radio sound, and conversation content between the master and another person); and a speaker unit for voice-transmitting robot dialogue or greeting dialogue to the master, wherein the conversational AI module includes: a voice authentication means for recognizing the master's voice from the microphone unit during a conversation and performing master authentication; a voice-to-text conversion unit for converting sound information eavesdropped on through the microphone unit into a text sentence or converting the master's voice input through the microphone unit into a sentence-form master dialogue (user dialogue); an eavesdropping memory for storing the text sentence obtained by eavesdropping; and an eavesdropping memory for storing the eavesdropping content. A conversation generation unit that generates robot lines or greeting lines to be delivered to the master through the speaker unit from conversation topics using artificial intelligence; a text-to-speech conversion unit that converts the robot lines or greeting lines generated by the conversation generation unit into speech and delivers the speech to the master through the speaker unit; and a conversation black box memory for storing conversation content (including offline and online conversation content) and corresponding keywords in chronological order using the robot lines.The robot includes a memory diagnostic unit that generates questions and corresponding answers from past memories stored in the conversation black box memory or eavesdropping memory, and determines the degree to which the master's answer matches the correct answer as a score. Thus, the robot can perform personal authentication through the master's voice during a conversation with the master, and simultaneously continuously test the master's memory and cognitive abilities over a predetermined period using the conversational AI module in daily life, thereby enabling early diagnosis of dementia as well as the performance of cognitive function reinforcement training.

[0019] In the present invention, the Master's dialogue is preferred to include the dialogue of other people conversing with the Master within the line of sight.

[0020] The past memory of the present invention is content stored in a conversation black box memory or an eavesdropping memory, and may include the owner's personal information, past conversation content, or sound information heard in the past through the microphone unit.

[0021] The above memory diagnostic unit includes: a memory generation means that provides a question generated from a past memory stored in a conversation black box memory or an eavesdropping memory and the corresponding answer; and

[0022] It may include a memory ability scoring means that determines, as a score, how closely the master's answer matches the correct answer to a question generated by the above-mentioned memory generation means.

[0023] It is preferred that the Master's personal information be stored by default in the eavesdropping memory or conversation black box memory of the present invention. The Master's personal information can be stored in the eavesdropping memory when the user is registered with the robot using an app, and it is preferred that it include at least one selected piece of information among memory content, past actions and history, hobbies, personal preferences, bucket list, family details, friendships and history, family event dates (birthdays, wedding anniversaries, etc.), sex, age, height, weight, mobile phone number, email address, SNS connection address, messenger connection address, remote video conferencing connection address, and mobile phone numbers of friends and family. Furthermore, the registration and additional management of the Master's personal information can be enabled by an app installed on the Master's terminal (user's terminal).

[0024] The memory generation means of the present invention can transmit past memories stored in a conversation black box memory or an eavesdropping memory to a super-large AI and request it to generate questions and corresponding answers to test the master's memory ability.

[0025] In addition, the above-mentioned memory generation means can enhance the Master's cognitive ability by providing the Master with the correct answer to the question or giving some hints to guide the Master to the correct answer when the Master's answer to the generated question is below a threshold score.

[0026] Another aspect of the above memory ability scoring means is that if the score of the Master's first answer to a question provided to the Master by the memory generation means is below a threshold, the memory ability scoring means controls the memory generation means to provide additional hints to the Master regarding the question, and then evaluates the Master's memory recovery ability by determining how close the Master's second answer is to the correct answer using a score.

[0027] The above-mentioned means for determining visibility may be a vision camera for visually determining whether the master is nearby.

[0028] Another aspect of the above-mentioned line of sight determination means may be a voice authentication means that captures the Master's voice from the microphone. If the Master's voice is properly captured and personal authentication is possible, it can be determined that the Master is within the line of sight.

[0029] Another aspect of the above-mentioned line-of-sight determination means may be a wireless communication connection means for determining whether a short-range communication connection with the Master's mobile phone is possible. That is, the presence of the Master within a short range can be confirmed by a short-range wireless communication protocol. Bluetooth is preferred for the above-mentioned short-range communication connection.

[0030] The above line of sight determination means determines that the owner is within the line of sight when the owner is captured by the vision camera, the owner's voice is authenticated by the voice authentication means, or a Bluetooth connection is established with the owner's mobile phone.

[0031] The above speaker unit may further be equipped with an automatic volume adjustment means that automatically increases the speaker volume when ambient noise is loud or when the distance from the master is somewhat far due to a line of sight judgment means.

[0032] If abnormal signs regarding memory and cognitive ability are detected by the above-mentioned memory ability scoring means, the control unit may provide a remote medical diagnostic service through an internet communication connection with a doctor.

[0033] Additionally, the robot is equipped with a display panel, and the control unit may display robot dialogue generated by a dialogue generation unit or questions and hints generated by a memory generation means as text on the display panel. In this case, the master can check the dialogue and question content through the text on the display panel.

[0034] The above voice authentication method automatically performs personal authentication of the owner via voice during a conversation with the robot without a separate authentication procedure, thereby providing the owner with convenience in the authentication process.

[0035] In addition, the above-mentioned conversational AI module transmits the Master's dialogue received through the microphone unit to a super-large AI on a server via the internet network, and in response, feeds back the robot dialogue generated by the super-large AI to the Master through the speaker unit, thereby allowing the conversation with the Master to continue.

[0036] In addition, the conversational AI module of the present invention may include a super-large AI within the conversational AI module.

[0037] In this case, the present invention provides the advantage of enabling a conversation between the master and the robot to take place even in an offline situation without a server or internet connection, using the interactive AI-robot friend device.

[0038] The super-large AI of the present invention is a next-generation AI that learns large-scale data using a supercomputing infrastructure with a computational speed much faster than that of a general computer, and generally has more than 150 billion parameters.

[0039] For example, the model 'GPT-3', released in 2020, has 175 billion parameters.

[0040] In addition, Naver, a South Korean portal company, announced 'Hyper Clova,' a massive AI model with 204 billion parameters, in late May 2021.

[0041] In addition, LG AI Research Institute unveiled 'EXAONE,' a massive AI with approximately 300 billion parameters, in December 2021.

[0042] The above conversational AI module transmits the Master's dialogue received through the microphone unit to a super-large AI on a server via the internet network, and in response, feeds back the robot dialogue generated by the super-large AI to the Master through the speaker unit, thereby allowing the conversation with the Master to continue.

[0043] Additionally, the control unit may include a dementia tracking management unit that observes the trend of change in the analysis results based on periodic examinations for dementia by the memory ability scoring means and informs the master of the risk of dementia, requests cognitive function enhancement training, or requests a remote medical diagnosis with a doctor.

[0044] The above dementia tracking management unit can determine the Master's dementia risk level using the above memory ability scoring means and display it on the display panel.

[0045] In addition, the above-mentioned conversational AI module uses a wireless communication connection means so that the owner can call the robot or the robot can call the owner's terminal to conduct a conversation with each other.

[0046] In addition, the above conversational AI module can have SNS conversations (e.g., Facebook), remote video conversations (e.g., WebEx, Zoom Meeting), and messenger conversations (e.g., KakaoTalk conversation) between the master and the robot via a wireless communication connection means.

[0047] The wireless communication connection means of the present invention may be a communication connection via the Internet, wireless mobile communication, Bluetooth, or Wi-Fi.

[0048] In addition, when the above conversational AI module determines that the Master is lonely by the loneliness judgment means, it can reduce the Master's loneliness through online conversations conducted between the Master and virtual friends generated by the virtual friend creation unit.

[0049] According to one embodiment of the present invention, a conversation method performed by a conversational AI-robot friend device using a super-large AI may include: a step of selecting a conversation topic by searching for materials that the master (robot user) is likely to be interested in from news of incidents and accidents and various issues on the internet, eavesdropping memory, and the contents of a conversation black box memory; a step of generating a story subject from the super-large AI based on the conversation topics and delivering it to the master through a speaker; a step of delivering the master's dialogue received through a microphone unit to the super-large AI, and in response, feeding back the dialogue generated by the super-large AI to the master through a speaker unit to continue the conversation with the master; a step of recognizing voice during the conversation to personally authenticate the master; a step of conducting a conversation between the master and the robot by the master calling the robot or the robot calling the master's terminal by means of a wireless communication connection means; a step of conducting an SNS conversation, remote video conversation, or messenger conversation between the master and the robot by means of a wireless communication connection means; and a step of conducting an online conversation between the master and virtual friends generated by a virtual friend generation unit when the master is determined to be lonely by means of a loneliness determination means.

[0050] In addition, according to one embodiment of the present invention, a method for diagnosing and alleviating dementia using a conversational method performed by a conversational AI-robot friend device using a super-large AI may include: a step of scoring the memory ability of a master (robot user) by asking the master about past memories and checking whether the answer is correct; a step of strengthening the master's memory recovery ability and cognitive ability by giving the master some hints regarding questions about past memories that cannot be answered; and a step of determining the master's dementia risk level based on the results analyzed by a memory diagnosis unit.

[0051] In addition, the robot of the present invention can be replaced by a chatbot equipped with a conversational AI module, a PC, a smart mirror, a TV, a laptop, or a mobile phone.

[0052] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0053] According to the means for solving the problem of the present invention described above, a conversational AI-robot friend device utilizing a super-large AI capable of conversing with a user (e.g., a user registered as the master of the robot) like a friend in offline and online situations is provided, which not only authenticates the user through conversation in daily life and diagnoses dementia early by testing the user's memory and cognitive abilities over a predetermined period, but also automatically performs cognitive function enhancement training during conversation, is provided, a conversational method using the same, and a method for diagnosing and alleviating dementia using the same.

[0054] In addition, the present invention provides an interactive AI-robot friend device using a super-large AI that can reduce the loneliness of a master through online conversations between the master and virtual friends generated by a virtual friend generation unit, and a conversation method using the same.

[0055] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist. Brief explanation of the drawing

[0056] FIGS. 1, FIGS. 2a and FIGS. 2b are embodiments of an interactive AI-robot friend device using a super-large AI according to the present invention. Figure 3 is an example of the contents stored in the eavesdropping memory and the conversation black box memory. FIG. 4a is an example of a memory diagnosis unit of an interactive AI-robot friend device according to the present invention. FIG. 4b is another embodiment of the memory diagnostic unit of an interactive AI-robot friend device according to the present invention. FIG. 5a shows an example of a friend conversation app for registering virtual friends in advance and a virtual friend personal information list memory 1 for storing personal information of virtual friends in order to have an online conversation between the master and virtual friends existing virtually on the robot. Figure 5b shows an example in which an online conversation connection is established between a virtual friend and a master through a communication connection between a master terminal and a wireless communication connection means. FIG. 6 shows an example of using a robot of a conversational AI-robot friend device using a super-large AI as a smart mirror (600). Specific details for implementing the invention

[0057] Embodiments of the present invention are described below with reference to the attached drawings to enable those skilled in the art to easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0058] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" or "indirectly connected" with other elements interposed between them.

[0059] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0060] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0061] Preferred embodiments of the present invention will be described in detail below using the attached drawings.

[0062] Hereinafter, the robot in the present invention may be used in combination with an AI-robot friend device.

[0063] Hereinafter, the virtual friend in the present invention includes a robot.

[0064] Hereinafter, the terms "master," "robot user," or "user" in the present invention may be used interchangeably with "user" of the conversational AI-robot friend device.

[0065] Hereinafter, the dialogue content that is transmitted to the master without alteration or modification by the dialogue generated by the super-large AI in the present invention is referred to as robot's dialogue.

[0066] Hereinafter, robot dialogue that takes place online between a virtual friend and a master in the present invention is referred to as virtual friend dialogue.

[0067] Hereinafter, in the present invention, past memory refers to the content of the Master's past conversations, the Master's personal information, and past sound information that the Master heard, stored in the conversation black box memory or the eavesdropping memory.

[0068] Hereinafter, in the present invention, memory refers to a conversation black box memory or an eavesdropping memory.

[0069] FIGS. 1, FIGS. 2a and FIGS. 2b are embodiments of a conversational AI-robot friend device using a super-large AI according to the present invention, wherein the conversational AI-robot friend device includes a robot (200) and a conversational AI module (500) integrated within the body of the robot (200).

[0070] The control unit (53) of the above conversational AI module (500) includes a visibility determination means (44) for determining whether there is a visible distance or visible range where conversation with the master is possible. In the present invention, the visibility determination means (44) can determine whether the master's appearance is captured in an image input from a vision camera (63), a face shape authentication means (63a) for determining whether the master's voice is captured at a predetermined volume or higher, a voice authentication means (71) for determining whether the master's voice is captured at a predetermined volume or higher, or a wireless communication connection means (37) for determining whether the master's wearable device (not shown) or mobile phone (20) is properly connected via Bluetooth, by setting a predetermined frequency cutoff for the signal.

[0071] For example, if the frequency cutoff is set to 10 times, the visibility determination means (44) can determine that the owner is not within the visibility range when the owner's appearance is not captured more than 10 times per minute by the face shape authentication means (63a), or when the volume of the owner's voice recognized by the voice authentication means (71) is not captured more than a predetermined volume more than 10 times per minute, or when the Bluetooth connection with the owner's mobile phone (20) is not established more than 10 times per minute.

[0072] The above-mentioned wearable device (not shown) may be any one selected from a smart watch, an electronic ring, an electronic necklace, or an electronic bracelet.

[0073] The above face shape authentication means (63a) can be pre-learned by images of the Master's face shape from various angles labeled as Master, and subsequently by a face shape recognition artificial intelligence neural network that recognizes the Master's face shape obtained by a vision camera (63).

[0074] Since the vision camera (63) cannot always see the front of the master's face, it must be able to recognize the shape of the master's face taken from various angles other than the front. Therefore, the face shape authentication means (63a) can recognize the master's face taken from various angles other than the front.

[0075] In the present invention, the facial shape recognition artificial intelligence neural network is supervised learning through images of the owner's face from various angles (e.g., front, back, and side images) obtained by a vision camera (63) during the process of registering the owner's personal information, and after the registration of personal information, it is preferred to optimize through semi-supervised learning based on the supervised learning of the facial shape recognition artificial intelligence neural network.

[0076] The above voice authentication means (71) may be equipped with a volume check means capable of checking whether the master's voice is greater than a predetermined volume.

[0077] The robot (200) includes a microphone unit (81b) for eavesdropping on external sounds or conversation sounds within the line of sight; and a speaker unit (81a) for delivering robot's dialogue or greeting lines to the master, and the conversational AI module (500) includes a voice authentication means (71) for recognizing the master's voice input from the microphone unit (81b) and determining whether it is the master; an eavesdropping memory (28a) for converting the eavesdropping content within the line of sight input from the microphone unit (81b) into a sentence form by a voice-to-text conversion unit (27a) and storing it; a dialogue generation unit (70) for generating robot dialogue and greeting lines to be delivered to the master by artificial intelligence; and a text-to-voice conversion unit (27b) for converting the robot dialogue or greeting lines generated by the dialogue generation unit (70) into voice and delivering voice to the master through the speaker unit (81a). The robot (200) includes a conversation black box memory (28b) for storing the content of a conversation with the master using robot dialogue generated by the conversation generation unit (70); a keyword identification artificial intelligence (400) for classifying the keywords of the sentences; and a memory diagnosis unit (80) that extracts and generates questions and correct answers to the questions from past memories stored in the conversation black box memory (28b) or the eavesdropping memory (28a), and determines the degree to which the master's answer to the generated questions matches the correct answer as a score. Thus, the robot (200) can authenticate the master's voice while conversing with the master, and at the same time, continuously test the master's memory and cognitive abilities over a predetermined period by the conversational AI module (500) in daily life to diagnose dementia early and perform cognitive function reinforcement training.

[0078] The above voice authentication means (71) enables personal authentication of the master by voice during a conversation with the robot (200) without a separate authentication procedure, thereby providing the master with convenience in the authentication procedure.

[0079] In addition, the contents stored in the conversation black box memory (28b) or the eavesdropping memory (28a) can be used to test the Master's ability to remember past conversations.

[0080] The eavesdropping memory (28a) and conversation black box memory (28b) of the present invention are preferably made of a storage means including NAND flash, DRAM, or HDD (Hard Disc Drive), but are more preferably made of a semiconductor memory.

[0081] FIG. 2a is an example of a conversational AI module (500) of a conversational AI-robot friend device for having online or offline conversations according to the present invention.

[0082] In the present invention, offline conversation refers to the conversation between the master and the robot (200) through the microphone unit (81b) and the speaker unit (81a), and online conversation refers to the conversation between the master terminal (300) and the robot (200).

[0083] The above master terminal (300) includes a device capable of text, voice, and video calls via a wireless communication connection, such as a mobile device, PC, smart watch, smart glasses, or XR headset device.

[0084] The keyword identification artificial intelligence (400) of the present invention can be implemented by an artificial intelligence neural network trained using sentences labeled according to various keywords, and can identify and extract one or more keywords by analyzing given sentences or online conversation content.

[0085] For example, the eavesdropping memory (28a) of the present invention can store a text sentence obtained by a voice-to-text conversion unit (27a) that converts sound information eavesdropped on through a microphone unit (81b) into a sentence form, along with the corresponding topic word. At this time, the topic word identification artificial intelligence (400) can analyze the text sentence to find the corresponding topic word, and the eavesdropping memory (28a) stores the text sentence and the corresponding topic word together.

[0086] In addition, the conversation black box memory (28b) of the present invention can also store conversation sentences and corresponding keywords together.

[0087] The above sound information may be intercepted content including the sound of the Master speaking with another person or sounds the Master heard (e.g., TV sound, radio sound).

[0088] The interest topic search unit (77) selects conversation topics that include topics the owner has been statistically interested in for a certain period from the contents stored in memory (28a, 28b).

[0089] Since the corresponding topic words are already assigned and stored together with each sentence in the memory (28a, 28b), the interest topic search unit (77) can statistically determine which topic word the owner is more interested in by calculating the number of identical topic words stored in the memory (28a, 28b) for a predetermined period.

[0090] Another aspect of the above-mentioned interest topic search unit (77) is that it can select conversation topics from news about incidents and accidents and various issues on the internet and YouTube using a super-large AI (16).

[0091] To this end, the interest topic search unit (77) may request the super-large AI (16) to select and provide feedback on recent accidents and issues and conversation topics of interest from news and various issues on the internet and YouTube.

[0092] The above keywords may include personality, appearance, cosmetics, beauty, fortune, weather, fine dust, natural disaster, insurance, real estate, exchange rate, loan, apartment, property, stock investment, bank, economy, prices, tax, cryptocurrency, international issues, housework (cooking, cleaning, laundry), fashion (clothing, hair), youth issues, career issues, family issues, child-rearing, retirement life, friend, marriage, divorce, holidays, vacation, living expenses, remodeling, home appliance breakdown and after-sales service, hobby, religion, business, history, geography, science and technology, information and communication, law, military, war, oil, space, nature, biology, incidents and accidents, education, studying abroad, immigration, college admission, employment, medical care, health, disease, death, drinking, smoking, drugs, medicine (nutritional supplements), shopping, dining out, transportation, automobile, traffic accident, sports, indoor exercise, internet, movie, art, drama, novel, music, song, game, celebrity, food, pet, tourism, political issues, population issues, social issues, environmental issues, and related words.

[0093] For example, words related to the topic keyword "incidents and accidents" could be car accidents, fires, airplane crashes, ship sinkings, crimes, etc., and words related to car accidents could be car theft, murder cases, violent incidents, theft cases, etc.

[0094] The conversation generation unit (70) of the present invention is for generating greeting lines and robot lines to conduct a conversation between the master and the robot (200), and may include a greeting generation means (70a) that provides everyday greeting lines within a visible range and a chatting generation means (70b) that generates robot lines for conversation topics extracted by the interest topic search unit (77).

[0095] In the present invention, the greeting generating means (70a) is preferred to generate a greeting such as a greeting and deliver it to the master through the speaker unit (81a) after a predetermined amount of time (e.g., 5 hours or more) has elapsed since the robot (200) had a conversation with the master within the line of sight and simultaneously passed the International Date Line.

[0096] For example, the greeting generation means (70a) can generate greeting lines such as "Hello, Master, the weather is cold today." or "Hello. Did you sleep well? Master, the weather is cold today. Did you have breakfast?" The generated greeting lines can be delivered to the Master through the speaker unit (81a) via the text-to-speech conversion unit (27b).

[0097] Another aspect of the above greeting generation means (70a) is to request the super-large AI (16) to generate a greeting line. Subsequently, the greeting generation means (70a) can receive feedback on the greeting line generated by the super-large AI (16) and transmit it to the master through the text-to-speech conversion unit (27b) and the speaker unit (81a).

[0098] For example, the greeting generation means (70a) can request a greeting message from the super-large AI (16) as follows, and the super-large AI (16) can receive feedback on the generated greeting message and deliver it to the master.

[0099] Example of a greeting generation means (70a) requesting a greeting message from a super-large AI (16):

[0100] Try to make a morning greeting sentence for me while thinking me as your friend.

[0101] Example of a greeting generated by super-large AI (16):

[0102] Good morning, friend! I hope you have a great start to your day today!

[0103] The chatting generation means (70b) can transmit a conversation topic to the super-large AI (16) via the wireless communication connection means (37), and at the same time, request the super-large AI (16) to generate the first robot's dialogue using the conversation topic. At this time, the chatting generation means (70b) can receive feedback (receive) the first robot's dialogue generated by the super-large AI (16) via the wireless communication connection means (37) and transmit it to the master through the text-to-speech conversion unit (27b) and the speaker unit (81a). Afterwards, the chatting generation means (70b) catches the master's dialogue regarding the first robot's dialogue from the microphone unit (81b) and transmits it to the super-large AI (16) via the wireless communication connection means (37). At this time, the chatting generation means (70b) can continue the conversation by receiving feedback on the second robot's dialogue from the super-large AI (16) and transmitting it back to the master.

[0104] The above master's dialogue can be obtained by a voice-to-text converter (27a) that converts the master's voice input through the microphone unit (81b) into a sentence.

[0105] For example, after delivering a greeting message generated by a greeting generation means (70a) to the master, the chat generation means (70b) collaborates with a super-large AI (16) to generate a first robot message regarding the topic of conversation and delivers it to the master, and the robot (200) can develop a conversation with the master by delivering the master's response to the super-large AI (16) again.

[0106] At this time, the control unit (53) controls the voice-to-text conversion unit (27a) and the wireless communication connection means (37) so that the master's dialogue input through the microphone unit (81b) is transmitted to the super-large AI (16) on the server (15) via the internet network (18), and in response, controls the chat generation means (70b) and the text-to-voice conversion unit (27b) so that the robot dialogue generated by the super-large AI (16) is fed back to the master through the speaker unit (81a).

[0107] The memory diagnostic unit (80) can generate questions and answers to the questions from past memories stored in memory (28a, 28b) to test the master's memory ability, and can determine the degree to which the master's answer to the question matches the answers as a score.

[0108] The above memory diagnostic unit (80) may request the super-large AI (16) to generate questions and answers to test the master's memory ability by reading out a portion of the past memories stored in memory (28a, 28b) and transmitting it to the super-large AI (16) in order to generate questions about past memories and answers to the questions, and to provide feedback.

[0109] Additionally, the memory diagnostic unit (80) can read out a portion of past memories stored in memory (28a, 28b) and transmit it to the super-large AI (16), and request the super-large AI (16) to generate hints and answers to test the master's memory ability and provide feedback.

[0110] Some of the above past memories are preferred to be sentence content stored in a predetermined past time interval randomly selected from memory (28a, 28b). The older the randomly selected sentence content is in the past, the higher the score can be assigned when evaluating memory ability.

[0111] In addition, the robot (200) of the conversational AI-robot friend device of the present invention is equipped with a display panel (14) so ​​as to display robot dialogue generated by the conversation generation unit (70) or questions and hints generated by the memory diagnosis unit (80) as text. In this case, the owner can check these text contents on the display panel (14).

[0112] Additionally, the conversational AI module (500) may include a dementia tracking management unit (39) that observes the trend of change in the analysis results of periodic examinations for dementia by the memory diagnosis unit (80) and informs the owner of the risk of dementia, requests training to strengthen cognitive function, or requests a remote medical diagnosis with a doctor.

[0113] The above dementia tracking management unit (39) can determine the risk of dementia of the master based on the memory ability evaluation score.

[0114] In addition, the conversational AI module (500) of the robot (200) can conduct an online conversation between the master and the robot (200) by means of a wireless communication connection means (37).

[0115] The online conversation of the present invention may include telephone conversation, SNS conversation (e.g., Facebook), remote video conversation (e.g., WebEx, Zoom Meeting, video call), or messenger conversation (e.g., KakaoTalk conversation, text conversation, email conversation).

[0116] For example, the robot (200) may receive a call from the master's mobile phone (20) or the robot (200) may call the master's mobile phone (20) to conduct an online phone conversation with the master. At this time, the conversational AI module (500) of the robot (200) can transmit robot dialogue generated by the super-large AI (16) to the master's mobile phone (20) via a wireless communication connection means (37), and receive the master's voice via the wireless communication connection means (37) and transmit it to the super-large AI (16) to conduct a phone conversation between the robot (200) and the master.

[0117] The online conversation sentences between the above master and the robot (200) and the corresponding keywords are stored together in the conversation black box memory (28b) and can be used when evaluating the master's memory ability regarding past memories.

[0118] FIG. 2b is another embodiment of the conversational AI module (500) of the conversational AI-robot friend device according to the present invention, which includes a super-large AI (16) within the conversational AI module (500).

[0119] In this case, since it is included in the super-large AI (16) within the conversational AI module (500), it provides the advantage of minimizing the delay time due to the internet communication connection required to obtain robot dialogue from the super-large AI (16).

[0120] For example, the chatting generation means (70b) can directly transmit the conversation topic determined by the interest topic search unit (77) to the super-large AI (16), and at the same time, request the super-large AI (16) to generate the first robot's dialogue using the said conversation topic and provide feedback. At this time, the chatting generation means (70b) can receive the first robot's dialogue generated by the super-large AI (16) and transmit it to the master through the text-to-speech conversion unit (27b) and the speaker unit (81a). Afterwards, the chatting generation means (70b) receives the master's dialogue regarding the first robot's dialogue through the microphone unit (81b) and the speech-to-text conversion unit (27a) and transmits it to the super-large AI (16), and at this time, the super-large AI (16) generates the second robot's dialogue.

[0121] Additionally, the memory diagnostic unit (80) may request the super-large AI (16) to generate questions and answers to test the master's memory ability by reading out parts of past memories stored in memory (28a, 28b) and delivering them directly to the super-large AI (16) in order to generate questions about past memories and answers to the corresponding questions.

[0122] FIG. 3(a) is an example of the contents stored in the eavesdropping memory (28a), and FIG. 3(b) is an example of the contents stored in the conversation black box memory (28b).

[0123] In each memory (28a, 28b), keywords and storage times may be stored along with sentence content (dialogue content). At this time, one or more keywords may be selected for each sentence.

[0124] The keywords stored in the eavesdropping memory (28a) can be selected by a keyword identification artificial intelligence (400) that finds keywords for text sentence content.

[0125] Additionally, the topic word stored in the conversation black box memory (28b) may be a conversation topic selected by the interest topic search unit (77).

[0126] For example, sound information intercepted through the microphone unit (81b) is converted into a text sentence by the voice-to-text conversion unit (27a), and the keyword identification artificial intelligence (400) classifies the keywords of the text sentence. At this time, the text sentence, keywords, and storage time (current time) can be stored together in the interception memory (28a).

[0127] It is preferred that the owner's personal information be stored by default in the eavesdropping memory (28a) to the conversation black box memory (28b) of the present invention.

[0128] FIG. 4a is an example of a memory diagnostic unit (80) of an interactive AI-robot friend device according to the present invention, which can evaluate the memory ability of the master through cooperation with a super-large AI (16).

[0129] For example, the memory diagnostic unit (80) may retrieve past memories stored in memory (28a, 28b) and transmit them to the super-large AI (16), and may request the super-large AI (16) to generate questions and corresponding answers to test the Master's memory ability from the past memories. Additionally, the memory diagnostic unit (80) may receive the questions generated by the super-large AI (16), transmit the content of the questions to the Master through the text-to-speech conversion unit (27b) and the speaker unit (81a), and wait for the Master's answer. The memory diagnostic unit (80) may capture the Master's answer to the question from the microphone unit (81b), transmit these answers to the super-large AI (16) through the speech-to-text conversion unit (27a) and the wireless communication connection means (37), and may request the super-large AI (16) to convert the Master's answer into a score and inform the Master how much it matches the correct answer.

[0130] Additionally, the memory diagnostic unit (80) can request the super-large AI (16) to generate hints and corresponding answers to test the Master's memory ability from the past memories while transmitting past memories stored in the memory (28a, 28b) to the super-large AI (16). The memory diagnostic unit (80) can receive the hints generated by the super-large AI (16), transmit the hint content to the Master through the text-to-speech conversion unit (27b) and the speaker unit (81a), and wait for the Master's response. The memory diagnostic unit (80) can capture the Master's response to the hints from the microphone unit (81b), transmit these responses to the super-large AI (16) through the speech-to-text conversion unit (27a) and the wireless communication connection means (37), and request the super-large AI (16) to convert the Master's response into a score and inform the Master how closely it matches the answer. In this case, the higher the score, the better the Master's memory recovery ability is judged to be.

[0131] Preferably, the memory diagnostic unit (80) provides a hint to the master again and waits for the master's answer if the master's first answer to the question generated by the super-large AI (16) is below a threshold value. Afterward, the memory diagnostic unit (80) transmits the master's second answer to the hint to the super-large AI (16) and requests the super-large AI (16) to convert the second answer into a score and inform the master how closely it matches the correct answer, thereby allowing the evaluation of the master's memory recovery ability to proceed.

[0132] FIG. 4b is another embodiment of a memory diagnostic unit (80) of an interactive AI-robot friend device according to the present invention, wherein the memory diagnostic unit (80) is composed of a memory generation means (73) and a memory ability scoring means (74) to evaluate the memory and cognitive abilities of the master.

[0133] The memory generation means (73) can receive feedback from the super-large AI (16) by transmitting a portion of past memories stored in memory (28a, 28b) to the super-large AI (16) and by requesting the super-large AI (16) to generate questions and answers to test the master's memory ability from the past memories.

[0134] At this time, the memory generation means (73) can convey the above question to the master through the text-to-speech conversion unit (27b) and the speaker unit (81a).

[0135] Additionally, the memory generation means (73) can receive feedback from the super-large AI (16) regarding the hints and answers to test the master's memory ability by requesting the super-large AI (16) to generate a part of the past memory stored in the memory (28a, 28b).

[0136] At this time, the memory generation means (73) can transmit the above hint to the master through the text-to-speech conversion unit (27b) and the speaker unit (81a).

[0137] The memory generation means (73) can share the correct answer provided by the super-large AI (16) with the memory ability scoring means (74), and the memory ability scoring means (74) can receive the answer from the master, calculate the degree to which the answer is close to the correct answer by converting it into a score, and share the score with the memory generation means (73).

[0138] For example, the memory ability scoring means (74) captures the master's first answer to a question generated by the super-large AI (16) through the microphone unit (81b) and the voice-to-text conversion unit (27a), calculates how close it is to the correct answer by converting it into a score, and if the score is below a threshold, the memory generation means (73) provides the hint to the master through the text-to-voice conversion unit (27b) and the speaker unit (81a), and then the memory ability scoring means (74) captures the master's second answer through the microphone unit (81b) and the voice-to-text conversion unit (27a), compares it with the correct answer, and determines how close the master's second answer is to the correct answer by calculating a score, thereby conducting an evaluation of the master's memory recovery ability.

[0139] FIG. 5a shows an example of a friend conversation app (48) for registering virtual friends in advance and a virtual friend personal information list memory 1 (23a) for storing personal information of registered virtual friends in order to perform online conversations between virtual friends and the master on the robot (200).

[0140] In the present invention, virtual friends include robots. Therefore, the online conversation of the present invention is applicable not only to virtual friends but also to robots.

[0141] The registration of the above virtual friends can be done by the friend conversation app (48) installed on the master terminal (300) and the virtual friend personal information list memory 1 (23a).

[0142] At this time, you can select and register the desired virtual friends on the friend chat app (48) or change the corresponding options or personal information of the virtual friends.

[0143] In the same way, the corresponding options or personal information of the robot (200) can be changed.

[0144] The above master terminal (300) may be any one of the selected electronic devices among a mobile device, PC, smart watch, smart glasses, and XR headset device.

[0145] The contents of the virtual friend personal information list memory 1 (23a) on the above master terminal (300) are preferred to be shared by the virtual friend personal information list memory 2 (23b) on the conversational AI module (500).

[0146] To this end, the contents of the virtual friend profile list memory 1 (23a) are received by the robot's wireless communication connection means (37) and copied to the virtual friend profile list memory 2 (23b) on the conversational AI module (500).

[0147] The above personal information is preferred to include one or more selected pieces of information among speaker ID, voice ID, language (nationality), name (nickname), age, gender, religion, place of residence, place of birth, occupation, education level, hobbies, personal preferences, social media address, and email address.

[0148] For example, if the Master attempts to converse with a virtual friend whose nationality (language) is set to English in their personal information, the Master becomes able to converse with the virtual friend in English and study English.

[0149] The friend conversation app (48) of the present invention includes a virtual friend creation unit (not shown) for creating virtual friends, and the virtual friend creation unit (not shown) may include a face creation unit capable of restoring and reproducing the face of a virtual friend by a speaker ID (speaker identification) assigned to each virtual friend; a voice creation unit capable of restoring and reproducing the voice by a voice ID assigned to each virtual friend; and a virtual face expression creation unit that changes the facial expression and lip shape of the virtual friend by being synchronized with the voice of the virtual friend.

[0150] The above-mentioned face generation unit and virtual face expression generation unit can be implemented by artificial intelligence.

[0151] The face generation unit allows the owner to select the face of the virtual friend they want on the friend conversation app (48) during virtual friend registration, and each virtual friend's face is assigned a speaker ID (speaker identification), so that later the friend conversation app (48) can restore and play the virtual friend's face by the speaker ID.

[0152] In other words, the face generation unit not only provides various options for the virtual friend's facial features in the display window during virtual friend registration, but can also restore and play the virtual friend's face later using the speaker ID.

[0153] Additionally, the voice generation unit can select a voice or tone desired by the owner on the friend conversation app (48) during virtual friend registration, and can restore and play the corresponding voice by the voice ID to output it.

[0154] That is, the voice generation unit not only provides various options for the voice of the virtual friend on the friend conversation app (48), but can also restore and play the voice later using the voice ID.

[0155] In the present invention, the initial values ​​of virtual friends and their personal information in the virtual friend personal information list memory 1 (23a) are preferred to be automatically generated randomly in 10 or fewer people according to the owner's preferences as shown in the owner's personal information, and if a change is desired, it is possible to change, add, and delete using the friend chat app (48).

[0156] For example, the owner can use a friend chat app (48) on a mobile phone (20) to find and register desired virtual friends or change the personal information of virtual friends.

[0157] Additionally, the owner can use the owner's mobile phone (20) to add or modify virtual friends on the friend conversation app (48), and can select one of the virtual friends they want to talk to and press the call (20a) button to have a phone conversation. In this case, the owner can see the face corresponding to the virtual friend's speaker ID through the display window of the friend conversation app (48) on the mobile phone (20), and can hear the virtual friend's phone voice modulated with a tone corresponding to the voice ID.

[0158] Additionally, the memory contents of the virtual friend profile list registered or modified by the friend conversation app (48) are preferred to be shared by the other remaining master terminals (300) (electronic devices including mobile devices, PCs, smartwatches, smart glasses, and XR headsets).

[0159] Additionally, during a video call between the master and the robot, the robot (200) can transmit a photo of the robot's own face or a video of the scene, captured by the image sensor (21) of the robot hand, to the master's terminal (300) via a wireless communication connection means (37).

[0160] FIG. 5b shows an example of an interactive AI module (500) for conducting an online conversation between a master and one selected virtual friend among multiple virtual friends.

[0161] In this example, one of the virtual friends requests an online conversation with the Master, allowing the Master to have a conversation.

[0162] Additionally, in this example, the owner can select one of the virtual friends and request an online conversation to proceed with the chat.

[0163] In addition, in this example, an online conversation can be conducted between the master and the robot (200).

[0164] The above online conversation may include telephone conversations, SNS conversations, remote video conversations, or messenger conversations.

[0165] In the present invention, in order to individually store and manage the online conversation content between the master and each virtual friend for each virtual friend, the conversation black box memory (28b) can be composed of multiple black boxes. That is, the online conversation content and corresponding keywords shared between the virtual friend and the master can be stored in a black box independently assigned to each virtual friend.

[0166] For example, Black Box 1 (40a) is assigned to virtual friend 1, Black Box 2 (40b) to virtual friend 2, and Black Box 3 (40c) to virtual friend 3.

[0167] In addition, separately from this, a black box 4 (40d) may be allocated for online and offline conversations between the robot (200) and the master.

[0168] Therefore, the content of online conversations between the master and virtual friends is stored in black boxes (40a, 40b, 40c) that exist individually for each virtual friend, so that privacy between virtual friends is maintained and can also be used as a past memory to continue the conversation during the next online conversation.

[0169] In addition, in this example, the control unit (53) of the conversational AI-robot friend device includes a loneliness determination means (46) for determining whether the master is in a lonely state. The loneliness determination means (46) can determine that the master is in a lonely state if the cumulative amount of conversation between the master and a third party is less than or equal to a predetermined value by a voice authentication means (71) that performs authentication of the master's voice.

[0170] In this case, the control unit (53) can select one of the virtual friends and control it to make a call to the master terminal (300) through the auto-dialing means (78).

[0171] Another aspect of the above loneliness determination means (46) is that if the amount of bright facial expression activity of the master is below a predetermined value for a predetermined period of time or longer by the facial expression recognition means, the master can be determined to be in a lonely state.

[0172] The above amount of bright facial expression activity is preferred to be calculated as the ratio of bright facial expressions during a specified period.

[0173] The above facial expression recognition means can be pre-trained by an image dataset labeled with various facial expressions and subsequently by an artificial intelligence neural network that recognizes the facial expression of the master obtained by a vision camera (63).

[0174] The image dataset labeled with the above various facial expressions is preferred to be a database of facial images labeled with grades such as sad expression, neutral expression, smiling expression, big smile expression, and happy expression, and the amount of bright facial expression activity can be calculated through Equation 1.

[0175]

[0176] FIG. 5b shows an example in which an online conversation connection between a virtual friend and a master is established by a communication connection between a master terminal (300) and a wireless communication connection means (37).

[0177] In the case of an online conversation using voice, the conversational AI module (500) receives a virtual friend dialogue generated by the super-large AI (16) via an internet connection means (33), and transmits it as voice to a friend conversation app (48) on the master terminal (300) via a conversation generation unit (70), a text-to-speech conversion unit (27b), and a wireless communication connection means (37). Additionally, the conversational AI module (500) receives the master's voice from the master terminal (300) via a wireless communication connection means (37), transmits it to the super-large AI (16) via a voice-to-text conversion unit (27a), a conversation generation unit (70), and an internet connection means (33), and requests the generation of the next virtual friend dialogue, thereby enabling an online voice conversation between the virtual friend and the master.

[0178] In the same way, voice conversations between the robot (200) and the master can be conducted online.

[0179] On the other hand, in the case of online conversation using text, the conversational AI module (500) receives a virtual friend dialogue generated by the super-large AI (16) through an internet connection means (33) and transmits it to a friend conversation app (48) residing on the master terminal (300) through a wireless communication connection means (37). Subsequently, the master text generated from the master terminal (300) is received through a wireless communication connection means (37) and transmitted to the super-large AI (16) through an internet connection means (33), and by requesting the generation of the next virtual friend dialogue, a text conversation between the virtual friend and the master can proceed.

[0180] In the same way, text conversations between the robot (200) and the master can be conducted online.

[0181] Another aspect of the conversational AI module (500) according to Fig. 5b is that the owner can select one of the virtual friends and make a call using the friend conversation app (48) on the owner's terminal (300). In this case, the corresponding speaker ID information of the selected virtual friend is transmitted to the conversational AI module (500).

[0182] The speaker selection unit (29) selects the corresponding speaker ID of the virtual friend selected by the master terminal (300) through a wireless communication connection means (37). Receive the corresponding speaker ID of a virtual friend randomly selected by the loneliness judgment means (46).

[0183] Past online conversation content extracted from a black box corresponding to a speaker ID can be transmitted to an interest topic search unit (77) that derives conversation topics.

[0184] Additionally, personal information extracted from the virtual friend personal information list memory 2 (23b) corresponding to the speaker ID can be transmitted to the interest topic search unit (77) that derives conversation topics.

[0185] For example, if the Master selects one of the virtual friends and calls them using the friend chat app (48) on the Master's terminal (300), The speaker selection unit (29) extracts the speaker ID from the received signal of the wireless communication connection means (37) to determine which of the virtual friends the master has called, and the control unit (53) can read out the personal information of the virtual friend corresponding to the speaker ID from the virtual friend personal information list memory 2 (23b) and transmit it to the interest topic search unit (77).

[0186] Additionally, the control unit (53) can read the contents of the black box corresponding to the speaker ID and transmit them to the interest topic search unit (77) in order to derive a conversation topic from the past online conversation between the virtual friend and the master.

[0187] At this time, the above-mentioned topic of interest search unit (77) derives conversation topics from the contents of the black box and personal information that are read and transmitted, and the chat generation means (70b) of the conversation generation unit (70) generates virtual friend dialogue with the help of the super-large AI (16) using the derived conversation topics, and can transmit this to the friend conversation app (48) on the master terminal (300) via the text-to-speech conversion unit (27b) and the wireless communication connection means (37).

[0188] Additionally, the conversational AI module (500) receives the master's voice from the master terminal (300) through a wireless communication connection means (37), transmits it to the super-large AI (16) via a voice-to-text converter (27a), a chat generation means (70b), and an internet connection means (33), and then the chat generation means (70b) receives feedback on the next virtual friend dialogue from the super-large AI, thereby allowing the phone conversation between the virtual friend and the master to continue.

[0189] Another aspect of the conversational AI module (500) according to FIG. 5b is that the control unit (53) is equipped with a loneliness determination means (46) for determining whether the master is lonely, and if the master is determined to be lonely, the loneliness determination means (46) can select one of the virtual friends as the caller and call the master. In this case, the master's loneliness can be reduced.

[0190] The above phone caller can be identified by a speaker ID uniquely assigned to each virtual friend.

[0191] For example, the loneliness determination means (46) can make a call to the master terminal (300) through the auto-dialing means (78) using the speaker ID of a randomly selected virtual friend. In this case, the friend conversation app (48) on the master terminal (300) displays the face and name of the virtual friend corresponding to the speaker ID on the master terminal (300), so that the master can check who is calling.

[0192] Additionally, the control unit (53) can read out personal information of a corresponding virtual friend from a virtual friend personal information list memory 2 (23b) corresponding to a speaker ID provided by the speaker selection unit (29), and read out the contents of a black box corresponding to the speaker ID and transmit them to the interest topic search unit (77).

[0193] At this time, the above-mentioned topic of interest search unit (77) derives conversation topics from the contents of the black box and personal information that were extracted, and the chat generation means (70b) of the conversation generation unit (70) generates virtual friend dialogue with the help of the super-large AI (16) using the derived conversation topics, and transmits this to the friend conversation app (48) on the master terminal (300) via the text-to-speech conversion unit (27b) and the wireless communication connection means (37) so that a phone conversation between the virtual friend and the master can be conducted.

[0194] FIG. 6 is an embodiment in which a robot of a conversational AI-robot friend device using a super-large AI is used as a smart mirror (600), and the conversational AI-robot friend device includes a smart mirror (600) and a conversational AI module (500) integrated within the body of the smart mirror (600).

[0195] At this time, the owner can talk while looking at the face of a virtual friend or the face of a robot (60) displayed on the smart mirror (600).

[0196] The vision camera (63) captures and recognizes the shape of the master's face, the microphone (81b) listens to external sounds or conversation sounds within the line of sight, and the speaker unit (81a) delivers robot dialogue to the master in voice.

[0197] The above conversational AI module (500) may include a control unit (53), a face shape authentication means (63a), a voice authentication means (71), an eavesdropping memory (28a), a conversation generation unit (70), a topic of interest identification unit (77), a conversation black box memory (28b), a keyword identification artificial intelligence (400), and a memory diagnosis unit (80) as exemplified in the present invention.

[0198] The smart mirror (600) can display text of robot dialogue generated by the conversation generation unit (70) or questions and hints generated by the memory diagnosis unit (80). In this case, the owner can check these text contents on the smart mirror (600).

[0199] In addition, the risk of dementia of the owner can be displayed on the smart mirror (600).

[0200] As can be seen from the embodiments shown, the robot in the present invention can be replaced by a chatbot, a PC (personal computer), a laptop, a mobile phone, a smart mirror, or a television.

[0201] Meanwhile, a conversation method using a conversational AI-robot friend device according to one embodiment of the present invention includes: a step S101 of determining whether the robot is within a line of sight where it can converse with the master (robot user) by means of a line of sight determination means; a step S103 of storing the conversation content between the master and the robot in a conversation black box memory (28b); a step S104 of selecting a conversation topic by requesting the super-large AI (16) by means of an interest topic search unit (77); a step S106 of the chat generation means (70b) requesting the super-large AI (16) to generate a robot line based on the conversation topic selected in step S104 or the master's line (robot user's line), and delivering the generated robot line to the master; and a step S108 of authenticating the master by recognizing voice during the conversation by means of a voice authentication means (71), thereby enabling offline conversation between the master (robot user) and the robot.

[0202] The above conversation method may further include the step of the master calling the robot (200) using a mobile phone (20) or the robot (200) making a phone call to the master's mobile phone (20).

[0203] Another aspect of the conversation method using the conversational AI-robot friend device is: in step S109, registering and saving personal information of virtual friends through a friend conversation app (48) installed on the master terminal (300); in step S110, having a conversation with one of the virtual friends by the master using the mobile phone (20) to make a call or having one of the virtual friends call the master's mobile phone (20); in step S111, saving the conversation content between the master and the virtual friends in the black box of the conversation black box memory (28b); in step S112, selecting a conversation topic by requesting the creation of a conversation topic from the super-large AI (16) by the interest topic search unit (77). And in step S113, the chat generation means (70b) can have an online conversation between the master (robot user) and virtual friends, including the step of requesting virtual friend dialogue from the super-large AI (16) based on conversation topics selected in step S112 and delivering the generated virtual friend dialogue to the master.

[0204] Additionally, the method for diagnosing and alleviating dementia using a conversational method according to the present invention may include: in step S114, the memory diagnostic unit (80) retrieves past memories and transmits them to the super-large AI (16), and requests the super-large AI (16) to generate questions and answers or hints and answers to test the memory ability of the master from the past memories; in step S115, the master is asked questions using the questions or hints generated in step S114, and the answer is checked to see if it matches the correct answer, thereby scoring the master's memory ability or determining the master's risk of dementia; and in step S116, the master is given answers or some hints regarding past memories that cannot be answered among the questions generated in step S114, thereby strengthening the cognitive ability of the master (robot user).

[0205] In the description above, steps S101 to S116 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0206] A conversational method using an interactive AI-robot friend device according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means. The program instructions may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0207] In addition, the conversation method using the aforementioned conversational AI-robot friend device can also be implemented in the form of an application on a mobile phone or a cloud server.

[0208] In addition, the robot of the aforementioned conversational AI-robot friend device can be replaced or represented by any system such as a chatbot, PC, smart mirror, TV, laptop, or mobile phone that is equipped with artificial intelligence or communicates with an artificial intelligence platform to converse with the owner.

[0209] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0210] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0211] 14: Display Panel 16: Supermassive AI 18: Internet network 20: Mobile phone 23a: Virtual Friend Personal Info List Memory 1 23b: Virtual Friend Personal Info List Memory 2 27: Wireless connection means 27a: Speech-to-text converter 27b: Text-to-speech converter 28a: Wiretap memory 28b: Conversation black box memory 33: Means of Internet Connection 37: Wireless communication connection means 39: Dementia Tracking and Management Department 44: Means of Determining Visibility 46: Means of Judging Loneliness 48: Friend Chat App 53: Control unit 63: Vision Camera 63a: Facial feature authentication means 70: Dialogue Generation Section 70a: Means of creating greetings 70b: Means of creating chats 71: Voice authentication method 73: Means of Memory Generation 74: Memory Ability Scoring Methods 77: Exploration of Topics of Interest 78: Auto-dialing means 80: Memory Diagnosis Section 81a: Speaker section 81b: Microphone section 200: Robot 300: Master's terminal 400: Keyword Identification AI 500: Conversational AI Module 600: Smart Mirror

Claims

Claim 1 delete Claim 2 A conversational AI module integrated within a robot; virtual friends existing within the robot; and a user terminal that is allowed to connect online with the robot; wherein the user terminal includes a friend conversation app capable of registering virtual friends within the robot to perform online conversations with the virtual friends within the robot; and a virtual friend personal information list memory 1 that stores personal information of the registered virtual friends; and wherein the conversational AI module includes a wireless communication connection means for performing an online conversation connection between the user terminal and the virtual friends within the robot; a virtual friend personal information list memory 2 that shares the registration contents of the virtual friend personal information list memory 1; a conversation black box memory for storing online conversation content shared between the virtual friends within the robot and the user terminal; an interest topic search unit that derives conversation topics from past online conversation content read from the conversation black box memory and personal information read from the virtual friend personal information list memory 2; a conversation generation unit that generates dialogue or greetings of virtual friends from conversation topics derived from the interest topic search unit; and a loneliness determination means for determining whether the user is in a lonely state. A conversational AI-robot friend device comprising: an auto-dialing means for randomly selecting one of the virtual friends within the robot as the caller and making a call to the user's terminal when the user is determined to be in a lonely state by the above-mentioned loneliness determination means; wherein the virtual friends within the robot conduct an online conversation with the user's terminal, and the friend conversation app displays the face and name of the caller on the user's terminal so that the user can check who called. Claim 3 delete Claim 4 delete Claim 5 In claim 2, the conversational AI-robot friend device further comprises: a super-large AI; and a memory diagnostic unit; wherein the memory diagnostic unit reads out a portion of past memories stored in the conversational black box memory or eavesdropping memory and transmits it to the super-large AI, and requests the super-large AI to generate and provide feedback on questions and answers or hints and answers to test the user's memory ability, or transmits the user's answer to the questions or hints to the super-large AI and requests the super-large AI to convert the answer into a score and inform the user of how much it matches the answer. Claim 6 In claim 2, the conversational AI-robot friend device further comprises: a super-large AI; and a memory diagnostic unit; wherein the memory diagnostic unit includes: a memory generation means that reads out a portion of past memories stored in the conversational black box memory or eavesdropping memory and transmits it to the super-large AI, and requests feedback by generating a question and an answer or a hint and an answer for a question to test the user's memory ability; and a memory ability scoring means that determines as a score how much the user's answer to the question or hint matches the answer. Claim 7 In claim 6, the above-mentioned memory ability scoring means controls the memory generation means to provide the above-mentioned hint to the user when the score of the above-mentioned answer to the above-mentioned question is below a reference value, and proceeds to evaluate the user's memory recovery ability by determining as a score how close the user's answer to the above-mentioned hint is to the correct answer. Claim 8 In claim 2, the conversational AI module further comprises a dementia tracking management unit that observes the trend of change in analysis results following periodic examinations for dementia and informs the user of the risk of dementia, requests cognitive function reinforcement training, or requests a remote medical diagnosis with a doctor. Claim 9 In claim 2, the conversational AI module further includes a keyword discrimination artificial intelligence learned using sentences labeled according to various keywords, and the keyword discrimination artificial intelligence analyzes text sentences or online conversation content to identify and extract one or more keywords, a conversational AI-robot friend device. Claim 10 delete Claim 11 delete Claim 12 In claim 2, the above-mentioned topic of interest search unit requests the super-large AI to select and provide feedback on the above-mentioned conversation topics from news about incidents, accidents, or various issues on the Internet or social media, in a conversational AI-robot friend device. Claim 13 In claim 2, the conversational AI-robot friend device further comprises a super-large AI; and the conversation generation unit comprises a greeting generation means for generating the greeting; and a chatting generation means for generating the dialogue. Claim 14 In claim 13, the above greeting generation means requests the super-large AI to generate the above greeting and generates the above greeting by receiving feedback therefrom, a conversational AI-robot friend device. Claim 15 In claim 13, the chat generation means is a conversational AI-robot friend device that generates the next line of dialogue of the virtual friend by receiving feedback from the super-large AI by transmitting the conversation topic to the super-large AI to generate the line of dialogue of the virtual friend, or by transmitting the user's dialogue that responded to the line of dialogue of the virtual friend to the super-large AI and receiving feedback from the super-large AI. Claim 16 In claim 2, the conversational AI-robot friend device comprises: a virtual friend generating unit that generates virtual friends, wherein the virtual friend generating unit includes: a face generating unit capable of restoring and reproducing the faces of virtual friends by a speaker ID assigned to each face of virtual friends; a voice generating unit capable of restoring and reproducing the voices of virtual friends by a voice ID assigned to each face of virtual friends; and a virtual face expression generating unit that changes the facial expressions and lip shapes of virtual friends by being synchronized with the voices of virtual friends. Claim 17 In claim 16, the conversational AI-robot friend device comprises a call button means that allows the user to select and call one of the virtual friends they wish to call on the friend conversation app, displays a face corresponding to the speaker ID of the virtual friend being called on the display window of the friend conversation app, and outputs a telephone voice modulated with a tone corresponding to the voice ID of the virtual friend being called. Claim 18 delete Claim 19 In claim 2, the above loneliness determination means determines that the user is in a lonely state when the cumulative amount of conversation with a third party of the user is below a predetermined value, or determines that the user is in a lonely state when the amount of bright facial expression activity of the user is below a predetermined value for a period of time or longer. Claim 20 In claim 2, the conversational AI-robot friend device further comprises a super-large AI; and the conversational AI module receives a dialogue of the virtual friend generated by the super-large AI and transmits it to a friend conversation app on the user's terminal via the wireless communication connection means, or receives a dialogue of the user generated from the user's terminal via the wireless communication connection means and transmits it to the super-large AI while requesting the generation of the next dialogue of the virtual friend, thereby continuing the online conversation between the virtual friends and the user. Claim 21 delete Claim 22 A conversation method of a conversational AI-robot friend device according to claim 2, comprising: a step of registering virtual friends through a friend conversation app installed on a terminal of a robot user; and a step of the robot user selecting one of the virtual friends using a mobile phone to make a call or one of the virtual friends calling the robot user's mobile phone to have a conversation; thereby performing an online conversation between a robot user and virtual friends. Claim 23 A conversational method of an interactive AI-robot friend device that performs dementia diagnosis and alleviation, comprising: a step of retrieving past memories and transmitting them to a super-large AI, and requesting the super-large AI to generate a question and a correct answer or a hint and a correct answer to test the robot user's memory ability from the past memories; a step of asking the robot user the question and checking whether the answer matches the correct answer, thereby scoring the robot user's memory ability or determining the risk of dementia; and a step of providing the robot user with the hint regarding the question that cannot be answered among the questions to strengthen the robot user's memory restoration ability and cognitive ability.

Citation Information

Patent Citations

  • Interaction between mobile robot and user, System forsame

    KR1020060131458A

  • Instant message service providing method using bot

    KR1020140047491A

  • Robot for preventing dementia

    KR1020190010295A

  • system and method for preventing depression of user

    KR1020190048804A

  • System for providing active artificial intelligence image character based on mobile device

    KR1020210040208A