Conversational ai-robot friend device using hyperscale ai, and dementia diagnosis and alleviation method using same

The interactive AI-robot friend device uses super-large AI for conversational diagnosis and cognitive training to address the challenges of dementia identification and management in aging populations, offering effective early detection and alleviation of symptoms.

WO2024228486A1PCT designated stage expired Publication Date: 2024-11-07YOO JAE CHERN +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2024/004841
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-03
Filing Date
2024-04-11
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Current methods for diagnosing and managing dementia are inadequate, particularly in identifying early symptoms and providing effective prevention and alleviation, especially in aging populations where observation and intervention are challenging due to increasing nuclear families and individuals living alone.

Method used

An interactive AI-robot friend device utilizing super-large AI for offline and online conversations, which includes personal authentication, memory diagnosis, and cognitive training, to identify dementia risk factors and provide early diagnosis and alleviation through conversation-based methods, using a combination of voice recognition, memory testing, and cognitive enhancement techniques.

Benefits of technology

The device enables early and accurate diagnosis of dementia, reduces the risk of progression, and alleviates symptoms by providing personalized cognitive training, effectively addressing the challenges of dementia management in aging populations through continuous monitoring and intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024004841_07112024_PF_FP_ABST
    Figure KR2024004841_07112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a conversational AI-robot friend device using hyperscale AI, and a conversation method using same, the device using a robot that can converse, as if a friend, with people, thereby determining whether a user has dementia symptoms through conversations with the user in everyday life. In particular, provided are a conversational hyperscale AI-robot friend device using hyperscale AI, a conversation method using same, and a dementia diagnosis and alleviation method using same, the device using a conversational AI module, which includes a conversation generation unit for generating a topic of conversation to be had with a user and a memory diagnosis unit for asking about past memories, so as to test the memory and cognitive abilities of the user over a period of time in everyday life while performing personal authentication through the voice of the user, thereby making an early diagnosis of dementia and automatically performing intensified training on cognitive functions during conversation.
Need to check novelty before this filing date? Find Prior Art

Description

A conversational AI-robot companion device using super-large AI and a method for diagnosing and alleviating dementia using the same.

[0001] The present invention relates to a conversational AI-robot friend device using a super-large AI that can converse with a user (e.g., a user registered as the owner of the robot) like a friend in offline and online situations, and can also determine whether or not the user has dementia symptoms through conversations with the user in daily life, and a conversation method using the same. In particular, the present invention provides a conversational AI-robot friend device using a super-large AI that not only performs personal authentication through the user's voice and tests the user's memory and cognitive ability in daily life over a predetermined period of time to make an early diagnosis of dementia, but also automatically performs cognitive function enhancement training during conversation, by a conversational AI module including a conversation generation unit that creates a topic for conversation with the user and a memory diagnosis unit that asks about past memories.

[0002] Dementia is the greatest fear for those approaching old age. According to one survey, nearly 40% of people chose dementia as their greatest fear, surpassing cancer (20.09%) as the disease they most fear in old age. Dementia not only causes memory loss but also makes basic human activities impossible, causing significant suffering and conflict within families. Consequently, it is one of the most socially costly diseases. Therefore, dementia should be addressed systematically and technologically, rather than by individuals.

[0003] South Korea has one of the world's fastest-growing populations. Consequently, the country is also experiencing one of the fastest-growing rates of dementia. According to the Korean Society for Biotechnology and Bioengineering, the elderly population increased by 60 percent from 6.13 million in 2013 to 9.84 million in 2023, while the number of dementia patients increased by 77 percent, from 570,000 to 1.01 million, during the same period. Furthermore, by 2022, dementia patients are projected to account for 20.89% of the elderly population aged 80-84, and 38.96% of those aged 85 and older. In short, 59.85% of those aged 80 and older suffer from dementia, meaning one in two people will be living with dementia. As we enter an era of centenarians and a hyper-aging population, early diagnosis and prevention of dementia are urgently needed, not only for the promotion of individual health but also for the health and development of society.

[0004] Fortunately, dementia is not an incurable disease, but rather preventable. Modern medical experts agree that early diagnosis and management, combined with enhanced medication, can significantly improve outcomes. To reduce the prevalence of dementia, early diagnosis and prevention must be prioritized.

[0005] Dementia isn't simply diagnosed when a person has memory loss. Dementia is a complex cognitive impairment that requires impairment in daily life activities, such as making phone calls, using public transportation, or washing, which were previously easy to accomplish independently. Therefore, for this diagnosis, careful observation and information gathering in daily life are crucial, rather than simple hospital visits. However, with the aging population, the trend toward nuclear families, and the rise of single-person households, the number of people who fall through the cracks of this monitoring and care will only increase.

[0006] To improve these problems, we believe that robots will play a significant role in accurately understanding the user's condition, early diagnosis of dementia, and even prevention by increasing brain development and activity while conversing with the robot user in daily life and observing, collecting, and analyzing data on the user's daily life skills and conversation skills.

[0007] (Patent Document 1) Korean Patent Gazette, No. 2059903 (Registered on December 20, 2019)

[0008] The present invention relates to a conversational AI-robot friend device using a super-large AI, which can perform personal authentication without a separate user authentication procedure in daily life by using a robot capable of having offline and online conversations with a user (e.g., a user registered as the owner of the robot) like a friend, and which can determine whether there is dementia symptom and alleviate dementia symptom through offline and online conversations with the user, and a conversation method using the same, in order to solve the problems of the prior art described above. Specifically, the present invention provides a conversational AI-robot friend device using a super-large AI, which can analyze a user's conversation during a conversation by a conversational AI module including an interest topic search unit that searches for a topic of conversation to conduct a conversation with the user, a conversation generation unit that creates a conversation from the topic of conversation, and a memory diagnosis unit that asks about past memories, and can determine the user's dementia risk factors based on the analyzed results and simultaneously perform user authentication, and a conversation method using the same, and a method of diagnosing and alleviating dementia through the same.

[0009] The above-mentioned interest topic search section can find topics or conversations that may interest the user from past conversations, news on the Internet, what the user watches on TV, radio intercepts, and intercepted conversations between the user and other people.

[0010] In addition, the present invention aims to provide a conversational AI-robot friend device using a super-large AI that can slow down or prevent the progression of dementia by providing a memory diagnosis unit that helps restore the user's memory by providing the correct answer or hint to a question that the conversational AI module cannot answer in order to solve the problems of the prior art described above, a conversational AI module, and a conversation method using the same, and a method for diagnosing and alleviating dementia through the same.

[0011] In addition, the present invention aims to provide a conversational AI-robot friend device using a super-large AI and a conversation method using the same, which enables the robot to perform online conversations such as phone conversations, SNS conversations, remote video conversations, and messenger conversations with the user through a wireless communication connection means in addition to offline conversations with the user (e.g., a user registered as the owner of the robot), in order to solve the problems of the prior art described above.

[0012] In addition, the present invention aims to provide a conversational AI-robot friend device using a super-large AI capable of conducting online conversations between pre-registered virtual friends and users (e.g., users registered as owners of robots) and a conversation method using the same, in order to solve the problems of the prior art described above.

[0013] Additionally, in the present invention, the robot may be a chatbot, a personal computer (PC), a laptop, a mobile phone, a TV, or a smart mirror.

[0014] However, the technical tasks to be achieved by the embodiments of the present invention are not limited to the technical tasks described above, and other technical tasks may exist.

[0015] Hereinafter, the person using the robot will be referred to as "User," "Robot User," or "Owner." For example, "Owner" or "Robot User" may be a user of an AI-robot friend device who has been approved in advance through the user registration process provided by the app.

[0016] Additionally, the “owner’s terminal” may be the terminal of the user.

[0017] Additionally, the "Master's lines" may be lines from the user or from a robot user.

[0018] As a technical means for achieving the above-described technical task, a conversational AI-robot friend device using a super-large AI according to one embodiment of the present invention includes a robot and a conversational AI (artificial intelligence) module, and the robot includes a line-of-sight determination means for determining whether the robot is within a line-of-sight distance (a predetermined distance) within which a conversation with the owner is possible; a microphone unit for eavesdropping on sound information (including the owner's voice, a telephone conversation, TV sounds, radio sounds, and conversations between the owner and other people) within the line-of-sight; and a speaker unit for transmitting robot dialogue or greeting dialogue to the owner by voice, wherein the conversational AI module includes a voice authentication means for recognizing the owner's voice from the microphone unit during a conversation and authenticating the owner; a voice-to-text conversion unit for converting sound information eavesdropped through the microphone unit into text sentences or converting the owner's speech input through the microphone unit into the owner's dialogue (user's dialogue) in sentence form; an eavesdropping memory for storing text sentences obtained through the eavesdropping; an eavesdropping memory for storing the eavesdropping content; A dialogue generation unit that generates robot dialogue or greeting dialogue to be delivered to the owner through the speaker unit from conversation topics using artificial intelligence; a text-to-voice conversion unit that converts the robot dialogue or greeting dialogue generated by the dialogue generation unit into voice and delivers the voice to the owner through the speaker unit; a dialogue black box memory for storing the conversation content (including offline and online conversation content) and corresponding keywords exchanged with the owner using the robot dialogue by time zone;And including a memory diagnosis unit that generates questions and corresponding answers from past memories stored in the conversation black box memory or wiretapping memory, and determines the degree to which the owner's answer to the question matches the correct answer with a score, the robot can authenticate the owner through the owner's voice during a conversation with the owner, and at the same time, can continuously test the owner's memory and cognitive ability over a predetermined period of time through the conversational AI module in daily life, thereby enabling early diagnosis of dementia and performing cognitive function enhancement training.

[0019] It is preferred that the lines of the master of the present invention also include lines of other people who are conversing with the master within the range of vision.

[0020] The past memory of the present invention is stored in a conversation black box memory or wiretapping memory, and may include the owner's personal information, past conversation contents, or sound information heard in the past through the microphone.

[0021] The above memory diagnosis unit comprises a memory generation means that provides questions and corresponding answers generated from past memories stored in a conversation black box memory or wiretapping memory; and

[0022] It may include a memory ability scoring means that scores the extent to which the owner's answers to questions generated by the above memory generating means match the correct answers.

[0023] It is preferred that the owner's personal information be stored by default in the wiretapping memory or conversation black box memory of the present invention. The owner's personal information can be stored in the wiretapping memory when registering as a user on the robot using an app, and it is preferred that it include at least one or more pieces of information selected from among memories, past activities and history, hobbies, personal preferences, bucket lists, family information, friendships and history, family event dates (birthdays, wedding anniversaries, etc.), sex, age, height, weight, mobile phone number, email address, SNS connection address, messenger connection address, remote video conference connection address, and mobile phone numbers of friends and family. In addition, the registration and additional management of the owner's personal information can be enabled by an app installed on the owner's terminal (user's terminal).

[0024] The memory generation means of the present invention can transmit past memories stored in a conversation black box memory or wiretapping memory to a super-large AI, and request that the AI ​​generate questions and corresponding answers to test the owner's memory ability.

[0025] In addition, the above memory generation means can enhance the owner's cognitive ability by informing the owner of the correct answer to the question or giving some hints to induce the owner to answer the question if the owner's answer to the generated question is below the standard score.

[0026] Another aspect of the memory ability scoring means is that if the score of the owner's first answer to a question provided to the owner by the memory generation means is below a reference value, the memory ability scoring means controls the memory generation means to provide the owner with an additional hint for the question, and thereafter, the owner's memory restoration ability can be evaluated by judging how close the owner's second answer is to the correct answer as a score.

[0027] The above visibility determination means may be a vision camera to visually determine whether the owner is nearby.

[0028] Another aspect of the above-mentioned visibility determination method may be a voice authentication method that captures the owner's voice from the microphone. If the owner's voice is properly captured and personal authentication is possible, the owner can be determined to be within the visibility range.

[0029] Another aspect of the above-mentioned visibility determination means may be a wireless communication connection means that determines whether a short-range communication connection is possible with the owner's mobile phone. In other words, the owner's presence within a short range can be confirmed using a short-range wireless communication protocol. Bluetooth is preferred for this short-range communication connection.

[0030] The above visibility determination means determines that the owner is within the visibility range when the owner is captured by the vision camera, the owner's voice is authenticated by the voice authentication means, or the owner's cell phone is connected via Bluetooth.

[0031] The above speaker unit may further be equipped with an automatic volume control means to automatically increase the speaker volume when the surrounding noise is loud or the distance from the owner is somewhat far due to a means of determining visibility.

[0032] If abnormalities in memory and cognitive abilities are found by the above memory ability scoring means, the control unit can provide remote medical diagnosis services through an Internet communication connection with a doctor.

[0033] Additionally, the robot is equipped with a display panel, and the control unit can display robot dialogue generated by the dialogue generation unit, or questions and hints generated by the memory generation unit, in text on the display panel. In this case, the owner can check the dialogue and question content through the text on the display panel.

[0034] The above voice authentication method automatically authenticates the owner's identity through voice during a conversation with the robot without a separate authentication procedure, providing the owner with convenience in the authentication procedure.

[0035] In addition, the above-mentioned conversational AI module transmits the owner's lines received through the microphone to the super-large AI on the server via the Internet network, and in response, the super-large AI generates robot lines and feeds them back to the owner through the speaker, so that the conversation with the owner can continue.

[0036] Additionally, the conversational AI module of the present invention can include a super-large AI within the conversational AI module.

[0037] In this case, it provides the advantage of being able to conduct a conversation between the owner and the robot using the interactive AI-robot friend device of the present invention even in an offline situation without a server or internet connection.

[0038] The ultra-large AI of the present invention is a next-generation AI that learns large amounts of data using supercomputing infrastructure with a computational speed much faster than that of a general computer, and typically has more than 150 billion parameters.

[0039] For example, the model 'GPT-3' released in 2020 has 175 billion parameters.

[0040] Additionally, Naver, a Korean portal company, announced 'Hyper Clova', a super-large AI model with 204 billion parameters, at the end of May 2021.

[0041] Additionally, LG AI Research Center unveiled EXAONE, a super-large AI with approximately 300 billion parameters, in December 2021.

[0042] The above conversational AI module transmits the owner's lines received through the microphone to the super-large AI on the server via the Internet, and in response, the super-large AI generates robot lines and feeds them back to the owner through the speaker, so that the conversation with the owner can continue.

[0043] In addition, the control unit may include a dementia tracking management unit that observes the trend of changes in the analysis results according to periodic tests for dementia using the memory ability scoring means and informs the owner of the risk of dementia, requests cognitive function enhancement training, or requests remote medical diagnosis with a doctor.

[0044] The above dementia tracking management unit can determine the owner's dementia risk level by the above memory ability scoring means and display it on the display panel.

[0045] In addition, the above-mentioned conversational AI module can use a wireless communication connection means to enable the owner to call the robot or the robot to call the owner's terminal to conduct a conversation between the two.

[0046] In addition, the above-mentioned conversational AI module can enable SNS conversations (e.g., Facebook), remote video conversations (e.g., WebEx, Zoom Meeting), messenger conversations (e.g., KakaoTalk conversations, group chat rooms), and conversations between the owner and multiple virtual friends in a group chat room through wireless communication connection means.

[0047] The wireless communication connection means of the present invention may be a communication connection via the Internet, wireless mobile communication, Bluetooth or Wi-Fi.

[0048] In addition, the above-mentioned conversational AI module can reduce the loneliness of the owner by conducting an online conversation between the owner and virtual friends created by the virtual friend creation unit when the owner is determined to be lonely by the loneliness judgment means.

[0049] According to one embodiment of the present invention, a conversation method performed by a conversational AI-robot friend device using a super-large AI includes the steps of: selecting a conversation topic by searching for topics that may be of interest to the owner (robot user) from news of incidents and accidents on the Internet, various issues, wiretapped memories, and conversation black box memories; generating a story subject from the super-large AI based on the conversation topics and delivering it to the owner through a speaker; delivering the owner's lines received through a microphone to the super-large AI, and feeding back lines generated by the super-large AI in response to the lines to the owner through the speaker to continue the conversation with the owner; recognizing the owner's voice during the conversation to personally authenticate the owner; conducting a conversation between the owner and the robot by making a call to the robot or the robot making a call to the owner's terminal through a wireless communication connection means; conducting a SNS conversation, remote video conversation, or messenger conversation between the owner and the robot through a wireless communication connection means; And when the owner is judged to be lonely by the loneliness judgment means, it may include a step of conducting an online conversation between the owner and virtual friends created by the virtual friend creation unit.

[0050] In addition, according to one embodiment of the present invention, a method for diagnosing and alleviating dementia using a conversation method performed by a conversational AI-robot friend device using a super-large AI may include a step of scoring the owner's memory ability by asking the owner (robot user) about past memories and checking whether the answers are correct; a step of giving the owner some hints for questions about past memories that the owner cannot answer, thereby enhancing the owner's memory restoration ability and cognitive ability; and a step of determining the owner's dementia risk level using the results analyzed by the memory diagnosis unit.

[0051] Additionally, the robot of the present invention can be replaced by a chatbot, PC, smart mirror, TV, laptop, or mobile phone equipped with a conversational AI module.

[0052] The above-described problem-solving methods are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, additional embodiments may be included in the drawings and detailed description of the invention.

[0053] According to the above-mentioned means of solving the problem of the present invention, a robot capable of conversing with a user (e.g., a user registered as the owner of the robot) like a friend in offline and online situations is used to authenticate the user through conversation with the user in daily life, and at the same time, the user's memory and cognitive ability are tested over a predetermined period of time to diagnose dementia early, and a conversation method using the same is provided, and a method of diagnosing and alleviating dementia using the same is provided.

[0054] In addition, the present invention provides a conversational AI-robot friend device using a super-large AI that can reduce the loneliness of the owner through online conversation between the owner and virtual friends generated by a virtual friend generation unit, and a conversation method using the same.

[0055] However, the effects that can be obtained from this center are not limited to the effects described above, and other effects may exist.

[0056] FIG. 1, FIG. 2a and FIG. 2b are examples of an interactive AI-robot friend device using a super-large AI according to the present invention.

[0057] Figure 3 is an example of storage contents for wiretapping memory and conversation black box memory.

[0058] Figure 4a is an example of a memory diagnosis unit of a conversational AI-robot friend device according to the present invention.

[0059] FIG. 4b is another embodiment of a memory diagnosis unit of a conversational AI-robot friend device according to the present invention.

[0060] Figure 5a shows an example of a friend chat app for registering virtual friends in advance to enable online conversations between the owner and virtual friends virtually existing on the robot, and a virtual friend personal information list memory 1 for storing personal information of the virtual friends.

[0061] Figure 5b shows an example of an online conversation connection between a virtual friend and the owner through a communication connection between the owner's terminal and a wireless communication connection means.

[0062] Figure 6 shows an example of using a robot of an interactive AI-robot friend device using a super-large AI as a smart mirror (600).

[0063] Below, with reference to the attached drawings, embodiments of the present invention are described in detail to facilitate easy implementation by those skilled in the art. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity, and similar reference numerals have been used throughout the specification to indicate similar elements.

[0064] Throughout this specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" or "indirectly connected" with another element in between.

[0065] Throughout this specification, when it is said that a member is located “on,” “above,” “upper,” “lower,” “lower” or “lower” another member, this includes not only cases where the member is in contact with the other member, but also cases where another member exists between the two members.

[0066] Throughout this specification, whenever a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0067] Hereinafter, a preferred embodiment of the present invention will be described in detail using the attached drawings.

[0068] Hereinafter, the robot in the present invention can be used interchangeably with an AI-robot friend device.

[0069] Hereinafter, the virtual friend in the present invention includes a robot.

[0070] Hereinafter, the owner, robot user or user in the present invention may be interchangeably used with the user of the interactive AI-robot friend device.

[0071] Hereinafter, in the present invention, the dialogue content generated by the super-large AI and delivered to the owner without modification or modification is referred to as robot dialogue.

[0072] Hereinafter, in the present invention, the robot dialogue that takes place online between a virtual friend and the owner is referred to as a virtual friend dialogue.

[0073] Hereinafter, in the present invention, past memory refers to the contents of the owner's past conversations stored in the conversation black box memory or wiretapping memory, the owner's personal information, and information on past sounds heard by the owner.

[0074] Hereinafter, in the present invention, memory refers to a conversation black box memory or a wiretapping memory.

[0075] FIG. 1, FIG. 2a and FIG. 2b are examples of an interactive AI-robot friend device using a super-large AI according to the present invention, wherein the interactive AI-robot friend device includes a robot (200) and an interactive AI module (500) integrated within the body of the robot (200).

[0076] The control unit (53) of the above-described conversational AI module (500) includes a visibility determination means (44) for determining whether the owner is within a visible distance or visible range where a conversation with the owner is possible. In the present invention, the visibility determination means (44) may be a facial shape authentication means (63a) for determining whether the owner's appearance is captured in an image input from a vision camera (63), a voice authentication means (71) for determining whether the owner's voice is captured above a predetermined volume, or a wireless communication connection means (37) for determining whether the owner's wearable device (not shown) or mobile phone (20) is properly connected to the communication via Bluetooth by setting a predetermined frequency cutoff for the determination.

[0077] For example, when the frequency cutoff is set to 10 times, the visibility determination means (44) can determine that the owner is not within the visibility range when the owner's appearance is not captured by the facial shape authentication means (63a) more than 10 times per minute, the owner's voice recognized by the voice authentication means (71) is not captured above a predetermined level more than 10 times per minute, or the Bluetooth connection with the owner's cell phone (20) is not established more than 10 times per minute.

[0078] The above wearable device (not shown) may be any one selected from a smart watch, an electronic ring, an electronic necklace, and an electronic bracelet.

[0079] The above facial shape authentication means (63a) can be achieved by a facial shape recognition artificial intelligence neural network that is pre-learned by images of the owner's face shape from various angles marked as the owner, and then recognizes the owner's face shape obtained by a vision camera (63).

[0080] Since the vision camera (63) cannot always see the owner's face from the front, it must be able to recognize the owner's face captured from various angles other than the front. Accordingly, the facial shape authentication means (63a) can also recognize the owner's face captured from various angles other than the front.

[0081] In the present invention, the facial shape recognition artificial intelligence neural network is preferably optimized by supervised learning using images of the owner's face from various angles (e.g., front, back, and side images) obtained by a vision camera (63) during the process of registering the owner's personal information, and then by semi-supervised learning based on the supervised learning facial shape recognition artificial intelligence neural network after the personal information is registered.

[0082] The above voice authentication means (71) may be equipped with a volume check means that can check whether the owner's voice is above a predetermined volume.

[0083] A robot (200) comprises a microphone unit (81b) for eavesdropping on external sounds or conversation sounds within its field of vision; and a speaker unit (81a) for transmitting robot dialogue or greeting dialogue to the owner, wherein the conversational AI module (500) comprises: a voice authentication means (71) for recognizing the owner's voice input from the microphone unit (81b) to determine whether the owner is the robot; an eavesdropping memory (28a) for converting the wiretapped content input from the microphone unit (81b) within its field of vision into sentence form by a voice-to-text conversion unit (27a) and storing the converted content; a dialogue generation unit (70) for generating robot dialogue and greeting dialogue to be transmitted to the owner by artificial intelligence; a text-to-voice conversion unit (27b) for converting the robot dialogue or greeting dialogue generated by the dialogue generation unit (70) into voice and transmitting the voice to the owner through the speaker unit (81a); A conversation black box memory (28b) for storing the conversation with the owner using the robot dialogue generated by the above-mentioned conversation generation unit (70); a subject matter determination artificial intelligence (400) for classifying the subject matter of a sentence; and a memory diagnosis unit (80) for extracting and generating questions and correct answers to the questions from past memories stored in the above-mentioned conversation black box memory (28b) or wiretapping memory (28a), and judging the degree to which the owner's answer to the generated question matches the correct answer with a score, so that the robot (200) can authenticate the owner's voice while the owner is having a conversation, and at the same time, the owner's memory and cognitive ability can be continuously tested over a predetermined period of time by the above-mentioned conversational AI module (500) in daily life, so that not only can dementia be diagnosed early, but also cognitive function strengthening training can be performed.

[0084] The above voice authentication means (71) provides convenience to the owner in terms of authentication procedures by authenticating the owner's identity through voice during a conversation with the robot (200) without a separate authentication procedure.

[0085] In addition, the contents stored in the above conversation black box memory (28b) or wiretapping memory (28a) can be used to test the ability of the owner to remember the past by using the contents of the owner's past conversations.

[0086] The wiretapping memory (28a) and the conversation black box memory (28b) of the present invention preferably use a storage means including NAND flash or DRAM, HDD (Hard Disc Drive), but it is more preferred to use a semiconductor memory.

[0087] FIG. 2a is an embodiment of a conversational AI module (500) of a conversational AI-robot friend device for online or offline conversation according to the present invention.

[0088] In the present invention, offline conversation refers to a conversation between the owner and the robot (200) through the microphone section (81b) and the speaker section (81a), and online conversation refers to a conversation between the owner's terminal (300) and the robot (200).

[0089] The above-mentioned owner terminal (300) includes a device capable of text, voice, and video calls through a wireless communication connection, such as a mobile device, PC, smart watch, smart glasses, or XR headset device.

[0090] The artificial intelligence (400) for determining the subject of the present invention can be implemented by an artificial intelligence neural network trained using sentences labeled according to various subject terms, and can analyze given sentences or online conversation contents to determine and extract one or more subject terms.

[0091] For example, the wiretapping memory (28a) of the present invention can store text sentences and corresponding keywords obtained by the voice-to-text conversion unit (27a) that converts sound information wiretapped through the microphone unit (81b) into sentence form. At this time, the keyword determination artificial intelligence (400) can analyze the text sentences to find the corresponding keywords, and the wiretapping memory (28a) stores the text sentences and corresponding keywords together.

[0092] Additionally, the conversation black box memory (28b) of the present invention can store conversation sentences and corresponding keywords together.

[0093] The above sound information may be wiretapped content including the owner's voice speaking to another person or sounds heard by the owner (e.g. TV sound, radio sound).

[0094] The topic of interest search unit (77) selects conversation topics that include keywords that the owner was statistically interested in for a certain period of time from the contents stored in the memory (28a, 28b).

[0095] Since the corresponding subject words are already allocated and stored for each sentence in the above memory (28a, 28b), the interest subject search unit (77) can statistically determine which subject words the owner is more interested in by calculating the number of identical subject words stored in the memory (28a, 28b) over a given period of time.

[0096] Another aspect of the above-mentioned topic search unit (77) is that it can select topics of conversation from news of incidents and accidents and various issues on the Internet and YouTube using a super-large AI (16).

[0097] To this end, the topic of interest exploration unit (77) can request the super-large AI (16) to select and provide feedback on topics of conversation that are of recent interest and issues from news of accidents and incidents on the Internet and YouTube, as well as various issues.

[0098] The above keywords may include personality, appearance, cosmetics, beauty, fortune telling, weather, fine dust, natural disasters, insurance, real estate, exchange rates, loans, apartments, property, stock investment, banks, economy, prices, taxes, virtual currency, international issues, housework (cooking, cleaning, laundry), fashion (clothing, hair), youth issues, career issues, family issues, childcare, retirement life, friends, marriage, divorce, holidays, vacations, living expenses, remodeling, appliance breakdown and A / S, hobbies, religion, business, history, geography, science and technology, information and communication, law, military, war, oil, space, nature, biology, accidents and incidents, education, studying abroad, immigration, college admissions, employment, medical care, health, illness, death, drinking, smoking, drugs, medicine (nutritional supplements), shopping, eating out, transportation, cars, traffic accidents, sports, indoor exercise, internet, movies, art, dramas, novels, music, songs, games, lovers, food, pets, tourism, political issues, population issues, social issues, environmental issues, and related words.

[0099] For example, words related to the subject word accident could be car accident, fire, plane crash, ship sinking, crime, etc., and words related to car accident could be car theft, murder, violence, theft, etc.

[0100] The conversation generation unit (70) of the present invention is for generating greeting lines and robot lines for conducting a conversation between the owner and the robot (200), and may include a greeting generation unit (70a) that provides everyday greeting lines within the visible range, and a chat generation unit (70b) that generates robot lines for conversation topics extracted by the topic of interest search unit (77).

[0101] In the present invention, it is preferred that the greeting generation means (70a) generates a greeting line such as a greeting and transmits it to the owner through the speaker unit (81a) after a predetermined period of time (e.g., 5 hours or more) has passed since the robot (200) has been in contact with the owner within the visible range and has simultaneously crossed the international date line.

[0102] For example, the greeting generation means (70a) can generate a greeting line such as "Hello, master. It's cold today." or "Hello. Did you sleep well? Master. It's cold today. Did you have breakfast?" The generated greeting line can be delivered to the master through the speaker unit (81a) via the text-to-speech conversion unit (27b).

[0103] Another aspect of the above greeting generation means (70a) is that it can request the super-large AI (16) to generate a greeting line. Thereafter, the greeting generation means (70a) can receive feedback on the greeting line generated by the super-large AI (16) and transmit it to the owner through the text-to-speech conversion unit (27b) and the speaker unit (81a).

[0104] For example, the greeting generation means (70a) requests the super-large AI (16) to say a greeting line as follows, and the super-large AI (16) can receive feedback on the generated greeting and deliver it to the owner.

[0105] Example of a greeting generation means (70a) requesting a greeting line from a super-large AI (16):

[0106] Try to make a morning greeting sentence for me while thinking me as your friend.

[0107] Example of a greeting generated by a super-large AI (16):

[0108] Good morning, friend! I hope you have a great start to your day today!

[0109] The above chat generation means (70b) can transmit a conversation topic to the super-large AI (16) through the wireless communication connection means (37), and at the same time, request the super-large AI (16) to generate a first robot dialogue using the conversation topic. At this time, the chat generation means (70b) can receive feedback on the first robot dialogue generated by the super-large AI (16) through the wireless communication connection means (37) and transmit it to the owner through the text-to-voice conversion unit (27b) and the speaker unit (81a). Thereafter, the chat generation means (70b) catches the owner's dialogue for the first robot dialogue from the microphone unit (81b) and transmits it to the super-large AI (16) through the wireless communication connection means (37). At this time, the chat generation means (70b) can continue the conversation by receiving feedback on the second robot dialogue from the super-large AI (16) and relaying it back to the owner.

[0110] The above master's dialogue can be obtained by a voice-to-text conversion unit (27a) that converts the master's speech input through the microphone unit (81b) into sentences.

[0111] For example, after the greeting line generated by the greeting generation means (70a) is delivered to the owner, the chat generation means (70b) collaborates with the super-large AI (16) to generate the first robot line on the conversation topic and deliver it to the owner, and the owner's response to it is delivered again to the super-large AI (16), so that the robot (200) can carry out a conversation with the owner.

[0112] At this time, the control unit (53) controls the voice-to-text conversion unit (27a) and the wireless communication connection means (37) so that the owner's dialogue input through the microphone unit (81b) is transmitted to the super-large AI (16) on the server (15) through the Internet network (18), and in response, the robot dialogue generated by the super-large AI (16) is fed back to the owner through the speaker unit (81a) so that the chat generation means (70b) and the text-to-voice conversion unit (27b) are controlled.

[0113] The memory diagnostic unit (80) can generate questions and answers to the questions from past memories stored in the memory (28a, 28b) to test the owner's memory ability, and can determine the degree to which the owner's answer to the question matches the answer by scoring.

[0114] The above memory diagnostic unit (80) reads out some of the past memories stored in the memory (28a, 28b) and transmits them to the super-large AI (16) to generate questions about past memories and answers to the questions, and can request the super-large AI (16) to generate questions and answers to test the owner's memory ability and provide feedback.

[0115] In addition, the memory diagnosis unit (80) can retrieve some of the past memories stored in the memory (28a, 28b) and transmit them to the super AI (16), and request the super AI (16) to generate hints and answers to test the owner's memory ability and provide feedback.

[0116] It is preferred that some of the above-mentioned past memories be sentences stored in a randomly selected, predetermined past time period from memory (28a, 28b). The further back in time the randomly selected sentences are from the past, the higher the score can be assigned when evaluating memory ability.

[0117] In addition, the robot (200) of the interactive AI-robot friend device of the present invention is equipped with a display panel (14) and can display robot dialogue generated by the dialogue generation unit (70) or questions and hints generated by the memory diagnosis unit (80) in text form. In this case, the owner can check these text contents on the display panel (14).

[0118] In addition, the conversational AI module (500) may include a dementia tracking management unit (39) that observes the trend of changes in the analysis results according to periodic tests for dementia by the memory diagnosis unit (80) and informs the owner of the risk of dementia, requests cognitive function enhancement training, or requests remote medical diagnosis with a doctor.

[0119] The above dementia tracking management department (39) can determine the owner's dementia risk level based on the above memory ability evaluation score.

[0120] Additionally, the conversational AI module (500) of the robot (200) can conduct an online conversation between the owner and the robot (200) through a wireless communication connection means (37).

[0121] The online conversation of the present invention may include a telephone conversation, a SNS conversation (e.g., Facebook), a remote video conversation (e.g., WebEx, Zoom meeting, video call), a messenger conversation (e.g., KakaoTalk conversation, text conversation, email conversation, group chat room conversation), or a conversation between the owner and multiple virtual friends in a group chat room.

[0122] For example, the robot (200) can receive a call from the owner's cell phone (20) or the robot (200) can call the owner's cell phone (20) and conduct an online phone conversation with the owner. At this time, the conversational AI module (500) of the robot (200) can transmit the robot dialogue generated by the super-large AI (16) to the owner's cell phone (20) through the wireless communication connection means (37), and the owner's voice can be received through the wireless communication connection means (37) and transmitted to the super-large AI (16) to conduct a phone conversation between the robot (200) and the owner.

[0123] The online conversation sentences and corresponding keywords between the above owner and the robot (200) are stored together in the conversation black box memory (28b) and can be used to evaluate the owner's memory ability for past memories.

[0124] FIG. 2b is another embodiment of a conversational AI module (500) of a conversational AI-robot friend device according to the present invention, which includes a super-large AI (16) within the conversational AI module (500).

[0125] In this case, since the conversational AI module (500) is included in the super-large AI (16), it provides the advantage of minimizing the delay time due to the Internet communication connection required to obtain robot dialogue from the super-large AI (16).

[0126] For example, the chat generation means (70b) can directly transmit the conversation topic determined by the topic of interest search unit (77) to the super-large AI (16), and at the same time, request the super-large AI (16) to generate a first robot dialogue based on the conversation topic and provide feedback. At this time, the chat generation means (70b) can receive the first robot's dialogue generated by the super-large AI (16) and transmit it to the owner through the text-to-voice conversion unit (27b) and the speaker unit (81a). Thereafter, the chat generation means (70b) receives the owner's dialogue for the first robot's dialogue through the microphone unit (81b) and the voice-to-text conversion unit (27a) and transmits it to the super-large AI (16), and at this time, the super-large AI (16) generates a second robot dialogue.

[0127] In addition, the memory diagnostic unit (80) can request the super AI (16) to generate questions and answers to test the owner's memory ability by reading out some of the past memories stored in the memory (28a, 28b) and directly transmitting them to the super AI (16) to generate questions and answers to the questions about the past memories.

[0128] Figure 3 (a) is an example of storage contents for a wiretapping memory (28a), and Figure 3 (b) is an example of storage contents for a conversation black box memory (28b).

[0129] In each memory (28a, 28b), a subject and a storage time can be stored together with the sentence content (conversation content). At this time, one or more subject words can be selected for each sentence.

[0130] The subject words stored in the wiretapping memory (28a) can be selected by the subject word determination artificial intelligence (400) that finds the subject words for the text sentence content.

[0131] Additionally, the subject matter stored in the conversation black box memory (28b) may be a conversation topic selected by the topic of interest search unit (77).

[0132] For example, audio information eavesdropped through a microphone unit (81b) is converted into text sentences by a voice-to-text conversion unit (27a), and a keyword determination artificial intelligence (400) classifies the keywords of the text sentences. At this time, the text sentences, keywords, and storage time (current time) can be stored together in the eavesdropping memory (28a).

[0133] It is preferred that the owner's personal information be stored as default in the wiretapping memory (28a) or conversation black box memory (28b) of the present invention.

[0134] Fig. 4a is an example of a memory diagnosis unit (80) of a conversational AI-robot friend device according to the present invention, which can evaluate the owner's memory ability through cooperation with a super-large AI (16).

[0135] For example, the memory diagnosis unit (80) can read out past memories stored in the memories (28a, 28b) and transmit them to the super-large AI (16), and request the super-large AI (16) to generate questions and corresponding answers to test the owner's memory ability from the past memories. In addition, the memory diagnosis unit (80) can receive the questions generated by the super-large AI (16), transmit the content of the questions to the owner through the text-to-speech conversion unit (27b) and the speaker unit (81a), and wait for the owner's answer. The memory diagnosis unit (80) can capture the owner's answers to the questions from the microphone unit (81b), and transmit these answers to the super-large AI (16) through the voice-to-text conversion unit (27a) and the wireless communication connection means (37), and request the super-large AI (16) to convert the scores to indicate how much the owner's answers match the correct answers.

[0136] In addition, the memory diagnosis unit (80) can request the giant AI (16) to generate a hint and a corresponding correct answer for testing the owner's memory ability from the past memories while transmitting the past memories stored in the memories (28a, 28b). The memory diagnosis unit (80) can receive the hint generated by the giant AI (16), transmit the hint content to the owner through the text-to-speech conversion unit (27b) and the speaker unit (81a), and wait for the owner's response. The memory diagnosis unit (80) can capture the owner's response to the hint from the microphone unit (81b), and transmit these responses to the giant AI (16) through the voice-to-text conversion unit (27a) and the wireless communication connection means (37), and request the giant AI (16) to convert the score into a score and inform the owner of how much the owner's response matches the correct answer. In this case, the higher the score, the better the owner's memory restoration ability is judged to be.

[0137] Preferably, if the score of the owner's first answer to the question generated by the super-large AI (16) is below a reference value, the memory diagnosis unit (80) provides the owner with a hint again and waits for the owner's answer. Thereafter, the memory diagnosis unit (80) transmits the owner's second answer to the hint to the super-large AI (16) and requests the super-large AI (16) to convert the score to indicate how closely the second answer matches the correct answer, thereby evaluating the owner's memory restoration ability.

[0138] FIG. 4b is another embodiment of a memory diagnosis unit (80) of an interactive AI-robot friend device according to the present invention. The memory diagnosis unit (80) is composed of a memory generation means (73) and a memory ability scoring means (74) to evaluate the owner's memory and cognitive abilities.

[0139] The memory generation means (73) transmits a portion of the past memories stored in the memory (28a, 28b) to the super-large AI (16), and requests the super-large AI (16) to generate questions and corresponding answers for testing the owner's memory ability from the past memories, thereby receiving feedback from the super-large AI (16) on the questions and corresponding answers for testing the owner's memory ability.

[0140] At this time, the memory generation means (73) can transmit the above question to the owner through the text-to-voice conversion unit (27b) and the speaker unit (81a).

[0141] In addition, the memory generation means (73) transmits some of the past memories stored in the memory (28a, 28b) to the super-large AI (16), and requests the super-large AI (16) to generate a hint and the corresponding correct answer for testing the owner's memory ability, so that the hint and the corresponding correct answer for testing the owner's memory ability can be fed back from the super-large AI (16).

[0142] At this time, the memory generation means (73) can convey the above hint to the owner through the text-to-voice conversion unit (27b) and the speaker unit (81a).

[0143] The memory generation means (73) can share the correct answer provided by the super-large AI (16) with the memory ability scoring means (74). At this time, the memory ability scoring means (74) can receive the owner's answer, calculate how close the answer is to the correct answer by converting it into a score, and share the score with the memory generation means (73).

[0144] For example, the memory ability scoring means (74) captures the owner's first answer to a question generated by the super-large AI (16) through the microphone unit (81b) and the voice-to-text conversion unit (27a), calculates how close it is to the correct answer by converting it into a score, and if the score is below the standard, the memory generation means (73) provides the owner with the hint through the text-to-voice conversion unit (27b) and the speaker unit (81a), and thereafter, the memory ability scoring means (74) captures the owner's second answer through the microphone unit (81b) and the voice-to-text conversion unit (27a), compares it with the correct answer, and determines how close the owner's second answer is to the correct answer as a score, thereby evaluating the owner's memory restoration ability.

[0145] FIG. 5a shows an example of a friend chat app (48) for registering virtual friends in advance to conduct online conversations between the owner and virtual friends virtually existing on the robot (200), and a virtual friend personal information list memory 1 (23a) for storing personal information of registered virtual friends.

[0146] In the present invention, the virtual friend includes a robot. Therefore, the online conversation of the present invention can be applied not only to virtual friends but also to robots.

[0147] Registration of the above virtual friends can be done through the friend chat app (48) installed on the owner's terminal (300) and the virtual friend personal information list memory 1 (23a).

[0148] At this time, you can select and register desired virtual friends on the friend chat app (48) or change the corresponding options or personal information of virtual friends.

[0149] In the same way, the corresponding options or personal information of the robot (200) can be changed.

[0150] The above owner's terminal (300) may be any one electronic device selected from among a mobile device, a PC, a smart watch, smart glasses, and an XR headset device.

[0151] It is preferred that the contents of the virtual friend personal information list memory 1 (23a) on the above master terminal (300) be shared by the virtual friend personal information list memory 2 (23b) on the conversational AI module (500).

[0152] To this end, the contents of the virtual friend profile list memory 1 (23a) are received by the robot's wireless communication connection means (37) and copied to the virtual friend profile list memory 2 (23b) on the conversational AI module (500).

[0153] It is preferred that the above personal information include at least one of the following: speaker ID, voice ID, language (nationality), name (nickname), age, gender, religion, residence, place of birth, occupation, education, hobbies, personal preferences, SNS address, and email address.

[0154] For example, if the owner attempts to talk to a virtual friend whose nationality (language) is set to English in his / her personal information, the owner will be able to talk to the virtual friend in English, allowing him / her to study English.

[0155] The friend conversation app (48) of the present invention includes a virtual friend creation unit (not shown) that creates virtual friends, and the virtual friend creation unit (not shown) may include a face creation unit that can restore and reproduce the face of a virtual friend by a speaker identification assigned to each virtual friend; a voice creation unit that can restore and reproduce the voice by a voice ID assigned to each virtual friend; and a virtual facial expression creation unit that changes the facial expression and lip shape of the virtual friend in synchronization with the voice of the virtual friend.

[0156] The above face generation unit and virtual facial expression generation unit can be implemented by artificial intelligence.

[0157] The face generation unit allows the owner to select the face of the virtual friend he or she wants on the friend chat app (48) during virtual friend registration, and each virtual friend's face is assigned a speaker identification, so that later the friend chat app (48) can restore and reproduce the face of the virtual friend by the speaker ID.

[0158] That is, the face generation unit not only provides various options for the virtual friend's facial appearance in a display window during virtual friend registration, but can also restore and reproduce the virtual friend's face later by speaker ID.

[0159] In addition, the voice generation unit can select the voice or tone desired by the owner on the friend conversation app (48) during virtual friend registration, and can restore and reproduce the voice by voice ID and output it.

[0160] That is, the voice generation unit not only provides various options for the voice of the virtual friend on the friend chat app (48), but can also restore and reproduce the voice later by voice ID.

[0161] In the present invention, it is preferred that the initial values ​​of the virtual friends and their personal information on the virtual friend personal information list memory 1 (23a) be automatically generated randomly to 10 or fewer people according to the owner's taste as shown in the owner's personal information, and if a change is desired, it is possible to change, add, and delete them using the friend chat app (48).

[0162] For example, the owner can use the friend chat app (48) on the mobile phone (20) to find and register desired virtual friends or change the personal information of virtual friends.

[0163] In addition, the owner can add or modify virtual friends on the friend chat app (48) using the owner's mobile phone (20), and can select one of the virtual friends with whom he or she wishes to make a phone call by pressing the call (20a) button. In this case, the owner can see the face corresponding to the virtual friend's speaker ID on the display window of the friend chat app (48) on the mobile phone (20), and can hear the virtual friend's phone voice modulated into a tone corresponding to the voice ID.

[0164] In addition, it is preferred that the virtual friend profile list memory contents registered or modified by the friend chat app (48) be shared by the remaining owner terminals (300) (electronic devices including mobile devices, PCs, smart watches, smart glasses, and XR headsets).

[0165] In addition, during a video call between the owner and the robot, the robot (200) can transmit a photo of its own face or a video of the scene taken by the image sensor (21) of the robot hand to the owner's terminal (300) through a wireless communication connection means (37).

[0166] FIG. 5b shows an example of a conversational AI module (500) for conducting an online conversation between the owner and one virtual friend selected from among a plurality of virtual friends.

[0167] In this example, one of your virtual friends can initiate an online conversation with you by requesting an online conversation with you.

[0168] Additionally, in this example, the owner can select one of the virtual friends and initiate an online conversation by requesting a conversation.

[0169] Additionally, in this example, an online conversation can be conducted between the owner and the robot (200).

[0170] The above online conversation may include a telephone conversation, a social media conversation, a remote video conversation, or a messenger conversation.

[0171] In the present invention, in order to individually store and manage online conversations between the owner and each virtual friend, the conversation black box memory (28b) may be configured with multiple black boxes. That is, the online conversations between the owner and the virtual friend and the corresponding keywords may be stored in black boxes independently assigned to each virtual friend.

[0172] For example, black box 1 (40a) is assigned to virtual friend 1, black box 2 (40b) is assigned to virtual friend 2, and black box 3 (40c) is assigned to virtual friend 3.

[0173] Additionally, separately from this, black box 4 (40d) can be allocated for online and offline conversations between the robot (200) and the owner.

[0174] Accordingly, the content of online conversations between the owner and virtual friends is stored in a black box (40a, 40b, 40c) that exists individually for each virtual friend, so that not only privacy is maintained between virtual friends, but it can also be used as past memories that can be used to continue the conversation in the next online conversation.

[0175] Additionally, in this example, the control unit (53) of the interactive AI-robot friend device includes a loneliness determination means (46) for determining whether the owner is lonely. The loneliness determination means (46) can determine that the owner is lonely if the accumulated conversation volume of the owner with a third party is below a predetermined value through a voice authentication means (71) that performs authentication of the owner's voice.

[0176] In this case, the control unit (53) can select one of the virtual friends and control it to make a call to the owner's terminal (300) through the auto-dialing means (78).

[0177] Another aspect of the above loneliness judgment means (46) is that if the owner's bright facial expression activity level is below a predetermined level for a predetermined period of time, the owner can be judged to be lonely by the facial expression recognition means.

[0178] It is preferred that the above bright facial expression activity be calculated as the ratio of bright facial expressions during a given period.

[0179] The above facial expression recognition means can be achieved by an artificial intelligence neural network that is pre-trained by an image dataset labeled with various facial expressions and then recognizes the owner's facial expression obtained by a vision camera (63).

[0180] The image dataset labeled with the above-mentioned various facial expressions is preferably a facial image database labeled by dividing the grades into sad expression, neutral expression, smiling expression, big smiling expression, and happy expression, and in this case, the amount of bright facial expression activity can be calculated using mathematical expression 1.

[0181]

[0182] Figure 5b shows an example of an online conversation connection between a virtual friend and the owner through a communication connection between the owner's terminal (300) and a wireless communication connection means (37).

[0183] In the case of online conversation using voice, the conversational AI module (500) receives virtual friend dialogue generated by the super-large AI (16) through the Internet connection means (33), and transmits it as voice to the friend conversation app (48) on the owner's terminal (300) via the dialogue generation unit (70), the text-to-speech conversion unit (27b), and the wireless communication connection means (37). In addition, the conversational AI module (500) receives the owner's voice from the owner's terminal (300) through the wireless communication connection means (37), and transmits it to the super-large AI (16) via the voice-to-text conversion unit (27a), the dialogue generation unit (70), and the Internet connection means (33), thereby requesting the generation of the next virtual friend dialogue, thereby enabling an online voice conversation between the virtual friend and the owner.

[0184] In the same way, a voice conversation between a robot (200) and its owner can be conducted online.

[0185] On the other hand, in the case of online conversation using text, the conversational AI module (500) receives the virtual friend dialogue generated by the super-large AI (16) through the Internet connection means (33) and transmits it to the friend conversation app (48) residing on the owner's terminal (300) through the wireless communication connection means (37). Thereafter, the owner's text generated from the owner's terminal (300) is received through the wireless communication connection means (37) and transmitted to the super-large AI (16) through the Internet connection means (33) while requesting the generation of the next virtual friend dialogue, thereby enabling a text conversation between the virtual friend and the owner.

[0186] In the same way, text conversations between a robot (200) and its owner can be conducted online.

[0187] Another aspect of the conversational AI module (500) following FIG. 5b is that the owner can select one of their virtual friends and make a call using the friend chat app (48) on the owner's terminal (300). In this case, the corresponding speaker ID information of the selected virtual friend is transmitted to the conversational AI module (500).

[0188] The speaker selection unit (29) receives the corresponding speaker ID of the virtual friend selected by the owner's terminal (300) through the wireless communication connection means (37) or receives the corresponding speaker ID of the virtual friend randomly selected by the loneliness determination means (46).

[0189] Past online conversation content retrieved from the black box corresponding to the speaker ID can be transmitted to the topic of interest exploration unit (77) that derives conversation topics.

[0190] Additionally, personal information read from the virtual friend personal information list memory 2 (23b) corresponding to the speaker ID can be transmitted to the topic of interest search unit (77) that derives conversation topics.

[0191] For example, when the owner selects one of the virtual friends and makes a call using the friend chat app (48) on the owner's terminal (300), the speaker selection unit (29) extracts the speaker ID from the reception signal of the wireless communication connection means (37) to determine which of the virtual friends the owner made the call to, and the control unit (53) can read out the personal information of the virtual friend corresponding to the speaker ID from the virtual friend personal information list memory 2 (23b) and transmit it to the interest topic search unit (77).

[0192] In addition, the control unit (53) can read out the contents of the black box corresponding to the speaker ID and transmit it to the topic of interest search unit (77) to derive a conversation topic from the past online conversation between the virtual friend and the owner.

[0193] At this time, the interest topic search unit (77) derives conversation topics from the contents of the black box and personal information read out and transmitted, and the chat generation means (70b) of the conversation generation unit (70) generates virtual friend dialogue with the help of the super-large AI (16) using the derived conversation topics, and transmits it to the friend conversation app (48) on the owner's terminal (300) via the text-to-voice conversion unit (27b) and the wireless communication connection means (37).

[0194] In addition, the conversational AI module (500) receives the owner's voice from the owner's terminal (300) through a wireless communication connection means (37), and transmits it to the super-large AI (16) via a voice-to-text conversion unit (27a), a chat generation means (70b), and an internet connection means (33), and then the chat generation means (70b) receives feedback on the next virtual friend's dialogue from the super-large AI, thereby enabling the phone conversation between the virtual friend and the owner to continue.

[0195] Another aspect of the conversational AI module (500) following Fig. 5b is that the control unit (53) is equipped with a loneliness determination means (46) that determines whether the owner is lonely. If the owner is determined to be lonely, the loneliness determination means (46) can select one of the virtual friends as the caller and call the owner. In this case, the owner's loneliness can be reduced.

[0196] The above caller can be identified by a speaker ID uniquely assigned to each virtual friend.

[0197] For example, the loneliness judgment means (46) can make a call to the owner's terminal (300) using the auto-dialing means (78) using the speaker ID of a randomly selected virtual friend. In this case, the friend conversation app (48) on the owner's terminal (300) displays the face and name of the virtual friend corresponding to the speaker ID on the owner's terminal (300), so that the owner can confirm who the call came from.

[0198] In addition, the control unit (53) can read out the personal information of the corresponding virtual friend from the virtual friend personal information list memory 2 (23b) corresponding to the speaker ID provided by the speaker selection unit (29), read out the contents of the black box corresponding to the speaker ID, and transmit it to the topic of interest search unit (77).

[0199] At this time, the interest topic search unit (77) derives conversation topics from the contents of the extracted black box and personal information, and the chat generation means (70b) of the conversation generation unit (70) generates virtual friend dialogue with the derived conversation topics with the help of the super-large AI (16), and transmits it to the friend conversation app (48) on the owner's terminal (300) via the text-to-voice conversion unit (27b) and the wireless communication connection means (37), so that a phone conversation can be conducted between the virtual friend and the owner.

[0200] FIG. 6 is an example of a robot of an interactive AI-robot friend device using a super-large AI as a smart mirror (600). The interactive AI-robot friend device includes a smart mirror (600) and an interactive AI module (500) integrated into the body of the smart mirror (600).

[0201] At this time, the owner can converse while looking at the face of a virtual friend or robot (60) appearing on the smart mirror (600).

[0202] The vision camera (63) captures and recognizes the owner's facial shape, the microphone (81b) eavesdrops on external sounds or conversations within the field of vision, and the speaker unit (81a) transmits the robot's lines to the owner as voice.

[0203] The above-described conversational AI module (500) may include a control unit (53), a facial shape authentication means (63a), a voice authentication means (71), a wiretapping memory (28a), a dialogue generation unit (70), a topic of interest determination unit (77), a dialogue black box memory (28b), a keyword determination artificial intelligence (400), and a memory diagnosis unit (80) as exemplified in the present invention.

[0204] The smart mirror (600) can display robot dialogue generated by the dialogue generation unit (70) or questions and hints generated by the memory diagnosis unit (80) in text form. In this case, the owner can check these text contents on the smart mirror (600).

[0205] Additionally, the owner's dementia risk level can be displayed on the smart mirror (600).

[0206] As can be seen from this embodiment, the robot in the present invention can be replaced by a chatbot, a personal computer (PC), a laptop, a mobile phone, a smart mirror, or a television.

[0207] Meanwhile, a conversation method using a conversational AI-robot friend device according to one embodiment of the present invention comprises the steps of: in step S101, determining whether the robot is within a visible range where a conversation with the owner (robot user) is possible by a visibility determination means; in step S103, storing the conversation content between the owner and the robot in a conversation black box memory (28b); in step S104, requesting the super-large AI (16) to select a conversation topic by a topic of interest search unit (77); in step S106, requesting the super-large AI (16) to generate robot dialogue based on the conversation topic selected in step S104 or the owner's dialogue (robot user's dialogue) and transmitting the generated robot dialogue to the owner; and in step S108, recognizing a voice during the conversation by a voice authentication means (71) to authenticate the owner, thereby enabling offline conversation between the owner (robot user) and the robot.

[0208] The above conversation method may further include a step in which the owner uses a cell phone (20) to call the robot (200) or the robot (200) uses a cell phone (20) to call the owner's cell phone (20).

[0209] Another aspect of the conversation method using the conversational AI-robot friend device is a step of registering and storing personal information of virtual friends through a friend conversation app (48) installed on the owner's terminal (300) in step S109; a step of allowing the owner to select one of the virtual friends using a mobile phone (20) and call him or have one of the virtual friends call the owner's mobile phone (20) and have a conversation; a step of allowing the conversation between the owner and the virtual friends to be stored in the black box of the conversation black box memory (28b) in step S111; a step of allowing the conversation topic to be selected by requesting the super-large AI (16) to generate a conversation topic by the topic of interest search unit (77) in step S112; And in step S113, the chat generation means (70b) requests virtual friend dialogue from the super-large AI (16) based on the conversation topics selected in step S112, and transmits the generated virtual friend dialogue to the owner; including a step of enabling online conversation between the owner (robot user) and virtual friends.

[0210] In addition, the method for diagnosing and alleviating dementia using a conversation method according to the present invention may include a step in which, in step S114, the memory diagnosis unit (80) reads out past memories and transmits them to the super-large AI (16), and requests the super-large AI (16) to generate questions and correct answers or hints and correct answers to test the owner's memory ability from the past memories; a step in which, in step S115, the owner's memory ability is scored or the owner's dementia risk is determined by asking the owner questions using the questions or hints generated in step S114 and checking whether the answers are correct; and a step in which, in step S116, the owner (robot user) is given correct answers or partial hints for past memories that cannot be answered among the questions generated in step S114, thereby enhancing the owner's (robot user's) cognitive ability.

[0211] In the above description, steps S101 to S116 may be further divided into additional steps or combined into fewer steps, depending on the implementation example of the present invention. Furthermore, some steps may be omitted as needed, and the order of the steps may be changed.

[0212] The conversation method using the interactive AI-robot friend device according to one embodiment of the present invention can be implemented in the form of program commands that can be executed through various computer means. The program commands may be specially designed and configured for the present invention or may be known and available to those skilled in the computer software field. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

[0213] Additionally, the conversation method using the aforementioned conversational AI-robot friend device can also be implemented in the form of an application on a mobile phone or cloud server.

[0214] In addition, the robot of the aforementioned conversational AI-robot friend device can be replaced or represented by any system, such as a chatbot, PC, smart mirror, TV, laptop, or mobile phone, that is equipped with artificial intelligence or can communicate with an artificial intelligence platform to converse with the owner.

[0215] The above description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0216] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. Robot; and A conversational AI module integrated within the robot; The above robot, A microphone for eavesdropping on conversations; and Includes a speaker section for transmitting robot dialogue or greeting dialogue by voice; The above conversational AI module, A visibility determination means for determining whether a user registered in the above robot is within a conversational range; A voice authentication means for recognizing a voice input from the microphone during a conversation and determining whether the user is the user; A voice-to-text conversion unit that converts sound information eavesdropped through the above microphone unit into text sentences; A wiretapping memory for storing text sentences obtained by the above wiretapping; A dialogue generation unit that generates the robot dialogue or greeting dialogue from dialogue topics; A text-to-speech conversion unit that converts the robot dialogue or greeting dialogue generated by the dialogue generation unit into voice and outputs it through the speaker unit; and It includes a conversation black box memory for storing the conversation content and corresponding keywords between the user and the robot; Conversational AI-robot friend device.

2. Conversational AI module integrated within the robot; Virtual friends existing in the above robot; and A terminal of a user that is allowed to connect online with the robot; The above user's terminal is, A friend chat app that allows you to register your virtual friends or modify their personal information to conduct online conversations with them; and Includes a virtual friend personal information list memory 1 that stores personal information of the registered virtual friends, The above conversational AI module, A wireless communication connection means for performing an online conversation connection between the user's terminal and the virtual friends; Virtual friend personal information list memory 2 that shares the contents of the above virtual friend personal information list memory 1; A conversation black box memory consisting of black boxes independently allocated to each virtual friend to store the online conversation content and corresponding keywords exchanged between the virtual friends and the user's terminal; A speaker selection unit that receives the corresponding speaker ID of a virtual friend selected by the user's terminal through the wireless communication connection means; An interest topic search unit that derives conversation topics from past online conversation contents read from the black box corresponding to the speaker ID and personal information read from the virtual friend personal information list memory 2 corresponding to the speaker ID; and Including a dialogue generation unit that generates virtual friend dialogue or greeting dialogue from the above dialogue topic; An interactive AI-robot friend device that conducts an online conversation between the above virtual friends and the user's terminal.

3. In paragraph 1, The above-mentioned visibility determination means includes at least one selected from among a facial shape authentication means for determining whether the user's appearance is captured in an image input from a vision camera, a voice authentication means for determining whether the user's voice is captured above a predetermined volume through the microphone unit, and a wireless communication connection means for determining whether the user's wearable device or mobile phone is properly connected to a short-range communication connection.

4. In paragraph 3, The above facial shape authentication means is a conversational AI-robot friend device that is pre-learned by facial shape images of the user from various angles indicated by the user, and is performed by a facial shape recognition artificial intelligence neural network that recognizes the facial shape of the user obtained later by the vision camera.

5. In paragraph 1 or 2, The above interactive AI-robot friend device, Super-large AI; and Including a memory diagnostic section; The above memory diagnosis unit reads out a portion of past memories stored in the conversation black box memory or wiretapping memory and transmits it to the super-large AI, and requests the super-large AI to generate questions and correct answers or hints and correct answers to test the user's memory ability and provide feedback, or transmits the user's answers to the questions or hints to the super-large AI and requests the super-large AI to convert the answer into a score and inform the user of how much it matches the correct answer.

6. In paragraph 1 or 2, The above interactive AI-robot friend device, Super-large AI; and Including a memory diagnostic section; The above memory diagnosis unit reads out a portion of past memories stored in the conversation black box memory or wiretapping memory and transmits them to the super-large AI, and requests feedback by generating questions and answers or hints and answers to questions to test the user's memory ability; and An interactive AI-robot friend device comprising a memory ability scoring means for judging the degree to which the user's answer to the question or hint matches the correct answer.

7. In paragraph 6, The above memory ability scoring means controls the memory generation means to provide the user with the hint when the score of the answer to the question is below a reference value, and evaluates the user's memory restoration ability by judging the score based on how close the user's answer to the hint is to the correct answer.

8. In paragraph 1 or 2, The above conversational AI module further includes a dementia tracking management unit that observes changes in analysis results according to periodic tests for dementia disease and informs the user of the risk of dementia disease, requests cognitive function enhancement training, or requests remote medical diagnosis with a doctor.

9. In paragraph 1 or 2, The above conversational AI module further includes a subject-matter discrimination artificial intelligence learned using sentences labeled according to various subject terms, and the subject-matter discrimination artificial intelligence analyzes the text sentences or online conversation contents to discriminate and extract one or more subject terms.

10. In paragraph 1, The above conversational AI module further includes a subject-matter determination artificial intelligence that extracts a subject-matter from the text sentence, and stores the text sentence and the extracted subject-matter together in the wiretapping memory.

11. In paragraph 1, The conversational AI module further includes an interest topic search unit that selects conversation topics based on keywords that the user was statistically interested in for a predetermined period of time from the content stored in the wiretapping memory or conversation black box memory.

12. In paragraph 2 or paragraph 11, The above-mentioned topic search section is a conversational AI-robot friend device that requests a super-large AI to select conversation topics from news of incidents or accidents on the Internet or SNS and provide feedback.

13. In paragraph 1 or 2, The above interactive AI-robot friend device, Including more super-large AI; An interactive AI-robot friend device, wherein the above-mentioned conversation generation unit comprises a greeting generation means for generating the greeting dialogue; and a chat generation means for generating the robot dialogue or virtual friend dialogue.

14. In paragraph 13, The above greeting generation means is an interactive AI-robot friend device that requests the super-large AI to generate the greeting line and receives feedback to generate the greeting line.

15. In paragraph 13, The above chat generation means is a conversational AI-robot friend device that receives feedback by requesting the creation of the robot dialogue or virtual friend dialogue while transmitting the conversation topic to the super-large AI to create the robot dialogue or virtual friend dialogue, or by transmitting the dialogue of the user in response to the robot dialogue or virtual friend dialogue to the super-large AI to create the next robot dialogue or virtual friend dialogue by receiving feedback from the super-large AI.

16. In paragraph 2, The above friend chat app includes a virtual friend creation unit that creates the virtual friends, The above virtual friend creation unit includes a face creation unit capable of restoring and reproducing the faces of the virtual friends based on the speaker ID assigned to each face of the virtual friends; A voice generation unit capable of restoring and reproducing the voices of the virtual friends by the voice ID assigned to each face of the virtual friends; and An interactive AI-robot friend device comprising a virtual facial expression generation unit that changes the facial expression and lip shape of the virtual friends in synchronization with the voices of the virtual friends.

17. In paragraph 16, The above-mentioned friend conversation app includes a call button means for allowing the user to select and call one of the virtual friends with whom the user wishes to make a call on the above-mentioned friend conversation app, and displays a face corresponding to the speaker ID of the virtual friend in the call on the display window of the above-mentioned friend conversation app, and outputs a telephone voice modulated with a tone corresponding to the voice ID of the virtual friend in the call, an interactive AI-robot friend device.

18. In paragraph 2, The above conversational AI module is a loneliness judgment means for judging whether the user is lonely; and If the user is judged to be lonely by the loneliness judgment means, an auto-dialing means is further included to select one of the virtual friends as a caller and make a call to the user's terminal. The above-mentioned friend conversation app is a conversational AI-robot friend device that displays the face and name of the caller on the user's terminal, allowing the user to check who is calling.

19. In paragraph 18, The above loneliness judgment means determines that the user is lonely if the accumulated conversation volume of the user with a third party is below a predetermined amount, or determines that the user is lonely if the amount of bright facial expression activity of the user for a predetermined period of time is below a predetermined amount.

20. In paragraph 2, The above interactive AI-robot friend device, Including more super-large AI; The conversational AI module receives virtual friend dialogue generated by the super-large AI and transmits it to the friend dialogue app on the user's terminal via the wireless communication connection means, or receives the user's dialogue generated from the user's terminal via the wireless communication connection means and transmits it to the super-large AI while requesting the generation of the next virtual friend dialogue, thereby continuing the online conversation between the virtual friend and the user.

21. Regarding the conversation method of the conversational AI-robot friend device, A step of determining whether the robot user is within a range of vision that can be used for conversation by means of a range of vision determination means; A step of requesting a conversation topic from a super-large AI by the topic exploration department; and A method for conducting an offline conversation between a robot user and a robot using an interactive AI-robot friend device, including the step of requesting a super-large AI to generate robot dialogue based on the conversation topic or dialogue of the robot user by a chat generation means and transmitting the generated robot dialogue to the robot user.

22. Regarding the conversation method of the conversational AI-robot friend device, A step of registering virtual friends through a friend chat app installed on the robot user's terminal; and A method for conducting an online conversation between a robot user and a virtual friend of an interactive AI-robot friend device, comprising: a step in which the robot user selects one of the virtual friends using a mobile phone and makes a call to the robot user or one of the virtual friends makes a call to the robot user's mobile phone and makes a conversation; 23. Regarding the conversation method of the conversational AI-robot friend device, A step of requesting the super AI to generate questions and answers or hints and answers to test the memory ability of the robot user from past memories by extracting past memories and transmitting them to the super AI; A step of scoring the memory ability of the robot user or determining the risk of dementia by asking the robot user the question and checking whether the answer is correct; and A method of conversation of an interactive AI-robot friend device for diagnosing and alleviating dementia, including a step of providing the robot user with the hint for questions that the robot user cannot answer among the above questions, thereby enhancing the robot user's ability to restore memory and cognitive ability.

Citation Information

Patent Citations

  • Interaction between mobile robot and user, System forsame

    KR1020060131458A

  • Instant message service providing method using bot

    KR1020140047491A

  • Case for liquid lipstick and lip gloss

    KR1020220134927A

  • providing method of picture cards for recognition for the elderly

    KR102251902B1

  • System for artificial intelligence matching using media platform and method thereof

    KR102275291B1