Method and apparatus for providing ai-based conversational avatar service

Abstract

A method and apparatus for providing an AI-based conversational avatar service are provided. An AI-based conversational avatar service providing method according to one embodiment of the present invention includes the step of receiving user speech data including text entered into a conversation window from a user terminal, generating a response message through an artificial intelligence model based thereon, generating avatar response data including avatar voice and lip-synced video, and finally providing the avatar voice and video to the user.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for providing an AI-based conversational avatar service, and more specifically, to a method and apparatus that analyzes user input to generate voice and video of an avatar in real time and supports natural conversation by reflecting emotional responses. Background Technology

[0002] Typical interactive avatar systems rely on methods that provide predefined responses based on user input or support simple text-based interactions. These systems are often limited to static response patterns and exhibit restricted capabilities in expressing emotions or real-time reactions.

[0003] These existing systems have technical limitations in recognizing and reflecting user emotions. Because they fall short in providing natural conversation or immersion, they require the use of separate emotion analysis technology for recognition, or result in instances where avatar reactions are not sufficiently synchronized with real-time interaction with the user during the voice and video generation process.

[0004] Against this backdrop, reflecting user emotions and enhancing conversational immersion through real-time interaction between the avatar and the user are emerging as issues that need to be addressed in conversational avatar systems. Prior art literature

[0005] Republic of Korea Published Patent Application No. 10-2023-0111480 The problem to be solved

[0006] The present invention utilizes AI technology to analyze user input in real time, generates voice and video of an avatar in synchronization, and enhances the naturalness and immersion of conversation by integrating emotional responses.

[0007] In addition, it aims to implement more authentic conversational interactions by recognizing user emotions in real time and providing customized responses. means of solving the problem

[0008] A method for providing an AI-based conversational avatar service according to an embodiment of the present invention for achieving the above-mentioned purpose may include: receiving user speech data from a user terminal, including text entered by the user in a conversation window during the process of the user conversing with an avatar as a counterpart; generating a response message from the user speech data based on an artificial intelligence model for generating a response message; generating avatar response data including an avatar voice corresponding to the response message and a video lip-synced to the avatar voice based on an artificial intelligence model for generating an avatar; and providing the voice and video of the avatar to the user through the user terminal.

[0009] The step of generating the above response message includes the step of generating a response text to be provided as a response to a user utterance and emotion information to be assigned to the response text, and the step of generating the above avatar response data may include the step of generating the voice of the avatar by considering the response text and the emotion information together, and the step of generating the image of the avatar by considering the voice of the avatar and the emotion information together.

[0010] The step of receiving the user speech data includes receiving the user speech data from the user terminal, which further includes the user's voice and video, and the step of generating the response message may include converting the user's voice into spoken text based on an artificial intelligence model for speech recognition, generating user emotion information from the text entered by the user in the conversation window, the user's voice and video based on an artificial intelligence model for emotion judgment, and generating the response text and emotion information to be assigned to the response text by using the text entered by the user in the conversation window, the spoken text converted from the user's voice, and the user's emotion information as input values ​​for an artificial intelligence model for generating the response message.

[0011] The step of generating the above response message includes the step of dividing the above response message into a plurality of segments and the step of generating a plurality of emotion information to be assigned to each segment, and the step of generating the voice of the above avatar may include the step of generating a voice that expresses different emotions for each segment by reflecting the plurality of emotion information in the above response message.

[0012] The step of generating the above response message includes determining whether the above response message needs to be provided in the form of a chat window message, distinguishing and generating voice text to be provided as the voice of the avatar and script text to be provided through the chat window, and determining the position where the script text is to be provided within the sequence of voice texts; the step of generating the voice of the avatar includes generating the voice of the avatar using the voice text as the response text and synchronizing the voice of the avatar and the voice text; and the step of providing the voice and video of the avatar may include outputting the script text in the form of a chat window message during the playback of the voice of the avatar, in accordance with the position where the script text determined in the sequence of voice texts is to be provided.

[0013] The text for the script above may be text containing keywords of the text for the voice above.

[0014] The text for the script above may be text corresponding to information not included in the text for the voice above.

[0015] The step of generating the response message further includes the step of generating a response message while the user's utterance is not yet completely finished, based on real-time collected data of the user's utterance data; the step of generating the avatar response data further includes the step of generating a reaction of the avatar corresponding to the response message; and the step of providing the voice and video of the avatar may include the step of providing a reaction of the avatar while the user's utterance is not yet completely finished, based on the conversational leadership tendency of the avatar designated by the user.

[0016] The step of providing the voice and video of the avatar may include receiving a reaction from the user in real time while providing the voice and video of the avatar to the user, determining whether the user's reaction is a reaction intended to take over the initiative of the conversation, and, depending on the user's reaction, stopping the provision of the existing voice and video of the avatar and switching to and providing voice and video intended to hand over the conversation flow to the user.

[0017] Additionally, an AI-based conversational avatar service providing device according to one embodiment of the present invention includes a memory on which at least one program is recorded and a processor that executes said program, and said program may include instructions for performing steps of: receiving user speech data including text entered by a user in a conversation window during a conversation with an avatar as a counterpart from a user terminal; generating a response message from said user speech data based on an artificial intelligence model for generating a response message; generating avatar response data including an avatar voice corresponding to said response message and a video lip-synced to said avatar voice based on an artificial intelligence model for generating an avatar; and providing said avatar voice and video to the user through said user terminal. Effects of the invention

[0018] According to the present invention, a method and apparatus for providing an AI-based conversational avatar service can be provided, which enables more authentic interaction by providing a customized response tailored to the user's emotions through an AI-based analysis and response generation process, in order to further enhance the immersion and naturalness of the conversation, by having a conversational avatar receive a user script in a conversation with a user and generate a response text of the avatar through the script. Brief explanation of the drawing

[0019] FIG. 1 is a diagram showing subjects for providing an AI-based conversational avatar service according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention. FIG. 4 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention. FIG. 5 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention. FIG. 6 is a diagram showing the overall process of providing an AI-based conversational avatar service according to one embodiment of the present invention. FIG. 7 is a diagram showing a computer system according to one embodiment of the present invention. Specific details for implementing the invention

[0020] The present invention will be described in detail below with reference to the accompanying drawings. Hereinafter, repetitive descriptions and detailed descriptions of known functions and configurations that may unnecessarily obscure the essence of the invention are omitted. Embodiments of the present invention are provided to more fully explain the invention to those with average knowledge in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

[0021] Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0022] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0023] FIG. 1 is a diagram showing subjects for providing an AI-based conversational avatar service according to an embodiment of the present invention.

[0024] Referring to FIG. 1, an AI-based conversational avatar service provision method according to one embodiment of the present invention can be performed by a user terminal (110) and a service provision device (120).

[0025] The user terminal (110) is a wired or wireless communication terminal capable of data communication with an avatar creation device, and may be a device equipped with the function of transmitting and receiving data through various input / output devices. The user terminal can play a key role in an environment where the user conducts a conversation with an avatar as the counterpart. The user terminal (110) can operate in a wired or wireless communication environment and can be connected to a network to exchange data with various devices. It supports various communication technologies such as Wi-Fi, LTE, 5G, and Bluetooth, and through this, can interact with the avatar creation device and receive necessary data in real time.

[0026] The user terminal (110) can enable the user to input text into a conversation window through input devices such as a touchscreen, keyboard, or mouse. The script entered by the user is received in real time and can be transmitted to a service provider (120) to be used to generate a response message. If there is a voice input function, the conversation can be conducted based on voice data, thereby making the interaction with the avatar more natural.

[0027] The user terminal (110) is linked with the database (130) to store various user information, such as the user's conversation patterns, preferences, and emotional state, as well as the script entered by the user and the generated response text, and can utilize this information for future conversations. Beyond simple conversation records, this data plays an important role in enabling the AI ​​to generate user-customized responses or maintain a smoother flow of conversation, and also contributes to gradually improving interaction with the avatar through continuous learning. Through this, the user experience can be optimized and more personalized conversation services can be provided.

[0028] The user terminal (110) can provide the user with a video that is lip-synced with the avatar's voice in real time, allowing the user to interact with the avatar as if they were having a conversation with a real person. This real-time interaction provides the user with an immersive conversational experience, and all interactions occurring through the user terminal are processed in real time so that the flow of conversation is not interrupted. In addition, the user terminal can operate flexibly on various devices such as smartphones, tablets, and PCs, and can provide a user interface optimized for the screen size or input method of each device.

[0029] The service providing device (120) can perform various roles in the AI-based conversational avatar service and may be one of the core elements of the system. This device can receive data from the user terminal (110) and generate a response from the avatar based on the data. In the response generation process, AI models such as speech recognition, natural language processing (NLP), and sentiment analysis may be used, and through this process, an appropriate response can be provided to the user. Here, the AI ​​model is not limited to a specific type, and various types of models may be applied, and it may be a model capable of learning.

[0030] Additionally, the service providing device (120) can perform the function of generating a video that is lip-synced with the voice of the avatar. Through this, text or voice data entered by the user can be converted into natural voice and video of the avatar, and the interaction between the user and the avatar can be realistic.

[0031] This device can be designed to support simultaneous connections with multiple users and maintain stable performance even in large-scale concurrent access environments. To achieve this, load balancing algorithms or network optimization techniques may be applied, and conversation flows between various users can be managed independently.

[0032] Additionally, the service providing device (120) can be linked with a database (130) to store and analyze user data in order to provide a user-customized conversation experience. This data can be used to improve the quality of the conversation and continuously improve interaction with the user.

[0033] The database (130) can play an important role in the AI-based conversational avatar service and can perform functions such as storing, managing, and analyzing various data within the system. The database is linked with the user terminal (110) and the service providing device (120) to systematically store various information, such as user speech data, avatar response data, and conversation history. Through this, conversation records with each user are preserved and can be utilized as data for future conversation analysis and improvement.

[0034] Additionally, the database (130) can provide data necessary for the training of the AI ​​model, thereby enabling the service to be continuously improved. Based on the interaction data with the user, the AI ​​model can learn user preferences, conversation patterns, emotional states, etc., and contribute to generating more personalized responses.

[0035] FIG. 2 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to an embodiment of the present invention.

[0036] Referring to FIG. 2, a method for providing an AI-based conversational avatar service according to an embodiment of the present invention may be carried out in the following steps. First, a service providing device (120) may receive a user script (e.g., text entered by the user into a conversation window) from a user terminal (110) (S210). At this stage, the service providing device (120) may receive user speech data in real time and prepare for processing.

[0037] Next, the service providing device (120) can generate a response text of the avatar from the received user script based on an artificial intelligence model for generating responses (S220). In this process, the AI ​​analyzes the user's utterance to derive an appropriate response message and generates a response text so that the avatar can naturally participate in the conversation.

[0038] As an optional embodiment, the service providing device (120) can generate not only response text for a user utterance but also emotion information to be assigned to the text.

[0039] Afterward, the service providing device (120) can generate an avatar response including avatar voice and lip-synced video in response to the generated response text (S230). In this step, not only is the avatar's voice generated, but a video of lip-syncing and facial expressions and gestures of the avatar is also generated to match the voice, thereby providing a more natural response.

[0040] As an optional embodiment, the service providing device (120) can generate the voice and video of the avatar by considering the response text and the emotion information together when emotion information is generated. In this case, the voice of the avatar is based on the text and emotion information, and the video can also provide natural movements by reflecting the emotion information.

[0041] Finally, the service providing device (120) can provide an avatar response to the user through the user terminal (110) (S240). In this process, the user receives the avatar's voice and video in real time and can enjoy a natural conversation experience.

[0042] As such, the method for providing an AI-based conversational avatar service according to FIG. 2 can be achieved through a series of processes including generating a response based on user speech data, generating avatar voice and video, and providing the avatar response to the user.

[0043] FIG. 3 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention.

[0044] Referring to FIG. 3, a method for providing an AI-based conversational avatar service according to one embodiment of the present invention may include a series of processes for receiving voice and video, converting them into text and emotion information, and finally generating an appropriate response and providing it to a user through an avatar.

[0045] To examine more specifically, first, the service providing device (120) can receive speech data including the user's voice and video from the user terminal (110) (S310). During this process, voice and video data are collected in real time while the user interacts with the avatar, and this can be used as basic data for generating a response message.

[0046] After that, the service providing device (120) can convert the received voice into text based on an artificial intelligence model for voice recognition, and can generate user emotion information from the user's voice, video, and text entered in the conversation window using an artificial intelligence model for emotion judgment (S320). Through this, beyond simple text conversion, the user's voice intonation or facial expressions in the video can be analyzed to more accurately determine the emotional state.

[0047] Finally, the service providing device (120) can generate a response message by considering the converted text and emotion information (S330). At this time, the artificial intelligence model for generating the response message can generate an appropriate response message and emotion information corresponding to the message together based on the user's spoken text, the text converted into voice, and the emotion information. Through this, the avatar's response can be adjusted to match the user's state and conversation context.

[0048] Through this, the service providing device (120) can process voice and video together when receiving user speech data, and can generate response messages and emotion information based on information generated through emotion judgment and voice conversion.

[0049] FIG. 4 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention.

[0050] Referring to FIG. 4, a method for providing an AI-based conversational avatar service according to one embodiment of the present invention may include a series of processes for dividing a response message into a plurality of segments, assigning different emotional information to each segment, and then generating voices that reflect emotions for each segment based on the same.

[0051] To examine more specifically, first, the service providing device (120) can divide the response message into several sections (S410). In this process, appropriate sections are divided according to the meaning or context of the response message, and each section can be set to express different emotions.

[0052] Next, multiple emotional information to be assigned to each segment can be generated (S420). In this step, an artificial intelligence model for emotion judgment can analyze and assign an emotion suitable for each segment of the response message, thereby reflecting different emotional states for each segment.

[0053] After that, the service providing device (120) can generate voices expressing different emotions for each segment by reflecting the emotion information assigned to each segment (S430). The voices generated at this stage have different characteristics such as intonation, tone, and speed to match the emotion assigned to each segment, thereby allowing the avatar's response to maintain a more natural and emotional flow.

[0054] Through this, the service providing device (120) can divide the response message into multiple segments and automatically assign emotional information to each segment, thereby expressing the emotional depth of the conversation more richly and providing the user with a natural and emotionally immersive conversation experience.

[0055] FIG. 5 is a flowchart illustrating an operation method for providing an AI-based conversational avatar service according to another embodiment of the present invention.

[0056] A method for providing an AI-based conversational avatar service according to one embodiment of the present invention may determine whether there is a need to provide a conversation window message as an optional embodiment. Based on an artificial intelligence model for generating responses, the need for a conversation window message can be determined in situations where the voice of the avatar alone is insufficient or additional information is needed (S510).

[0057] Subsequently, a dialog box message distinct from the voice message can be generated (S520), which means generating a response text provided via voice and a text for the script to be provided through the dialog box separately. Through this distinction, the dialog box message can serve to convey important or additional information independently of the voice message.

[0058] As an optional embodiment, the text for the script may include the gist or keywords of the voice message. This is a method of summarizing important content and presenting it in text when the voice message is long or complex. This allows the user to quickly grasp key points through the dialogue window while listening to the voice.

[0059] As an optional embodiment, the text for the script may provide information that is unsuitable for voice delivery, such as links. Such information is often difficult or inappropriate to convey via voice messages. In this case, by providing functions such as allowing access to external data by clicking a link through the text for the script, richer information delivery to the user is enabled.

[0060] According to an optional embodiment, the text for the script is information shared through a chat window and may include visual materials such as illustrations and tables in addition to text. This means that during a conversation with an avatar, the user may receive not only simple text but also various visual information that is unsuitable for conveying via voice messages. These materials can be provided to the user in real time through the chat window, allowing the user to understand the conversation more richly and intuitively.

[0061] Next, the synchronization between the chat window message and the voice message can be determined (S530). This allows the chat window message to be synchronized so that it is provided at an appropriate timing in accordance with a specific point in time of the voice message, and helps the user have a richer conversational experience by utilizing voice and text simultaneously.

[0062] Finally, during the delivery of a voice message, a chat window message can be provided in accordance with the set sync (S540). This effectively maintains interaction between the voice message and the chat window message, and allows the user to check the main parts of the information conveyed by the avatar as text through the chat window. This method can be used to supplement additional information not covered in the voice message with text, or to emphasize important parts of the voice message.

[0063] In this way, the service providing device (120) can support the user in understanding the conversation content more intuitively and experiencing immersive interaction through the interaction between voice messages and conversation window messages.

[0064] FIG. 6 is a diagram showing the overall process of providing an AI-based conversational avatar service according to one embodiment of the present invention.

[0065] Referring to FIG. 6, an AI-based conversational avatar service provision method according to one embodiment of the present invention can be performed by a user terminal (110) and a service provision device (120).

[0066] The user terminal (110) can exchange data in real time with the service providing device (120) and can transmit text or voice data for a script for conversation. This data can be used as an input value for an AI model in the service providing device (120) and, after processing, can be converted into a response message of an avatar.

[0067] Additionally, the user terminal (110) can be linked with the database (130) to store information such as user scripts, conversation patterns, and preferences. This data can be used for future personalization of conversations and can play an important role in improving interaction with the avatar.

[0068] The service providing device (120) generates a response from an avatar based on text or voice data transmitted from a user terminal (110) and can transmit it back to the user terminal (110). In this process, AI models such as speech recognition, natural language processing (NLP), and sentiment analysis may be applied.

[0069] Additionally, the service providing device (120) can be linked with a database (130) to store and analyze user data. Based on this data, it can provide a user-customized conversational experience and contribute to the continuous learning and improvement of the AI ​​model.

[0070] The database (130) can serve as a key component for systematically storing scripts and conversation-related information entered from the user terminal (110). The information stored in this database (130) can be utilized as an essential resource for improving conversations and providing user-customized services in the future.

[0071] Specifically, the database (130) stores various information such as user speech data, conversation history, preferences, and emotional state, and this data can be analyzed in real time or periodically.

[0072] Additionally, the database (130) can be closely linked with the service provider (120) to provide important data for the AI ​​model's learning and response generation processes. Through this linkage, the database (130) can supply accurate and rich data necessary for the AI ​​model to learn user preferences, conversation patterns, emotional states, etc.

[0073] As a result, the AI ​​model can analyze the user's behavioral patterns based on the information in the database (130) and generate more personalized responses. Through this data provision and learning process, the database (130) can continuously improve the performance of the AI ​​model and provide the user with a more sophisticated and customized conversational experience.

[0074] FIG. 7 is a diagram showing a computer system according to one embodiment of the present invention.

[0075] Referring to FIG. 7, a computer system (1000) may include one or more processors (1010), memory (1030), user interface input device (1040), user interface output device (1050), and storage (1060) that communicate with each other via a bus (1020). Additionally, the computer system (1000) may further include a network interface (1070) connected to a network (1080). The processor (1010) may be a central processing unit or a semiconductor device that executes processing instructions stored in memory (1030) or storage (1060). Memory (1030) and storage (1060) may be various forms of volatile or non-volatile storage media. For example, memory may include ROM (1031) or RAM (1032). Explanation of the symbols

[0076] 110: User terminal 120: Service providing device 130: Database 1000: Computer System 1010: Processor 1020: Bus 1030: Memory 1031: ROM 1032: RAM 1040: User interface input device 1050: User interface output device 1060: Storage 1070: Network Interface 1080: Network

Claims

Claim 1 A service providing device receives user speech data from a user terminal, including text entered by the user into a chat window during the process of conversing with an avatar as the counterpart; a service providing device generates a response message from the user speech data based on an artificial intelligence model for generating a response message; a service providing device generates avatar response data including an avatar voice corresponding to the response message and a video lip-synced to the avatar voice based on an artificial intelligence model for generating an avatar; and a service providing device provides the voice and video of the avatar to the user through the user terminal, wherein the step of receiving user speech data includes receiving the user speech data from the user terminal, further including the voice and video of the user, and the step of generating the response message includes: converting the user's voice into spoken text based on an artificial intelligence model for speech recognition; and generating user emotion information from the text entered by the user into a chat window, the user's voice, and the video based on an artificial intelligence model for emotion judgment. The step of generating a response text and emotional information to be assigned to the response text is included, wherein the text entered by the user in the chat window, the utterance text converted from the user's voice, and the user's emotional information are input values ​​for an artificial intelligence model for generating a response message, and the step of generating the avatar response data includes: a step of generating the voice of the avatar by considering the response text and the emotional information together; and a step of generating the image of the avatar by considering the voice of the avatar and the emotional information together, and the step of generating the response message includes a step of determining whether the response message needs to be provided in the form of a chat window message.The method includes the step of generating voice text to be provided as the voice of the avatar and script text to be provided through the dialogue window by distinguishing between them; and the step of determining the position where the script text is to be provided within the sequence of voice texts, wherein the step of generating the voice of the avatar includes the step of generating the voice of the avatar by using the voice text as the response text. The method includes the step of synchronizing the voice of the avatar and the voice text, and the step of providing the voice and video of the avatar includes the step of outputting the script text in the form of a dialogue box message at a position where the script text determined in the sequence of the voice text is to be provided during the voice playback of the avatar, and the step of generating the response message further includes the step of generating the response message while the user's utterance has not completely ended, based on real-time collected data of the user's utterance data, and the step of generating the avatar response data further includes the step of generating the avatar's reaction corresponding to the response message, and the step of providing the voice and video of the avatar includes the step of providing the avatar's reaction while the user's utterance has not completely ended, based on the conversational leadership tendency of the avatar designated by the user, and the step of providing the voice and video of the avatar includes the step of receiving the user's reaction in real-time while providing the avatar's voice and video to the user; and the step of determining whether the user's reaction is a reaction intended to take over the initiative of the conversation. A method for providing an AI-based conversational avatar service, comprising the step of stopping the provision of voice and video of the existing avatar according to the reaction of the user, and switching to and providing voice and video to hand over the conversation flow to the user. Claim 2 delete Claim 3 delete Claim 4 A method for providing an AI-based conversational avatar service, wherein the step of generating the response message comprises: dividing the response message into a plurality of segments; and generating a plurality of emotion information to be assigned to each segment, and the step of generating the voice of the avatar comprises reflecting the plurality of emotion information in the response message to generate a voice expressing different emotions for each segment. Claim 5 delete Claim 6 A method for providing an AI-based conversational avatar service, wherein, in claim 1, the text for the script is a text containing a keyword of the text for the voice. Claim 7 A method for providing an AI-based conversational avatar service, wherein, in claim 1, the text for the script is text corresponding to information not included in the text for the voice. Claim 8 delete Claim 9 delete Claim 10 A memory on which at least one program is recorded; and a processor for executing said program, wherein the program includes instructions for performing the steps of: receiving user speech data from a user terminal, the user speech data including text entered by the user in a conversation window during a conversation with an avatar as the counterpart; generating a response message from said user speech data based on an artificial intelligence model for generating a response message; generating avatar response data including an avatar voice corresponding to said response message and a video lip-synced to said avatar voice based on an artificial intelligence model for generating an avatar; and providing the voice and video of said avatar to the user through said user terminal, wherein the step of receiving said user speech data includes receiving said user speech data from said user terminal, the user speech data including the user's voice and video further, and the step of generating the response message includes: converting said user's voice into spoken text based on an artificial intelligence model for speech recognition; and generating said user emotion information from the text entered by said user in the conversation window, said user's voice, and video based on an artificial intelligence model for emotion judgment. The step of generating a response text and emotional information to be assigned to the response text is included, wherein the text entered by the user in the chat window, the utterance text converted from the user's voice, and the user's emotional information are input values ​​for an artificial intelligence model for generating a response message, and the step of generating the avatar response data includes: a step of generating the voice of the avatar by considering the response text and the emotional information together; and a step of generating the image of the avatar by considering the voice of the avatar and the emotional information together, and the step of generating the response message includes a step of determining whether the response message needs to be provided in the form of a chat window message.The method includes the step of generating voice text to be provided as the voice of the avatar and script text to be provided through the dialogue window by distinguishing between them; and the step of determining the position where the script text is to be provided within the sequence of voice texts, wherein the step of generating the voice of the avatar includes the step of generating the voice of the avatar by using the voice text as the response text. The method includes the step of synchronizing the voice of the avatar and the voice text, and the step of providing the voice and video of the avatar includes the step of outputting the script text in the form of a dialogue box message at a position where the script text determined in the sequence of the voice text is to be provided during the voice playback of the avatar, and the step of generating the response message further includes the step of generating the response message while the user's utterance has not completely ended, based on real-time collected data of the user's utterance data, and the step of generating the avatar response data further includes the step of generating the avatar's reaction corresponding to the response message, and the step of providing the voice and video of the avatar includes the step of providing the avatar's reaction while the user's utterance has not completely ended, based on the conversational leadership tendency of the avatar designated by the user, and the step of providing the voice and video of the avatar includes the step of receiving the user's reaction in real-time while providing the avatar's voice and video to the user; and the step of determining whether the user's reaction is a reaction intended to take over the initiative of the conversation. An AI-based conversational avatar service providing device comprising the step of, depending on the reaction of the user, stopping the provision of voice and video of the existing avatar and switching to and providing voice and video to hand over the conversation flow to the user.

Images