Information processing device and information processing method

Abstract

To solve the problem that it is general to generate a response on the basis of single input and an ability for continuously tracking emotions and intentions of a user is limited, and thus, it is difficult to maintain a natural dialog in a conventional AI dialog system.SOLUTION: The present invention applies a technique named dead-reckoning for estimating a current position from a position and speed in the past in navigation and aviation to a dialog with AI, extracts important information from dialog data in the past of a user, and estimates current needs and emotions. Thus, the intentions of the user are understood, directivity of the dialog is defined, the emotions and degrees of interest are grasped as speed and force of the dialog, a psychological state of the user and context of the dialog are more accurately analyzed, and a response adaptable to the analyzed psychological state and context are generated. For that purpose, the emotions of the user are estimated, and directivity of the dialog is analyzed to generate a personalized response by a response generation unit to be constituted of a user emotion analysis unit, a dynamic dialog navigation analysis unit, a personalized response generation unit, and a learning and evolution unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] This invention relates to an artificial intelligence (AI) dialogue system, and in particular to a method for making inferences from user behavior and past dialogues and proceeding with the dialogue based on those inferences. [Background technology]

[0002] In recent years, advances in AI technology have led to the adoption of dialogue between AI and humans as a means of everyday communication. Its use is particularly expanding in fields such as customer support, education, and entertainment. Conventional AI dialogue systems typically generate responses based on direct user inquiries and commands. However, this approach has limitations in terms of continuously understanding the user's intentions and emotions and maintaining natural dialogue. For example, when a user makes an emotional statement, conventional systems have difficulty interpreting and responding to that emotion appropriately, which often results in a lack of natural conversation flow. Against this background, there is a demand for the development of systems that can engage in more human-like conversations by understanding not only the content of user comments but also past conversation history and context, and generating responses based on this. For example, a patent application previously filed by the applicant of this invention proposes recording conversations between AI and users in association with the user's emotions at the time of the conversation, and having AI generate responses based on the recorded conversation and the current conversation. (See Patent Document 1.) [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent application 2024-036021 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional AI dialogue systems typically generate responses based on single inputs, limiting their ability to continuously track and understand users' emotions and intent. This is because the systems treat each input as an independent event and lack the ability to continuously analyze the context of past conversations and the user's emotional fluctuations. This technical limitation makes it difficult for AI to accurately grasp the emotions and intent behind a user's statements, often resulting in fragmented and unnatural conversations. Furthermore, conventional systems have difficulty understanding and appropriately reflecting users' behavioral patterns and individual needs during a conversation, making it difficult to effectively provide the information and services users desire.

[0005] This invention addresses these challenges by applying dead reckoning to generate responses, making interactions between AI and humans more natural, consistent, and satisfying for users. [Means for solving the problem]

[0006] Dead reckoning navigation is a navigation technique used primarily in marine and aviation navigation to estimate a vehicle's current position from its past position and speed. This method is important for determining the estimated position of a moving object even when no external reference point is available. Dead reckoning starts from a known position and continuously tracks subsequent changes in speed and direction to update the position. Dead reckoning technology for moving objects Dead reckoning involves the following steps: 1. Setting the initial position First set the exact location. 2. Measuring speed and direction Calculates the distance traveled by a ship or aircraft, assuming it is traveling at a specific speed and direction. 3. Calculating Elapsed Time Measure the time it takes to travel. 4. Current location estimation The current position is estimated by adding the distance traveled, calculated based on the speed and direction over time, to the initial position.

[0007] By applying this technique to a dialogue system between AI and humans, or between AI and other AI, it is possible to extract important information from the user's previous dialogue data and use that information to infer the user's current needs and emotions. Specifically, it analyzes the user's intentions and needs from their statements and defines this as the direction of the dialogue. Furthermore, it captures the user's emotions and level of interest as the speed and momentum of the dialogue, allowing the AI ​​to more accurately analyze the user's psychological state and the context of the dialogue and generate responses that are adapted to them. This makes it possible to maintain a natural dialogue flow and improve the user experience. In particular, the information processing device according to the first aspect of the present invention receives input information from a user. and further includes at least one of a user profile, a conversation record, and various sensor information. an input information control unit that acquires input content including the Including the user's emotional state and interest level An emotion analysis unit that acquires user characteristic information and analyzes the input content User interaction A dynamic dialogue navigation analysis unit that acquires intention and background information, and a user's interest in the dialogue based on the information acquired by the input information supervision unit, the emotion analysis unit, and the dynamic dialogue navigation analysis unit. and Current state of needs and continuously grasp changes in the state, and taking into consideration changes in the user's interests and needs and fluctuations in emotions, predict the direction of the dialogue as a course, and according to the current state on the predicted course, The system is characterized by comprising a response generation unit that generates an active response appropriate to the context of the dialogue, and an output unit that outputs the active response generated by the response generation unit. Further, an information processing method according to a second aspect of the present invention is an information processing method executed by a computer, The information includes at least input information from the user, and further includes at least one of a user profile, a conversation record, and various sensor information. An input information unification step of acquiring input content, and analyzing the input content Including the user's emotional state and interest level A sentiment analysis step of acquiring user characteristic information, and analyzing the input content. User interaction A dynamic dialogue navigation analysis step for acquiring intention and background information, and a user's interest in the dialogue based on the information acquired by the input information integration step, the emotion analysis step, and the dynamic dialogue navigation analysis step. and Current state of needs and continuously grasp changes in the state, and taking into consideration changes in the user's interests and needs and fluctuations in emotions, predict the direction of the dialogue as a course, and according to the current state on the predicted course,The method is characterized by comprising a response generation step of generating an active response appropriate to the context of the dialogue, and an output step of outputting the response generated in the response generation step.

[0008] Application of AI to human dialogue The information processing device that applies dead reckoning in this invention applies the above-mentioned technology to the context of a dialogue with an AI. This allows the device to infer the user's "position" in the dialogue (i.e., the current state of their interests and needs) and use it to guide an appropriate response. This process consists of the following steps to deeply understand the user's needs and effectively progress the dialogue. Initial dialogue settings: At the start of a conversation, the system uses the user's profile and conversation history database to understand their past behavioral patterns, interests, and psychological trends, including text denoising, tokenization, normalization, and semantic feature extraction, to accurately identify the user's current state and needs. Collecting information through dialogue: Analyze additional user comments and reactions in real time to understand sentiment and intent. Use session management to continuously track the context of the conversation and capture changes in user interests and needs. Utilize real-time sentiment analysis tools to detect fluctuations in user sentiment and adjust the direction of the conversation appropriately. ·Evaluation of progress dialogue: The user's reaction time, input content, and depth are used to assess their level of interest and the complexity of the problem at that time, which is important for understanding user behavior patterns and emotional changes during a session. Generate responses based on inferred needs: Information from the initial and ongoing interactions is combined to infer the user's current needs and intent. Response templates and personalized suggestions are selected based on intent recognition and contextual information, and are further customized based on the user's past interactions, preferences, and emotional history. Feedback is used to periodically update the response generation and intent recognition models, driving continuous learning and evolution of the system. [Effects of the Invention]

[0009] Our information processing system, which applies dead reckoning, enables AI to not only respond reactively but also continuously predict the "course" of the user's interests and needs through dialogue, and provide appropriate information and support based on that prediction. This system infers changes in the user's intentions and emotions from not only what the user says but also the dialogue history and context, and generates adaptive responses. This enables the AI ​​to sensitively respond not only to the user's emotions but also to the intentions and needs behind them, making dialogue more natural and consistent and significantly improving the user experience. As a result, interactions with AI feel more human-like, increasing user satisfaction and expected to improve engagement. Furthermore, this evolving dialogue system uses user feedback as learning material to continuously improve its performance and provide more accurate dialogue assistance. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an overview of an information processing device to which dead reckoning is applied according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating a configuration of a terminal 10 of an information processing device to which dead reckoning is applied according to a first embodiment. [Figure 3] FIG. 1 is a diagram illustrating a configuration of functional blocks in a terminal 10 of an information processing device to which dead reckoning is applied according to a first embodiment. [Figure 4]FIG. 10 is a flowchart illustrating processing by a terminal 10 of an information processing device to which dead reckoning is applied according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a dialogue by response generation of the information processing device to which dead reckoning according to the first embodiment is applied. [Figure 6] FIG. 10 is a diagram illustrating an overview of an information processing device to which dead reckoning is applied according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following describes in detail the embodiments of the present invention with reference to the drawings. In the following, the same or corresponding parts in the drawings are given the same reference numerals, and their descriptions will not be repeated in principle.

[0012] (Embodiment 1)

[0013] FIG. 1 is a diagram illustrating an overview of an information processing system 1 according to a first embodiment. Referring to FIG. 1, the information processing system 1 according to the first embodiment is mainly a dialogue system between a user and an AI. Specifically, the information processing system 1 includes a terminal 10. The terminal 10 is, for example, a portable PC (Personal Computer) carried by the user, and the terminal 10 may have any shape. The terminal 10 includes input devices such as a microphone and a camera, a generative AI model that supports multimodal communication, a device for recording information, sensors for acquiring information such as position, temperature, tilt, and acceleration, a device for outputting information, and the like, and the user and the AI ​​dialogue with each other.

[0014] FIG. 2 is a diagram illustrating a terminal 10 of an information processing system 1 according to a first embodiment. Referring to FIG. 2, the terminal 10 includes a control unit 11, a camera 12, a communication I / F 13, an input device 14, an output device 15, a microphone 16, a storage device 17, an internal bus connecting each of the components, and various sensors. The control unit 11 includes a processing unit (CPU, GPU, NPU), a random access memory (RAM), and a read-only memory (ROM). The camera 12 acquires biometric data of a user and image data to be input to an AI. For example, an infrared camera may be used to acquire environmental information of a user. The microphone 16 acquires voice data of a user and environmental information of a user. The communication I / F 13 communicates with an external device. The input device 14 includes input by a user via touch, a mouse, or a keyboard. The output device 15 includes an information display device such as a display, a speaker, and the like. The storage device 17 includes various application programs, various generative AI models, profiles of users' hobbies and preferences, records of interactions between users and AI, and content such as music, images, and videos.

[0015] By consolidating all functions in Device 10 in this way, users can interact with the AI ​​without going through a network, shortening response times and improving processing efficiency on Device 10. In addition, users' emotional data and other personal information are processed without being transmitted outside the device, thereby strengthening user privacy protection.

[0016] 3 is a diagram illustrating a functional block configuration of terminal 10 according to the first embodiment. Referring to FIG. 3, terminal 10 includes an input information supervision unit 110, a response generation unit 120, a recording unit 130, and an output unit 140. Input information supervision unit 110 includes an input information acquisition unit 111, a user profile acquisition unit 112, and a dialogue record acquisition unit 113. Response generation unit 120 includes a user emotion estimation unit 121, a dynamic dialogue navigation analysis unit 122, a personalized response generation unit 123, and a learning and evolution unit 124.

[0017] Specifically, the input information control unit 110 acquires user input from the camera, microphone, touch (keyboard) operation, etc. of the terminal 10 via the input information acquisition unit 111, acquires content recorded in the storage device 17, and acquires environmental information from various sensors. Furthermore, the user profile acquisition unit 112 acquires the user profile recorded in the storage device 17 via the recording unit 130, and the dialogue record acquisition unit 113 acquires the most recent dialogue record stored in the storage device 17 via the recording unit 130, and also analyzes the content of the current dialogue, extracts dialogue records that fit the content of the dialogue from the storage device 17, and acquires them via the recording unit 130 (input content).

[0018] The response generation unit 120 is a multimodal AI, consisting of a single generative AI model or an AI whose functions are individually modularized and integrated into one. The response generation unit 120 can be a single generative AI model, or it can be a group of multiple models trained with specialized knowledge. In this case, it is possible to select which model to use to generate a response depending on the content of the dialogue. This allows each model to be lightweight and compact, making it possible to run it on a device with limited resources. Furthermore, by connecting to a network via the communication I / F 13, it is possible to search for information on the Internet and include it in the response generation. Furthermore, if there is information in the dialogue that the AI ​​has not yet learned, it may be able to communicate with AI on other devices to generate a response related to the information the user is looking for. In this case, it will be necessary to consider the user's privacy to prevent the user's personal information or the content of the dialogue from being leaked to external networks. However, if it is absolutely necessary to request information from an external AI and send the content of the dialogue or some personal information, it may be possible to inform the user of this and obtain their permission. The user emotion analysis unit 121 analyzes input content, such as text acquired by the input information control unit 110, voice tone acquired from a microphone, facial expressions acquired from a camera, user profile, and past conversations, to acquire user characteristic information, such as the user's emotional state, strength, psychological state, psychological tendency, psychological pattern, and behavioral pattern. Specifically, emotions can be estimated by analyzing text content, the user's voice, sighs, breathing sounds, and the user's facial expressions, such as eyelid opening, gaze, eyebrow movement, presence or absence of nose wrinkles, mouth movement, mouth opening, and pupil dilation. The emotions and psychological states that can be estimated include joy, anger, sadness, fun, calmness, surprise, satisfaction, boredom, disappointment, fear, relief, and anxiety. While the emotion analysis unit 121 can estimate emotions using only one piece of information, combining multiple pieces of information enables more accurate estimation of user emotions. Furthermore, emotions can also be estimated by acquiring biometric data using a wearable device. The dynamic dialogue navigation analysis unit 122 uses natural language processing technology to extract the intention contained in the user's words from the input content, and also identifies the user's needs and obtains the dialogue intention and background information that will determine the direction of the dialogue. Specifically, it performs semantic analysis of text and audio data, integrating language patterns, contextual clues, and the user's previous interactions to identify the user's unique intent. The initial analysis stage involves tokenizing the input data and extracting relevant entities and phrases to abstract the basic elements of the user's question or request. Next, using deep learning models and other methods, the system extracts high-level contextual relevance from these elements and maps them to specific intents and needs. This stage evaluates not only the meaning of the user's words, but also how they relate to previous interactions and the user's profile. As the system gains a deeper understanding of intents and needs, the dynamic dialogue navigation analyzer 122 derives the optimal dialogue route from the user's current request and previous behavioral patterns. The personalized response generation unit 123 then forms information and suggestions customized to the user's needs based on the analyzed direction of the dialogue, and further adjusts the response using personalized data obtained from the user's past dialogue history, while determining the momentum of the dialogue from the strength of the user's emotions and generating an appropriate response. The Learning and Evolution Unit 124 collects user feedback on responses to improve the quality of responses, and performs AI learning at appropriate times, such as at the end of a conversation, when the system is inactive.

[0019] The recording unit 130 retrieves and stores data such as user profiles, records of dialogue content, audio, images, and videos obtained in advance through questionnaires or the like when the user starts using the system from the memory device 17 of the terminal 10.

[0020] The output unit 140 renders responses to the user as text, images, or videos on a display, outputs them through speakers, or projects them onto a wall. Furthermore, by outputting information to control the robot's motors, it becomes possible to operate the robot, and it is conceivable that responses can be expressed in sign language to hearing-impaired users through the robot's movements. In this case, by using AI that has learned sign language and acquiring the user's sign language expressions as input from a camera, people with hearing impairments can also interact with the AI.

[0021] 4 is a flow diagram illustrating the process of generating a response to which dead reckoning is applied in terminal 10 according to the first embodiment. Based on this flow diagram, the process of generating a response by the functional blocks will be described. When it recognizes an event such as turning on the power of terminal 10 or detecting a sensor response, it starts a dialogue. When the dialogue starts, the user profile acquisition unit 112 of the input information control unit 110 acquires the user profile from the recording unit 130 (process P0). This user profile is created when the user starts using the system by asking questions and tests to learn about the user's personality, character, and psychological patterns, and analyzing the answers to those questions and recording them as a profile in the storage device 17. Furthermore, if the content of the dialogue with the user is analyzed and information about the user's personality is found, this information can be added to the profile and recorded, making it possible to generate more appropriate responses to the user.

[0022] Next, the dialogue record acquisition unit 113 of the input information control unit 110 acquires past dialogue records between the AI ​​and the user (process P1). In addition to the most recent dialogue record, the dialogue record to be acquired is also acquired by analyzing the current dialogue content and extracting dialogue records that match the dialogue content. Since the number of dialogue records increases as the number of dialogues increases, by recording the dialogues with index information such as the date and time of the dialogue, emotions, and a summary of the dialogue added to the dialogue content, it is possible to efficiently extract dialogue records that match the current dialogue content. Next, the input information acquisition unit 111 of the input information control unit 110 acquires (process P2) the input information that the user inputs to the terminal 10. Here, in addition to input information from the camera, microphone, and various sensors of the terminal 10, content such as audio, images, and videos recorded by the user and stored in the recording unit is also acquired as input information. The information obtained by the input information control unit 110, which may consist of one or a combination of the user profile, dialogue record, and input information, is analyzed by the response generation unit 120 as multimodal input content.

[0023] First, the user emotion analysis unit 121 of the response generation unit 120 analyzes the input content acquired by the input information management unit 110 to analyze the user's emotions (process P3). Based on input information from the camera and microphone of the terminal 10, as well as input from various sensors, the user profile, and conversation records, the analysis acquires user characteristic information that will be the user's behavioral principles, such as the user's current emotional state and the strength of that emotion, the user's psychological state and psychological patterns.

[0024] Next, in the intention and needs analysis (process P4), the dynamic dialogue navigation analysis unit 122 of the response generation unit 120 makes full use of natural language processing (NLP) technology to analyze and clarify the motivations and goals behind the user's words, and obtain the intention and background information of the dialogue. For example, if a user asks a question about a specific product, the system can infer what information the user is looking for based on past purchase history and related questions, and generate a response that provides information that meets those needs. This process predicts the next steps the user will take and guides the conversation to move forward smoothly. This allows the response generator to go beyond simply reflecting information and provide knowledge that advances the user's needs and increases their satisfaction.

[0025] Then, the personalized response generation unit 123 of the response generation unit 120 combines multimodal input content such as input information from the user, user profile, dialogue records, and various sensor information with user characteristic information obtained from the user emotion analysis process, and intention and background information obtained from the intention and needs analysis, to generate a response optimized for each individual user (process P5). The response generation process begins by analyzing the user's input questions and comments and understanding past trends and preferences based on the user profile. At this stage, the system evaluates what topics the user has been interested in in the past and what responses they preferred, and applies that knowledge to the current conversation. At the same time, information gained from the user sentiment analysis process is used to understand the user's current emotional state and formulate an appropriate response. For example, if the user appears depressed, the system may use more careful wording or generate a response that shows empathy. The results of the intent and needs analysis process are used to understand what the user wants and what information would help them. This might include relevant product information, troubleshooting steps, or recommended actions. This analysis allows us to predict what the user will want next and generate a response to move the interaction forward smoothly. Ultimately, by combining all these elements, the personalized response generator 123 generates a response that meets the user's needs and provides a positive interaction experience. Responses are tailored based on information learned from the user's past interactions, the user's current state of mind, and the purpose and goals of the interaction, providing the user with appropriate suggestions and prompts to assist them.

[0026] The response generated by the output unit 140 is then output (process P6). The form of output at this time will depend on the form of the generated information. For example, if the output is text, an image, or video, it can be displayed on a display, or it can be projected onto a wall using a projector function. Alternatively, if it is audio, it can be output through a speaker, or even in the form of a signal to control the robot's movements.

[0027] Furthermore, feedback from the user regarding the generated response is obtained (process P7). The feedback is the user's response to the generated response. When response generation using dead reckoning according to this invention is applied to various service fields, the following feedback collection methods are possible: Direct Feedback: Customer support: Conduct a satisfaction survey after the interaction, asking users for numerical ratings and free-form feedback. Educational applications: Post-learning quizzes and tests assess learning outcomes and analyze how well users have understood the material. Entertainment: Prompts for feedback after gaming or watching on streaming services to help improve content recommendations. Indirect feedback: E-commerce: Use post-purchase reviews and ratings as feedback to improve product recommendations. Health care app: Estimates the effectiveness of a user's health improvement progress and activity data and provides personalized health advice. Automatic feedback: Smart home devices: Automatically detect your daily usage patterns and changes in your environment to automatically adjust settings and improve services. Automobiles: Analyzes the driver's driving style and the conditions inside and outside the vehicle to provide safer and more comfortable driving assistance. Social Feedback: Social Media: Analyzing user posts and reactions to identify trends or suggest content tailored to each individual user.

[0028] This user feedback is then analyzed (process P8) by the learning and evolution unit 124 of the response generation unit 120, which applies the concept of dead reckoning to the dialogue context to promote self-improvement and evolution of the AI. Specifically, the learning and evolution unit collects not only direct user feedback from completed dialogue sessions, but also implicit feedback from user behavior and choices, such as response times, selected dialogue options, and changes in the user's emotions during the dialogue. This data provides important learning material for the AI ​​to gain a deeper understanding of the user's dialogue style and preferences. The collected data is then analyzed using dead reckoning to identify patterns from past interaction records and current feedback to predict future response strategies. For example, if users consistently respond to certain phrases or keywords, the system can recognize this pattern and generate improved responses in similar situations. Additionally, if feedback analysis reveals changes in the user's emotions, the system may intentionally delay the response. Specifically, the system dynamically sets a response delay based on the user's emotional state. For example, if the user is stressed or angry, the system delays the response by a preset time (e.g., 2-5 seconds). This delay is implemented through a response timer in the user interface, which temporarily holds off the AI-generated response before sending it to the user. This delay is set to an appropriate length based on the user's current emotional state through updates to the emotion recognition algorithm. This delay is determined by an emotional state classifier that evaluates the user's emotions in real time and identifies the intensity and type of emotion (e.g., anger, sadness, joy). The timing adjustment algorithm then calculates the appropriate response timing based on the type and intensity of the emotion. This algorithm is adjusted to set a longer delay when the emotion is heightened and a shorter delay when the emotion is calm. The response scheduler then receives input from the timing adjustment algorithm and manages the actual response transmission. The scheduler works in sync with the rest of the system to ensure responses are delivered at the right time while maintaining a natural dialogue flow for the user. This delay allows the AI ​​to show empathy for the user and, if the user is emotionally upset, to pause for a moment rather than responding immediately, giving the user time to calm down before delivering a more thoughtful response. This approach allows the AI ​​to deliver more human-like dialogue and respond in a way that is in tune with the user's emotions. Additionally, the Learning and Evolution Department 124 fine-tunes the AI ​​model based on feedback, continuously making fine adjustments to improve the naturalness and appropriateness of interactions, including updating intent recognition models and response generation algorithms. The ultimate goal of the feedback analysis process is to continuously improve the user's experience during interactions and achieve more human-like, emotionally empathetic AI interactions. Through this process, the AI ​​adapts to emerging user behaviors and grows over time, becoming more personalized in its interactions with users.

[0029] Once this process is complete, the user profile and conversation record are updated (process P9). New information about the user's personality obtained as a result of the analysis is added or modified to the user profile. In addition, new conversations between the AI ​​and the user are indexed and added to the conversation record. This makes it easier to extract and retrieve past conversation history that is highly relevant to the current conversation.

[0030] So far, we have explained how dead reckoning is applied to user input to generate a response as a series of processes using flow diagrams, but the order is not limited to this. While this process can essentially be controlled by a program, it is preferable for the AI ​​to be self-regulating. In some advanced AI systems, machine learning models learn from data and automatically improve their performance over time. This includes using user feedback to fine-tune the model. This process is sometimes considered "self-regulation," where the AI ​​adjusts parameters independently to generate more effective responses. AI systems also gain the ability to make more complex decisions based on accumulated data and experience, enabling them to go beyond the program's initial design and address new scenarios and unknown problems.

[0031] FIG. 5 is a diagram illustrating an example of a dialogue using response generation using dead reckoning according to the first embodiment. Figure 3 shows the user profile and the response generated by the AI ​​by applying dead reckoning to the user's input. While this example only shows a partial user profile, it would be desirable to collect more information during interactions with the user to create a profile. Specifically, if a lot of information about the user's personality, such as the user's gender and age, occupation, hobbies, topics of interest, personality, values, beliefs, knowledge level, lifestyle, interaction preferences, decision-making style, risk response, stress tolerance, sources of motivation, learning style, interpersonal style, emotional fluctuations, reaction to the environment, and tendencies based on past behavior, is obtained and recorded or the AI ​​is trained to learn it, the AI ​​will be able to generate responses that are more tailored to the user. Next, we explain how the AI ​​applies dead reckoning to user input about a trip based on the user's profile and generates a response. The explanation also includes an explanation of how the AI ​​generates a response that captures the user's intent, as well as what the response would be if dead reckoning were not applied.

[0032] Dialogue A User input: "Hi, I'm planning a trip to Europe next month and it's my first time so I'd like some advice." AI Response: "Hi! Your trip to Europe sounds amazing. Which countries are you planning to visit?" Parsing user input and extracting intent: The AI ​​uses natural language processing to analyze the user's input. At this stage, it captures the keywords "travel to Europe" and "first time," making it clear that this is the user's first time visiting Europe and that they are looking for some advice. Because the user says, "It's my first time, so I'd like some advice," the AI ​​infers from this expression that the user may be feeling a bit anxious or excited. ·Generating responses based on intent: The AI ​​generates a response by asking about specific destinations, taking into account that the user is planning a trip and that it is their first visit. This prepares the AI ​​to provide specific advice based on the country the user is visiting. By asking "Which countries are you planning to visit?", the AI ​​aims to elicit more specific information from the user. The AI ​​also responds in a friendly tone and demonstrates empathy for the user, helping them feel at ease during the interaction. Ensuring continuity and appropriateness of dialogue: This response shows interest in the user and paves the way for more detailed advice based on the additional information the user provides. It's a strategic choice that naturally advances the conversation and is designed to encourage the user to dig deeper. Example of AI response when Dead Reckoning is not applied: AI: "Traveling in Europe is fun. There are so many options." In this case, the AI's response is general and doesn't deeply address the user's specific needs or the context of their first trip. This response misses an opportunity to discover what information the user is looking for, potentially reducing the quality of the interaction.

[0033] Dialogue B User input: "I'm thinking France and Italy. Paris and Rome both have a lot of sights I want to see." AI Response: "That's a great choice. In Paris, the Eiffel Tower and the Louvre are famous, and in Rome, the Colosseum and the Vatican Museums are recommended. What are you particularly interested in?" Parsing user input and extracting intent: The AI ​​uses natural language processing to analyze the user's input. At this stage, it extracts the specific locations "France," "Italy," "Paris," and "Rome." This reveals that the user is interested in seeing many of the attractions in these cities. Furthermore, when the user says, "There are many attractions I want to see in both cities," the AI ​​infers that the user is excited and expectant. ·Generating responses based on intent: By naming specific cities, the AI ​​suggests famous tourist attractions related to each city. The Eiffel Tower and Louvre in Paris, and the Colosseum and Vatican Museums in Rome are some of the most iconic landmarks in their respective cities and are typically of high interest to tourists. The AI ​​uses this information to further flesh out the conversation by suggesting specific places that may be of interest to the user, further engaging the user. Furthermore, the AI ​​uses empathetic language, saying, "That's a great choice," demonstrating an attitude that aligns with the user's positive emotions. Deeper dialogue and user interest exploration: AI can further explore a user's interests in specific types of attractions or activities by asking, "What are you particularly interested in?" This approach paves the way for eliciting additional information from users and providing even more personalized information. Example of AI response when Dead Reckoning is not applied: AI: "That's exciting! There are so many wonderful places in France and Italy. Have fun!" In this response, the AI ​​doesn't ask the user specifically what places they're interested in, but instead provides a general encouragement, missing an opportunity to deepen the conversation.

[0034] Dialogue C User input: "I definitely want to see the historical buildings. And I also want to enjoy the local cuisine." AI Response: "Then I suggest you enjoy a croissant in a small cafe in the Marais district in Paris, or authentic Italian food in Trastevere in Rome. What's your budget?" Parsing user input and extracting intent: Using natural language processing, the AI ​​extracts two key points from the user's input: "historical buildings" and "local cuisine." This indicates that the user is interested in cultural tourist attractions and local food culture. Furthermore, the user says, "I really want to see historical buildings. And I also want to enjoy local cuisine," so the AI ​​infers that the user is interested and excited. ·Generating responses based on intent: The AI ​​demonstrates empathy and provides specific recommendations that address the user's interests. Based on the user's interests, the AI ​​suggests specific neighborhoods in Paris and Rome. The Marais and Trastevere districts, respectively, are known for their cultural appeal and local cuisine, providing recommendations tailored to the user's needs. The selection of restaurants and cafes here is based on the unique experiences each neighborhood offers, pinpointing the best places to enjoy the "local cuisine" the user desires. This ensures that the AI ​​makes appropriate suggestions that meet the user's expectations and enhances user satisfaction. Continuing the dialogue and gathering more information: By asking, "What is your budget?", the AI ​​seeks additional information to further personalize its response, allowing it to further tailor its recommendations to the user's financial constraints and provide information that is useful to them. Example of AI response when Dead Reckoning is not applied: AI: "You can enjoy historical buildings and local cuisine in any city. Have lots of fun." In this case, the response is generic and lacks customized information based on the user's specific interests or geographic details.

[0035] Dialogue D User input: "Well, it's better if it's not too expensive. But I want to be comfortable, so I want to spend a moderate amount." AI Response: "I understand. I can provide you with information on cost-effective accommodation options and reasonably priced, reputable restaurants. I can also help you make reservations." Parsing user input and extracting intent: Using natural language processing technology, the AI ​​captures two key points from the user's words: "not too expensive" and "I want to be comfortable." These phrases reveal the user's needs: they want to be budget-conscious but also have a certain level of comfort. Furthermore, when the user says, "Hmm, not too expensive would be better. But I want to be comfortable, so I want to spend moderately," the AI ​​understands emotionally that the user has some concerns about the cost of travel, but also wants to ensure comfort. ·Generating responses based on intent: To address user expectations and concerns, AI chooses to provide information in a way that balances budget constraints with the need for comfort. To do this, it helps users find the balance between quality and price by suggesting "value-for-money accommodation options" and "reasonably priced, reputable restaurants." Furthermore, it aims to enhance planning and convenience by offering assistance with reservation arrangements. In this way, AI generates appropriate responses that are in tune with the user's emotions and needs. · Enhanced dialogue and user support: The AI's response encourages users to take specific actions by offering specific services (such as providing information on accommodation options and restaurants, and assisting with reservation arrangements), which is expected to enable users to effectively plan their trip and further deepen their trust in the AI. Example of AI response when Dead Reckoning is not applied: AI: "Europe is generally expensive, but you might be able to save a little by booking early." This response does not offer direct solutions or suggestions to the user's specific needs (balancing comfort and budget), nor does it offer any specific assistance with booking arrangements.

[0036] (Embodiment 2)

[0037] FIG. 6 is a diagram illustrating an overview of information processing system 1 according to the second embodiment. Referring to FIG. 6, information processing system 1 according to the second embodiment is mainly a dialogue system between a user and an AI. Specifically, information processing system 1 includes terminal 10. Terminal 10 is characterized by its character (robot) shape. Terminal 10 includes input devices such as a microphone and a camera, a generative AI model that supports multimodal communication, a device for recording information, sensors for acquiring information such as position, temperature, tilt, and acceleration, a device for outputting information, and the like, and allows dialogue between the user and the AI. Furthermore, because there is a limit to the amount of information that can be displayed on a small device that runs AI, the character's abdomen is equipped with a projection device as an output unit, which can project the generated responses onto a wall or floor. By incorporating a projector function, it is possible to visually display a large amount of information without being restricted by the screen size of the device. Furthermore, information displayed on walls and floors using the device's built-in projector functionality can be interactively touched, allowing users to interact and navigate the information by directly touching the projected screen. This type of interaction is achieved by combining touch-sensitive functionality with the image projected by the projector, allowing users to intuitively manipulate information without relying on physical devices or regular touchscreens. Specifically, it uses special sensor technology to accurately track the position of the projected image and the movements of the user's hands and fingers. This technology uses infrared and optical sensors to detect hand movements near the projection surface and transmit them to the system as touch actions. This action information is processed by the AI ​​system and can trigger actions such as zooming in on user-selected content, updating information, or generating new queries. The introduction of this interactive projection system is expected to enable a wide variety of uses, including group learning in educational environments, corporate presentations, and even home entertainment. Users will be able to use any flat surface as a display, without being bound by a physical interface, making interactions with AI more dynamic and intuitive. This will further expand the use of AI dialogue systems and improve the quality of the user experience.

[0038] Furthermore, it is conceivable that the device will have a function for casting to large TVs. Specifically, the device will be equipped with a casting function that utilizes wireless communication technology, allowing it to easily connect to large TVs and other display devices within a local network via Bluetooth or Wi-Fi. Using this function, responses from the AI ​​and graphical data generated during interactions, such as maps, statistical graphs, and educational videos, can be displayed on a large screen, allowing multiple users to view the content simultaneously.

[0039] Furthermore, the information processing system 1 according to the second embodiment may be configured to operate on the output projected into space. This allows the user to directly operate the projected information at any position within the space. This technology utilizes the concept of augmented reality (AR), and by overlaying digital information onto the real space, it achieves more dynamic and intuitive user interaction. Specifically, the visual content projected from the device's projector adapts to the surrounding space in real time and recognizes the user's gestures and movements directed at the projected image. This is achieved using advanced image recognition and motion sensing technologies that can capture not only the movements of the user's hands and fingers, but also the entire body. This allows users to give instructions to menus and icons that appear in the air without the need for physical contact. For example, if a user is viewing a presentation projected onto the wall of a conference room, they can simply wave their hand over a specific slide in the presentation to advance to the next slide. In educational settings, a teacher could also use their finger to swipe across an image projected onto the wall during a lesson to instantly display detailed explanations. By combining spatially projected output with its control functions, users can freely utilize their surroundings and enjoy new ways of interacting with information. This technology makes interaction with AI more natural and intuitive, and can be applied in a variety of scenarios.

[0040] In this way, incorporating cast and projector functions can significantly improve the user experience by providing a wealth of content beyond the information capacity constraints of compact devices and enabling large displays of information even in limited spaces. This will expand the scope of use for AI dialogue systems using dead reckoning, enabling them to be used in a wide range of environments, including educational settings, business meetings, and outdoor activities. This can particularly be seen in interactive presentations and collaborative work, where it can promote communication between participants and promote deeper understanding.

[0041] In addition, the information processing system 1 according to the second embodiment has a character-like shape, which makes it possible for the user to feel a sense of familiarity with the system, and also has the effect of making it easier to obtain the user's private, inner, natural emotions when interacting with the AI. This allows the generated responses to be more user-friendly.

[0042] The information processing device to which dead reckoning is applied according to the present invention has the following advantages. 1. By applying dead reckoning to response generation, the AI ​​behaves as if it is actively interested in user input. Dead-reckoning response generation allows AI to learn from a user's past interactions and apply that information to the current interaction. This generates personalized responses based on the user's input, making the AI ​​appear to be "proactively" responding to the user's needs and interests. For example, if a user asks about travel, the AI ​​can make specific suggestions based on the user's travel style and preferences learned from past interactions. AI's engagement with user input goes beyond simply generating a reactive response to asking further questions and eliciting more information from the user. This allows for a deeper understanding of the user's intent and needs, improving the quality of the interaction.

[0043] 2. Make users feel a sense of familiarity with AI, just like they would with a human. When AI remembers a user's past behavior and preferences and customizes responses based on them, the user feels that the AI ​​"understands" them. This personalized interaction helps users perceive the AI ​​as a partner in conversation, rather than just a machine, creating a sense of familiarity. People naturally feel close to those who understand and care about them. Therefore, when AI learns from past interactions and provides responses tailored to each individual user, it builds trust and creates a sense of familiarity.

[0044] 3. Dialogue between AI and humans will be promoted When AI provides users with relevant responses, they are likely to want to engage further. When responses appropriately address users' interests and needs, users are more likely to ask new questions or seek more detailed information. This invigorates the interaction with the AI, providing a richer, more fulfilling interaction experience. This process also increases user engagement, leading to increased trust and reliance on the AI. When AI actively shows interest and continually seeks interaction, users may feel that the AI ​​is genuinely listening to and trying to understand what they are saying, and they may become more actively involved in the interaction. Furthermore, in order for AI to apply dead reckoning to improve and enhance user interaction, it is possible to train AI to have empathetic communication skills. Specifically, it would be effective to train AI in psychology and active listening techniques. Active listening is the art of listening carefully to the other person and empathizing with their feelings and thoughts to achieve effective communication. AI can demonstrate interest in what the user is saying by carefully listening to them and summarizing and confirming what they have said.

[0045] The present invention has been specifically described above based on embodiments, but it goes without saying that it is not limited to these embodiments and that various modifications are possible within the scope of the gist of the present invention.

[0046] If the information processing device that applies the dead reckoning method of this invention is applied to services in various fields that utilize AI, it will be possible to provide more personalized and beneficial services to users by utilizing their past actions and needs. Specifically, (1) in the field of education, AI typically provides feedback based on students' current performance. By applying dead reckoning, it can provide personalized instruction based on students' past learning history and interests. AI can predict changes in students' understanding and interests and suggest optimal teaching materials and methods. This enables education to be tailored to students' understanding and interests, improving learning outcomes. (2) In the entertainment industry, it is common to recommend content based on a user's viewing history and preferences. By applying dead reckoning, it is possible to predict changes in preferences and interests based on a user's past viewing history and conversation flow, providing accurate recommendations and interactive dialogue. This will personalize the user's entertainment experience and increase satisfaction. (3) In the healthcare field, AI can provide advice based on a patient's current symptoms and data. By applying dead reckoning, AI can predict future health risks and needs based on a patient's medical history and current health status, and provide appropriate self-care advice and reminders. This will enable more effective health management for patients and improve the quality of preventive care. (4) In the retail and e-commerce sectors, it is common to recommend products based on a user's purchase history. By applying dead reckoning, it is possible to predict future purchasing intentions and preferences based on a user's purchase history and interests, and provide relevant products and promotions. This is expected to increase users' purchasing motivation and encourage repeat purchases. (5) In the transportation and travel industry, it is common to make plans based on a user's current schedule and preferences. By applying dead reckoning, future travel destinations and activities can be predicted based on the user's past travel history and preferences, and optimal destinations and transportation methods can be suggested. This can improve the user's travel experience and increase satisfaction. (6) In the smart home field, AI typically operates home appliances based on the resident's current behavioral patterns. By applying dead reckoning, AI can predict future behavior and needs based on the resident's past behavioral patterns and preferences, providing more appropriate appliance operation and home automation. This improves the comfort and convenience of residents and increases the value of smart homes. (7) Traditional customer support AI has mainly responded reactively based on the user's current inquiry. By applying dead reckoning, it can refer to the user's past inquiry history and issues to provide optimal advice and responses. AI can predict the user's inquiry content and the intent behind it, and propose appropriate solutions. This can provide users with a consistent support experience and potentially shorten the time it takes to resolve issues. In this way, in a variety of service fields, by generating responses that are not simply messages, but also content such as images and videos, and signals that control various electronic devices, it is possible to improve the user experience and increase satisfaction with the service. [Explanation of symbols]

[0047] 1 Information processing system, 10 Terminal, 11 Control unit, 12 Camera, 14 Input device, 15 Output device, 16 Microphone, 17 Memory unit, 110 Input information control unit, 111 Input information acquisition unit, 112 User profile acquisition unit, 113 Dialogue record acquisition unit, 120 response generation unit, 121 user emotion analysis unit, 122 dynamic dialogue navigation analysis unit, 123 Personalized response generation section, 124 Learning and evolution section, 130 Recording section, 140 Output section.

Claims

1. an input information control unit that acquires input content including at least input information from a user and further including at least one of a user profile, a dialogue record, and various sensor information; an emotion analysis unit that analyzes the input content and acquires user characteristic information including the user's emotional state and interest level; a dynamic dialogue navigation analysis unit that analyzes the input content and acquires the user's dialogue intention and background information; a response generation unit that continuously grasps the current state of the user's interests and needs in the dialogue and changes in said state based on the information acquired by said input information supervision unit, said emotion analysis unit, and said dynamic dialogue navigation analysis unit, predicts the direction of the dialogue as a course in consideration of the changes in the user's interests and needs and fluctuations in emotions, and generates an active response appropriate to the context of the dialogue in accordance with the current state on said predicted course; an output unit that outputs the active response generated by the response generation unit; An information processing device comprising:

2. 1. A computer-implemented information processing method, comprising: an input information integration step of acquiring input content including at least input information from a user and further including at least one of a user profile, a dialogue record, and various sensor information; an emotion analysis step of analyzing the input content to obtain user characteristic information including the user's emotional state and interest level; a dynamic dialogue navigation analysis step of analyzing the input content to obtain user dialogue intentions and background information; a response generation step of continuously grasping the current state and changes in the user's interests and needs in the dialogue based on the information acquired by the input information unification step, the emotion analysis step, and the dynamic dialogue navigation analysis step, predicting the direction of the dialogue as a course in consideration of the changes in the user's interests and needs and the fluctuations in emotions, and generating an active response appropriate to the context of the dialogue in accordance with the current state on the predicted course; an output step of outputting the active response generated in the response generation step; An information processing method comprising:

Images