Method for providing personalized information service by using generative ai, and server and apparatus for implementing same

Abstract

The present invention relates to a method for providing a personalized information service by using generative AI, and a server and an apparatus for implementing same. The server for providing a personalized information service by using generative AI, according to one embodiment of the present invention, comprises: a communication unit for receiving a first natural language command from a home appliance and transmitting, to the home appliance, a response message generated in response to the first natural language command; a prompt generation unit, which extracts first personal characteristic information or first personal preference information from the first natural language command so as to generate a prompt corresponding to the first natural language command on the basis of the first personal characteristic information or the first personal preference information; and a response generation unit for inputting the generated prompt into a generative AI module, and then generating a response message with respect to a result output therefrom.

Description

Method for providing personalized information services using generative AI, and a server and device for implementing the same

[0001] The present invention relates to a method for providing personalized information services using generative AI, and to a server and device for implementing the same.

[0002] Control methods for devices such as home appliances can be distinguished into direct human operation and control via natural language commands. In the case of direct operation, users can control the operation of appliances by manipulating remote controls, buttons, or dials. In the case of control via natural language commands, users input natural language commands into the appliance, and the appliance recognizes and operates accordingly.

[0003] However, since natural language commands spoken or entered by users are based on natural language that varies from person to person, there are significant difficulties in interpreting them and converting them into actual commands. In particular, accuracy in interpreting natural language commands is required depending on each user's past command input habits and the usage environment of the home appliance.

[0004] Accordingly, the present specification intends to describe a method and apparatus for providing personalized information services adaptively to a user.

[0005] This specification aims to solve the aforementioned problems by implementing a method and device that provides a user-customized answer during the process of a user asking a question and obtaining an answer through a home appliance.

[0006] In addition, this specification aims to provide a personalized service by reflecting the user's tendencies, characteristics, or preference information when processing the user's natural language commands.

[0007] In addition, the present specification provides a user-customized response to a user's natural language command and enables the response to be output through the voice preferred by the user.

[0008] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0009] A server providing a personalized information service using a generative AI according to an embodiment of the present invention includes a communication unit that receives a first natural language command from a home appliance and transmits a response message generated in response to the first natural language command to the home appliance; a prompt generation unit that extracts first personal characteristic information or first personal preference information from the first natural language command and generates a prompt corresponding to the first natural language command based on the first personal characteristic information or first personal preference information; and a response generation unit that inputs the generated prompt into a generative AI module and generates a response message regarding the calculated result.

[0010] A method for providing a personalized information service using a generative AI according to an embodiment of the present invention comprises the steps of: a communication unit of a server receiving a first natural language command from a home appliance and transmitting a response message generated in response to the first natural language command to the home appliance; a prompt generation unit of a server extracting first personal characteristic information or first personal preference information from the first natural language command and generating a prompt corresponding to the first natural language command based on the first personal characteristic information or first personal preference information; and a response generation unit of a server inputting the generated prompt into a generative AI module and generating a response message for the calculated result.

[0011] When the present invention is applied, a user-customized answer can be provided during the process of a user asking a question and obtaining an answer through a home appliance.

[0012] When the present invention is applied, personalized services can be provided by reflecting the user's tendencies, characteristics, or preference information when processing the user's natural language commands.

[0013] When the present invention is applied, a user-customized response is provided to the user's natural language command, and the response can also be output through the voice preferred by the user.

[0014] The effects of the present invention are not limited to the effects described above, and various effects of the present invention can be easily derived from the composition of the present invention.

[0015] FIG. 1 is a diagram showing the process of a home appliance changing a feature in accordance with a natural language command input according to an embodiment of the present invention.

[0016] FIG. 2 is a diagram showing the process in which, when a natural language command is input according to another embodiment of the present invention, the home appliance transmits the natural language command to a server accordingly and then receives a result from the server.

[0017] FIG. 3 is a diagram showing the process of generating a response result by processing voice commands in an information recommendation process according to an embodiment of the present invention.

[0018] FIG. 4 is a diagram showing the configuration of a server according to one embodiment of the present invention.

[0019] FIG. 5 is a diagram showing the process of processing a user's natural language command according to one embodiment of the present invention.

[0020] FIG. 6 is a diagram showing the process of a server according to an embodiment of the present invention continuously storing personal characteristic information or personal preference information obtained from a user.

[0021] FIG. 7 is a diagram showing the process of a server according to an embodiment of the present invention obtaining a prompt suitable for a natural language command and setting it as a template.

[0022] FIG. 8 is a diagram showing the process of a server adding a user's personal preference information to a prompt according to an embodiment of the present invention.

[0023] FIG. 9 is a diagram showing the process of a server according to an embodiment of the present invention updating a user's personal characteristic information or personal preference information through feedback or subsequent conversation.

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily implement the invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0025] To clearly explain the present invention, parts unrelated to the description have been omitted, and the same reference numerals are assigned to identical or similar components throughout the specification. Furthermore, some embodiments of the present invention are described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, identical components may have the same reference numeral whenever possible, even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the present invention, such detailed description may be omitted.

[0026] In describing the components of the present invention, terms such as first, second, A, B, (a), (b), etc., may be used. These terms are intended only to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by these terms. Where it is stated that a component is "connected," "combined," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that other components may be "interposed" between each component, or that each component may be "connected," "combined," or "connected" through other components.

[0027] In addition, for convenience of explanation in implementing the present invention, the components may be described in detail; however, these components may be implemented within a single device or module, or a single component may be divided and implemented across multiple devices or modules.

[0028] The home appliance described herein is a device that includes electronic products. The home appliance may be placed in a home, office, etc., and the home appliance may be moved by a person and placed in another location.

[0029] In addition, the home appliance described in this specification is a device that processes natural language commands or a device that transmits natural language commands to a server.

[0030] A home appliance according to an embodiment of the present invention includes a voice recognition device (IoT) capable of providing personalized information / responses optimized for the individual.

[0031] The language model described in this specification is a Large Language Model (LLM) as one embodiment, and the language model may be embedded within a home appliance or embedded in a server. Alternatively, the language model may be embedded in an external device, in which case the home appliance or server may obtain the results of the language model through the external device.

[0032] The personal information described in this specification refers to information generated during the use of a home appliance or entered by a user. Personal information may be classified into personal characteristic information and personal preference information.

[0033] In addition, personal information according to one embodiment of the present invention includes information generated during the use of another home appliance used by the user of the said home appliance, or information entered by the user. Personal information according to another embodiment of the present invention includes information generated or entered during the use of another home appliance placed in the same space as the said home appliance.

[0034] According to one embodiment of the present invention, personal preference information entered during the process of searching for specific content through a TV can be stored in a home appliance or server, and this information can be used in the process of processing natural language commands entered to search for specific content through a TV or other video home appliance.

[0035] Alternatively, personal characteristic information entered during the process of changing the color of the air conditioner may be stored in the home appliance or server, and this information may be used later in the process of processing natural language commands entered to search for a specific home appliance setting method through another home appliance.

[0036] FIG. 1 is a diagram showing the process of a home appliance changing a feature in accordance with a natural language command input according to an embodiment of the present invention.

[0037] A user (1) inputs a predetermined natural language command (voice or text) into a home appliance (100) (S3). The command can be input as voice or as text. The home appliance (device) (100) performs text conversion on the input natural language command and, for the preprocessed result, generates a result corresponding to the command (S5). To generate the result, the home appliance (100) may provide the voice natural language command or the text natural language command to a generative AI module, which is an LLM as an example (S5). The generative AI module may produce a result, and the home appliance (100) may output it as voice or text (S7).

[0038]

[0039] FIG. 2 is a diagram showing the process in which, when a natural language command is input according to another embodiment of the present invention, the home appliance transmits the natural language command to a server accordingly and then receives a result from the server.

[0040] S3 refers to FIG. 1. The home appliance (100) transmits the input natural language command to the server (500) (S11). At this time, if the input command is a voice command, the home appliance (100) can perform a preprocessing process to convert the voice command into text.

[0041] The preprocessing process of converting voice commands into text may be performed on a server (500). In this case, the home appliance (100) can transmit the input command to the server (500) as is.

[0042] Alternatively, the server (500) may not perform preprocessing on the voice natural language command.

[0043] To generate a result, the server (500) may provide a voice natural language command or a text natural language command to a generative AI module having an LLM as an example (S15). The generative AI module may produce a result, and the server (500) transmits the result to the home appliance (100) (S16), and the home appliance (100) may output it as voice or text (S17).

[0044] As seen in FIGS. 1 and 2, for natural language commands, the server (500) or the home appliance (100) can produce a suitable result.

[0045] When applying the embodiment of FIG. 1 or FIG. 2, a home appliance that performs voice recognition or text input can provide personalized information and responses optimized for the individual through generative AI.

[0046] The server (500) of FIG. 1 or FIG. 2 is a device for processing artificial intelligence commands and may include a generative AI module. Alternatively, the server (500) may obtain certain information by communicating with an external generative AI module. Accordingly, the server (500) may include an AI server that provides artificial intelligence functions in response to various user commands or requests from home appliances.

[0047] Hereinafter, the present specification examines embodiments focusing on voice natural language commands (abbreviated as voice commands) among natural language commands.

[0048]

[0049] FIG. 3 is a diagram showing a process of processing voice commands to generate a response result in an information recommendation process according to an embodiment of the present invention. It shows an embodiment in which a home appliance (100) receives voice input and transmits it to a server (500), as in the configuration of FIG. 2.

[0050] A voice command is input through a home appliance (100) (S21). For example, a voice command such as "recommend a movie" can be input through a home appliance (100) such as a TV or a refrigerator. The home appliance (100) temporarily stores the voice command as voice data, then converts the voice data into text and transmits it to a server (S22). Unlike S22, the server (500) may convert the voice data into text after the voice data is transmitted to the server (500).

[0051] If the text contains personal characteristic information (e.g., age, occupation, gender, etc.), the server (500) stores the personal characteristic information in a database (S23). Additionally, if the text contains personal preference information (e.g., favorite movie genre, actor, voice of favorite character, etc.), the server (500) stores the personal preference information in a database (S24).

[0052] Personal preference information included in the text can be continuously accumulated and stored thereafter. In addition, if the user (1) uses multiple home appliances, personal preference information collected through each home appliance can also be stored in the database (550).

[0053] Subsequently, the server (500) analyzes the meaning of the text and produces information corresponding to the topic type, content, and intent of the question (S25). For example, the server (500) can classify the topic type of the command, such as whether the command is a movie recommendation or a request for information about movies. Additionally, the server (500) can classify whether the content of the text or the intent of the command is a question or a command instructing a specific task.

[0054] In this process, the server (500) inputs text into the generative AI module (300) for natural language understanding of the text, and can analyze the meaning of the text and check the topic type, content, intent, etc. of the question. The server (500) can store and maintain prompts for text classification and identification, and input them into the generative AI module (300) by combining them with the input text.

[0055] Subsequently, the server (500) generates customized information using information corresponding to the topic type, content, intent, etc. of the question and sets it as a basic template (S26). The basic template includes basic information for generating a prompt. The server (500) can store prompts suitable for the topic type of the question in advance by type, and can load and use the stored prompts according to the type of question.

[0056] The server (500) retrieves personal preference information regarding the subject of the question from the database and adds it to the prompt (S27). Then, the server (500) sets the persona with the voice / tone / speech / voice tone style preferred by the user (S28). Afterwards, the server (500) finally configures the prompt including the question content, a basic template, and persona information (S29).

[0057] Then, the server (500) provides the final configured prompt to the generative AI module (300) to obtain a personalized response (S30). For example, if the generative AI module (300) is chatGPT, the server (500) can provide the final configured prompt to the GPT server through an API provided by GPT.

[0058] Then, the server (500) obtains a personalized response from the generative AI module. Then, the server (500) provides the response generated in the voice / tone / speech style / voice tone preferred by the user to the home appliance (100), and the home appliance (100) outputs it (S31).

[0059] When applying the embodiment of FIG. 3, a voice recognition home appliance (e.g., IoT) can provide personalized information and responses optimized for an individual through generative AI. A server (500) or a home appliance (100) can acquire personal characteristic information and personal preference information, etc., and provide information (expert-level information) and responses (tone, tone, style) optimized for an individual during a conversation between the user and the generative AI or in response to the user's question / request.

[0060] To this end, a home appliance (100) or server according to one embodiment of the present invention can obtain personal characteristic information and personal preference information through text corresponding to a command or content previously stored in a database. In addition, to provide information optimized for the individual (expert-level information) and to provide a response optimized for the individual, an optimized prompt can be generated, input to a generative AI module, and the result obtained.

[0061] Afterward, the server (500) collects and analyzes the user's reactions / feedback through conversation with the user and continuously updates the prompts and personas to be more suitable. Then, the server (500) records and collects personal information / tastes / preferences / interests while repeating the conversation / question / answer process with the user, stores personal characteristic information or personal preference information in a database, and can continuously update the database.

[0062] When applying the embodiment of FIG. 3, conversation or response through generative AI can be performed as a personalized response to the user, rather than a general user-customized response. That is, in the process of speech recognition and conversation using generative AI, the present invention can acquire and collect personal characteristic information or personal preference information and include it in the response message.

[0063] In this case, the level of response content to user conversation / question requests can extend to professional-level content, tailored to the user's characteristics or preferences. Furthermore, since personalized response messages are provided that include persona information rather than a fixed style, users do not experience monotony, which can lead to increased satisfaction with the responses and conversations.

[0064] The process of Fig. 3 is performed by a server (500), but according to another embodiment of the present invention, a home appliance (100) can perform the process of Fig. 3.

[0065]

[0066] FIG. 4 is a diagram showing the configuration of a server according to one embodiment of the present invention.

[0067] The communication unit (510) receives a first natural language command from the home appliance (100) and transmits a response message generated in response to the first natural language command to the home appliance.

[0068] The prompt generation unit (520) extracts first personal characteristic information or first personal preference information from the first natural language command and generates a prompt corresponding to the first natural language command based on the first personal characteristic information or first personal preference information. In addition, the prompt generation unit (520) can perform the task of storing or updating personal characteristic information or personal preference information in the database (550). The prompt generation unit (520) can store various prompts in a database and can automatically configure a pre-configured prompt that matches the topic type of the user's question.

[0069] The response generation unit (530) can generate a response message for the result produced after inputting the generated prompt into the generative AI module (300i, 300p). The response generation unit (530) can generate a response message with the voice of a specific character by applying personal preference information during the response message generation process.

[0070] The response generation unit (530) can store and retain various character models in the database (550) and can automatically set a character model that matches the topic type of the user's question. Alternatively, the prompt generation unit (520) can specify a specific character model within the prompt.

[0071] In another embodiment, the prompt generation unit (520) can generate a prompt to generate a response message with the voice of a specific character.

[0072] The database (550) stores personal characteristic information or personal preference information. This information is necessary later when the prompt generator (520) generates a prompt or the response generator (530) generates a response. More specifically, the prompt generator (520) can load information stored in the database (550) when generating a prompt suitable for the question topic and configuring and setting a character model. Additionally, the response generator (530) reflects the persona information stored in the database (550) so that the response message can be output in the voice of a specific character according to the character model.

[0073] Even if the natural language command to be processed does not include personal characteristic information or personal preference information, the prompt generation unit (520) can search for and load separate personal characteristic information or personal preference information stored in the database (550) and generate a prompt that includes it. The database (550) can accumulate and store various information entered by the user (1).

[0074] A generative AI module (300i) may be implemented within the server (500). Alternatively, a separate generative AI module (300p) may be deployed outside the server (500). The server (500), which collaborates with the generative AI module (300p) provided by an external partner server, inputs a prompt into the generative AI module (300p) according to a pre-agreed protocol. Then, the server (500) can receive the result corresponding to the input prompt from the generative AI module (300p).

[0075] The generative AI module (300i, 300p) is an example of GPT, and can generate a personalized response with content and style preferred by the user through a specific character model to be set in the question content, prompt, and response message transmitted via API.

[0076] The aforementioned components may be placed in the home appliance (100) in addition to the server (500).

[0077] A server (500) or a home appliance (100) may include an Automatic Speech Recognition (ASR) module (205) to process speech. Alternatively, the home appliance (100) may include an ASR module (205) and the home appliance (100) may transmit the result processed by the ASR module (205) (such as a text file or a voice file) to the server (500). The ASR module (205) may detect speech and acquire the user's voice data. Alternatively, the ASR module (205) may convert the acquired voice data into a text file.

[0078] Additionally, the home appliance (100) includes a prompt generation unit (520), a database (550), and a response generation unit (530), and a separate voice recognition device may include an ASR module (205) and perform voice recognition to detect the user's voice and acquire voice data.

[0079] That is, depending on the configuration of the server (500) and the home appliance (100), specific components may be included in the server (500) or the home appliance (100).

[0080] The server (500) or home appliance (100) may store personal characteristic information or personal preference information included in the text converted from the voice command, and after customizing the character and prompt to match the question topic, provide a prompt containing the question content to the generative AI module (300i, 300p) via API. Then, the server (500) or home appliance (100) generates a personalized response based on the result produced by the generative AI module (300i, 300p) and finally outputs the response to the user.

[0081] The configuration of Fig. 4 is a component of a server (500), but according to another embodiment of the present invention, a home appliance (100) may include the components of Fig. 4.

[0082]

[0083] FIG. 5 is a diagram showing the process of processing a user's natural language command according to one embodiment of the present invention. When voice is input, the home appliance (100) performs automatic speech recognition to generate text corresponding to the recognized voice and transmits it to the server (500) (S35, S36). According to another embodiment of the present invention, text generation may be performed at the server (500), in which case the home appliance (100) may convert the input voice into a voice file and then transmit the voice file to the server (500).

[0084] In this specification, voice files or text files converted from voice spoken by a user are all referred to as natural language commands. That is, a natural language command may be a voice file or a text file.

[0085] The prompt generation unit (520) of the server (500) can extract personal characteristic information and personal preference information from natural language commands (S37). The extracted personal characteristic information and personal preference information are stored in the database (550) along with the identification information of the home appliance (100) or the identification information of the user who owns the home appliance (100). Additionally, personal characteristic information and personal preference information not included in the natural language commands can be extracted from the database (550). For example, the prompt generation unit (520) can search the database (550) for and load personal characteristic information or personal preference information that the user (1) previously entered.

[0086] Personal characteristic information relates to the user's attributes and includes one or more of age, gender, region, and occupation. Personal preference information includes information such as fields or categories preferred by the user.

[0087] Additionally, personal preference information may include character information set as the output voice of the response message. This can be set by the user through the home appliance or by the user's personal preference information.

[0088] Accordingly, personal preference information may include either information about the character of the response message or persona information of the response message that is set or previously stored in the device (consumer appliance) that generated the collocation command.

[0089] The prompt generation unit (520) determines the type of natural language command (S38). For example, the prompt generation unit (520) determines whether the natural language command requests specific information (information category), or is a daily conversation (chat category), or requests control of a home appliance (control category), and determines the type of natural language command accordingly (S38).

[0090] And the prompt generation unit (520) generates a prompt using the type of natural language command, the natural language command, and the information extracted from S37 (or information extracted from the database, etc.) (S39).

[0091] The prompt generation unit (520) of the server (500) inputs the generated prompt into the generative AI module (300) (S40). The generative AI module (300) produces a result corresponding to the input prompt (S41). Then, the produced result is provided (S42). Subsequently, the response generation unit (530) of the server (500) generates a response message by reflecting personal characteristic information or personal preference information in the result provided by the generative AI module (300) (S43). Then, the communication unit (510) of the server (500) transmits the response message to the home appliance (100) (S44). The home appliance (100) outputs the response message (S45).

[0092] We will examine the embodiment of Fig. 5 in relation to movie recommendations.

[0093] When a user (1) says "recommend a movie," the home appliance (100) performs voice recognition and then transmits a natural language command to the server (500) (S35, S36). The server (500) can confirm "movie recommendation" in the natural language command. Additionally, the server (500) can use the identification information of the home appliance (100) that transmitted the natural language command to extract personal characteristic information from the database (550) such that the user (1) is male, is in their 30s, has a job as a teacher, and has an extroverted personality.

[0094] Additionally, the server (500) can extract personal preference information such as the user's (1) interests being "movies and science," the types of movies watched in the past being "Korean movies / action / actor_A," and the music listened to in the past being K-POP and ballads.

[0095] The prompt generation unit (520) of the server (500) can generate a prompt using a database in which personal characteristic information and personal preference information are stored.

[0096] Additionally, the prompt generation unit (520) can load a pre-secured prompt according to the type of natural language command. If the natural language command corresponds to the type of information request, the prompt generation unit (520) can check which topic the requested information corresponds to among topics such as "movies / music / news / science / finance / culture / education / law" and then load a prompt corresponding to that topic. Then, by applying the user's (1) personal characteristic information and personal preference information, the prompt generation unit (520) can generate a prompt corresponding to the received natural language command "recommend a movie" as PROMPT_INPUT.

[0097] PROMPT_INPUT = {"Male teacher in his 30s, Seoul appearance, Movie, + starring actor_A, prefer Korean action movies, recommend a movie"}

[0098] When such a prompt is input into the generative AI module (300), the generative AI module (300) can generate a result such as RESULT_OUTPUT.

[0099] RESULT_OUTPUT = {"I recommend the *Crime City* series as a Korean action movie starring Actor A. As Actor A's representative work, it is a series depicting the exploits of a homicide detective fighting crime. The thrilling action and exhilarating story will suit your taste well."}

[0100]

[0101] The server (500) can generate a response message by reflecting personal characteristic information or personal preference information in the RESULT_OUTPUT. For example, the server (500) can select a voice character (persona) suitable for the subject of movies. A voice character refers to various tones, voices, mannerisms, and styles such as a professor, scientist, engineer, critic, politician, entertainer, teacher, or lawyer. The server (500) can generate a voice message that outputs the RESULT_OUTPUT as voice in the voice of a specific character.

[0102] The server (500) transmits a voice message to the home appliance (100), and the home appliance (100) can output the voice message. If the server (500) generates a voice message using the voice of an actor named "Actor_A", the voice message output by the home appliance (100) is output in the voice of the actor, so that the user (1) can listen to information in the voice of the actor they prefer.

[0103] In the process of FIG. 5, the home appliance (100) can transmit a voice file, which is an example of a natural language command, to the server (500). In this case, the server (500) can analyze the voice file to determine the gender or age group of the user (1).

[0104]

[0105] When implementing the embodiment of FIG. 5, the server (500) stores personal characteristic information and personal preference information in a database (550) and can subsequently use them when generating a prompt or a response message. Additionally, the server (500) can obtain personal characteristic information and personal preference information through conversations or other commands performed with the user (1). Furthermore, the server (500) can load a prompt template suitable for the question.

[0106] When applying the embodiment of FIG. 5, even when different users ask the same question, the server (500) can provide a response, a response style, and information including the answer that is optimized for the user. To this end, the server (500) can include the user's question intent in the prompt during the process of generating the prompt, and can include personal characteristic information, personal preference information, persona information, etc. as input data in the prompt.

[0107]

[0108] By applying the embodiment of FIG. 5, a user-customized response can be generated when personal characteristic information and personal preference information are included in natural language commands through user utterance, or when personal characteristic information and personal preference information are included in the database. To this end, the server (500) can extract personal characteristic information and personal preference information from various conversation contents transmitted by the user.

[0109] For example, in the case of initial use, the server (500) can collect basic personal characteristic information and personal preference information through conversation with the generative AI module (300i, 300p). In addition, the server (500) can classify primary question types through user speech commands.

[0110] Classification of types can be performed directly by the server (500) or through a generative AI module (300i, 300p). Depending on the classified type, the server (500) can set a prompt suitable for the topic as a default template.

[0111] Since the basic template may require various input data, the server (500) can input personal characteristics / preference information that matches the question topic type into the basic prompt, and additionally include persona information (such as the voice of a specific character) in the prompt. Afterwards, the server (500), having obtained results through a generative AI module (300i, 300p) which is an example of GPT, can generate a personalized response.

[0112] Persona information includes information about a specific person's speech pattern or voice. Persona information can be used in the process of a generative AI module (300i, 300p) generating an answer to a question. Alternatively, the server (500) can use persona information in the process of generating a response message containing the result produced by the generative AI module (300i, 300p).

[0113]

[0114] FIG. 6 is a diagram showing the process of a server according to an embodiment of the present invention continuously storing personal characteristic information or personal preference information obtained from a user.

[0115] The server (500) collects conversation data containing questions, commands, etc. of the user (1) through one or more home appliances owned by the user (1) (S51). Then, the server (500) extracts personal characteristic information and personal preference information of the user from the conversation data (S52). Personal characteristic information includes the user's age, gender, and occupation as an example. Personal preference information includes the user's favorite field, character, movie genre, etc. as an example.

[0116] Personal information may include age, gender, occupation, personality, emotional state, region, interests, etc., and can be classified into personal characteristic information and personal preference information. However, such classification is made for the convenience of the embodiments and may be implemented as personal information in the actual implementation process.

[0117] Personal characteristic information is information that does not include different values, such as gender or age, or information with a limited scope of potential overlap, such as occupation (e.g., having a side job or changing jobs). Personal preference information is information that includes overlapping or different values ​​within the same category. For example, personal preference information may include multiple preferred fields within a movie category, such as science fiction movies and documentaries.

[0118] In one embodiment, when a user inputs a natural language command such as "recommend books for a male IT professional in his 30s to read," the server (500) extracts "30s," "male," and "IT profession" as personal characteristic information. It also extracts "books" as personal preference information.

[0119] Afterward, the server (500) compares the extracted personal characteristic information and personal preference information with the existing personal characteristic information and personal preference information stored in the database (S53). Then, if it is necessary to store the extracted data as a result of the comparison with the existing data, the extracted personal characteristic information and personal preference information is stored in the database (S54).

[0120] For example, if the user's age information stored by the server (500) in the past is 29 years old and the time of storage of such information is 2 years ago, the server (500) determines that the user has entered their 30s and updates the age information, which is personal characteristic information.

[0121] Additionally, if the user's hobby information that the server (500) previously stored is "movies," since new hobby information "books" has been extracted, the server (500) adds "books" to the user's personal preference information, which is hobby information.

[0122] Afterwards, the server (500) can delete the data if it is necessary to delete the previously stored data (S55).

[0123] In this way, if the text or voice file transmitted by the home appliance is checked and found to contain personal characteristic information or personal preference information, the information in the database (550) can be stored and continuously updated.

[0124] In addition, the server (500) can collect and analyze the content of questions by analyzing the user's voice data. Personal characteristic information is obtained through various methods, such as analyzing the voiceprint of the voice, and stored in the database (550).

[0125] When continuous conversation occurs between the user and the generative AI module (300), even if the conversation content includes personal characteristic information or personal preference information, it can be continuously stored and continuously updated in the database (550).

[0126] That is, the server (500) can estimate gender, age, personality, etc. through the analysis of the user's voice data, and can collect and obtain personal characteristic information or personal preference information through various channels, such as the user's voice recognition device account (ThinQ app account, etc.) and various SNS account information used through the home appliance (100).

[0127] The server (500) can collect personal characteristic information or personal preference information through the first question between the user (1) and the generative AI module (300). For example, after asking the question "Please tell me your gender, occupation, age, personality, and interests," the server (500) can store the response.

[0128] Table 1 below is a diagram showing the results of extracting personal preference information included in natural language commands entered by user (1).

[0129] Topic User Question (Command) Save personal preference information in database (personalized DB) Music Recommend "K-POP music", Recommend "Ballad music" Play song by "Singer_A" Topic "Music" K-POP (Korean Popular Music), Genre: Ballad, Singer: Singer_A Movie Recommend "Korean movies", Recommend "Action movies", Recommend movies where "Actor_A is the main character" Topic "Movie" Korean movies, Genre: Action, Lead: Actor_A News Find "Sports articles". Search "Baseball" news, Find "LG Twins" articles Topic "News" Sports, Baseball, LG Twins

[0130] As shown in Table 1, the server (500) stores and continuously updates personal preference information by subject category in the database when personal preference information is included in natural language commands.

[0131] Personal preference information may include various other types of information. For example, personal preference information may include one or more of field information that sets the scope of the response message, the length of the response message, or the level of information in the response message. Personal preference information may be set based on factors such as whether the user requires specialized information or a long, accurate response.

[0132]

[0133] Additionally, the server (500) may directly analyze natural language commands and input them into a generative AI module (300) capable of natural language understanding to analyze the meaning of the text and obtain the topic type, content, and intent of the question.

[0134] As a result, the server (500) can classify the topic type of the question into music, movies, news, etc. For example, if the natural language command is "recommend a movie," the server (500) determines the topic type of the question as "movies." For example, if the natural language command is "recommend music," the server (500) determines the topic type of the question as "music."

[0135]

[0136] FIG. 7 is a diagram showing the process of a server according to an embodiment of the present invention obtaining a prompt suitable for a natural language command and setting it as a template.

[0137] The server (500) obtains the subject type of the natural language command (S57). The server (500) loads a stored prompt corresponding to the subject type from the database (S58). Then, the server (500) sets the loaded prompt as a default template (S59).

[0138] For example, examples of prompt templates stored in the database (550) by question topic are as follows.

[0139] For example, if the question topic of a natural language command is "music," the server (500) can load a prompt from the database (550) containing content and examples that specify the generative AI module (300) to inform the user of the genre / composer / lyricist / singer of the music and the subject / overview / features of the music, and set it as a default template.

[0140] When the question topic of a natural language command is "movie", the server (500) can load a prompt from the database (550) containing content and examples that specify the generative AI module (300) to briefly inform the user of the movie genre / director / main actors / cast and the movie plot, and set it as a default template.

[0141] When the question topic of a natural language command is "news", the server (500) can load a prompt from the database (550) containing content and examples that specify the news media, title, reporter information, and a summary of the main content of the news to the generative AI module (300), and set it as a default template.

[0142] If a prompt suitable for the topic is set as the default template, the server (500) can obtain more accurate results from the generative AI module (300).

[0143]

[0144] FIG. 8 is a diagram showing the process of a server adding a user's personal preference information to a prompt according to an embodiment of the present invention. The server can add the personal preference information to the prompt generated in FIG. 7.

[0145] In one embodiment, as described in S57 in FIG. 7, the server (500) obtains the topic type of a natural language command (S57). Then, the server (500) loads the user's personal preference information related to that type from the database (S61). If the information that the user preferred according to the question topic type is stored in the database, the server (500) can load the information of that type. Additionally, the server (500) can load the voice character or persona information preferred by the user as personal preference information.

[0146] Afterward, the server (500) adds the loaded personal preference information to the prompt that was previously set as a template in S59 of FIG. 7 (S62).

[0147] Figure 8 is a process in which the server (500) retrieves personal preference information regarding the topic type of the question from the database (550) and additionally specifies it in the prompt.

[0148] In one embodiment, if the question topic is "music", the server (500) may specify in the prompt to recommend the latest songs among the ballad songs of K-POP, which is the genre preferred by the user, and the preferred singer A.

[0149] In another embodiment, when the question topic is "movie", the server (500) may specify in the prompt to recommend Korean movies, which are the genre preferred by the user, and action movies starring preferred actor A.

[0150] In another embodiment, if the question topic is "news", the server (500) may specify in the prompt to find baseball-related articles about the LG Twins, the preferred team, in sports articles, which is the user's preferred field.

[0151] In addition, the server (500) can set the persona as personal preference information with a voice, tone, manner of speech, and voice tone style that the individual prefers for the type of subject of the question.

[0152] For example, regarding a question topic related to music, the server (500) may specify in the prompt to guide and respond with the voice / tone / speech / voice tone / style of a personal favorite singer stored in the database.

[0153] In another embodiment, regarding a question topic related to movies, the server (500) may specify in the prompt to respond with the voice / tone / speech / voice tone / style of a personal favorite movie actor stored in the database.

[0154] In another embodiment, regarding a question topic related to news, the server (500) may specify in the prompt to respond to reading the news in the voice / tone / speech / voice tone / style of a personal preferred announcer stored in the database.

[0155] Meanwhile, in another embodiment of the present invention, persona-related information may generate a voice file using a specific character's sound source during the process in which the server (500) generates a voice response message from the result produced by the generative AI module (300).

[0156] When applying the above-described embodiment, the server (500) can retrieve personal characteristic information and personal preference information from the database (550) and then finally configure the prompt along with the question content.

[0157] That is, the server (500) can generate a prompt and character model to instruct the role and tasks of the generative AI module (300i, 300p) and a prompt including a setting example for it.

[0158] In this case, the natural language command actually entered by the user is a short sentence such as "recommend a movie," but the server (500) can load the user's personal characteristic information (e.g., the user is in their 30s, male, lives in Seoul, and is an extroverted teacher) from the database (500).

[0159] Additionally, the server (500) may use a prompt template that sets in detail what should be included in the answer to the prompt to be input into the generative AI module (300i, 300p). In this case, the prompt generated by the server (500) may include content instructing to "briefly introduce the movie genre / director / main actors / cast and the movie plot."

[0160] In addition, the server (500) can load the user's personal preference information from the database (550). For example, the server (500) can load information that the movies the user has watched in the past are Korean movies, and among them, the user prefers action movies starring actor_A. In this case, the server (500) can include in the prompt content that instructs the recommendation of movies starring actor_A, or requests the creation of a specific character style voice message that introduces the recommended movies in the style of actor_A's voice / tone / speech style / voice tone.

[0161] And the server (500) can provide a personalized response to the user by generating a prompt that includes the user's personal characteristic information, personal preference information, and persona information, and then inputting the prompt into a generative AI module (300i, 300p). In this case, the generated prompt, as seen in the example above, is the following PROMPT_INPUT as an example.

[0162] PROMPT_INPUT = {"Recommend movies that an extroverted male teacher in his 30s living in Seoul would like. Since he prefers Korean action movies starring Actor A, please recommend movies with this in mind. Briefly introduce the movie genre, director, lead actor, cast, and plot, and present the recommended movies using Actor A's voice and speaking style."}

[0163] As described above, the server (500) transmits a response message generated in the voice / tone / speech / voice tone style preferred by the user according to the configured persona to the home appliance (100), so that the home appliance (100) can output it. The persona configuration may be included in the user's personal preference information.

[0164] Persona settings can be configured according to personal preference styles based on the topic type of the user's question. Therefore, they may vary depending on whether the question is about music, movies, or news.

[0165] For example, if the type of question is music, the server (500) can generate a response message to guide and respond with the voice / tone / speech / voice tone / style of a personally preferred singer stored in the personal preference information.

[0166] For example, if the type of question is a movie, the server (500) can generate a response message to guide and respond with the voice / tone / speech / voice tone / style of a personally preferred movie actor stored in the personal preference information.

[0167] For example, if the type of question is news, the server (500) can generate a response message to read the news in the voice / tone / speech / voice tone / style of a personal preferred announcer stored in personal preference information.

[0168] The server (500) can store feedback received from the home appliance (100) after the home appliance (100) outputs a response message. Additionally, the server (500) can provide a conversation / response suitable for the user by collecting and analyzing the user's reactions / feedback while repeating the conversation / question / answer process with the user through the home appliance (100) and updating the prompt and persona information more suitable for the user.

[0169] FIG. 9 is a diagram showing the process of a server according to an embodiment of the present invention updating a user's personal characteristic information or personal preference information through feedback or subsequent conversation.

[0170] The server (500) receives a natural language command (including conversations, questions, or answers, etc.) entered by a user through the home appliance (100) (S65). In the process of obtaining the type of the received natural language command, if the natural language command contains new information that is incompatible with previously stored personal preference information or personal characteristic information, the new information is stored separately and a weight is set. Additionally, the server (500) lowers the weight of the incompatible personal preference information or personal characteristic information.

[0171] Afterwards, the server (500) uses additionally transmitted natural language commands (including conversations, questions, or answers, etc.) to adjust the weight of new information or the weight of previously stored information, and deletes information whose weight is below a standard (S67).

[0172] Incompatible information refers to a case where only one piece of information is valid, such as when the personal characteristic information was previously a male in his 30s residing in Seoul, but the newly entered information is a female in her 40s residing in Busan. In this case, since one piece of information must be deleted, the server (500) can set a weight for each piece of information and adjust the weight or delete unnecessary information based on the cumulative number of subsequent input pieces of information.

[0173] Processes S66 and S67 can also be applied to compatible information. Compatible information refers to cases where the movie preference was previously "Action" as personal preference information and subsequently entered as "SF". In this case, the server (500) can store both "Action" and "SF" as movie preferences. However, among the personal preference information stored once, the weight of information with lower importance can be lowered through feedback from the home appliance (100) and finally deleted.

[0174] For example, if "action" movies were initially personal preference information but at some point "action" movies were not watched at all or negative feedback was entered regarding the movie recommendation results for "action" movies, the server (500) can remove "action" movies from the movie preferences among the personal preference information.

[0175] When applying the embodiment of FIG. 9, the server (500) records and collects information related to personal characteristics, tastes, preferences, and interests while repeating the conversation / question / answer process with the user, and continuously updates the database (550). As a result, as the number of conversations with the user increases, the server (500) can provide more personalized conversations and responses.

[0176] In the embodiment of FIG. 9, when the communication unit (510) receives a new natural language command containing feedback on a response message, the prompt generation unit (520) can generate new personal characteristic information or new personal preference information corresponding to the feedback (i.e., obtainable through the feedback).

[0177] Additionally, the prompt generation unit (520) can determine the compatibility of the personal characteristic information stored in the database (550) and the new personal characteristic information.

[0178] Likewise, the prompt generation unit (520) can determine the compatibility of the personal preference information stored in the database (550) and the new personal preference information.

[0179] Based on the judgment result, the prompt generation unit (520) can increase the accuracy of personal characteristic information and personal preference information by adjusting the weights of the information stored in the database (550).

[0180] When applying an embodiment of the present invention, the server (500) can automatically configure an optimized prompt using personal characteristic information, personal preference information, and the content of a natural language command entered by the user, depending on the question topic.

[0181] In providing answers during this process, the server (500) can set a character model optimized for the style preferred by the user, thereby increasing the fun and intimacy of the conversation with the user and increasing user satisfaction. Additionally, the server (500) can increase response satisfaction by providing response information preferred by the user regarding the type and field of the question requested by the user.

[0182] The server (500) can provide more satisfactory conversations / responses by collecting and analyzing user reactions / feedback to response messages. Additionally, by continuously recording, collecting, and updating personal characteristic information or personal preference information in a database, the server can provide more personalized conversations / responses as the user (1) accumulates usage processes, such as entering various questions and checking responses.

[0183] A server (500) according to an embodiment of the present invention may include a database related to a customer's routine.

[0184] For example, LG Electronics’ generative AI technology “FURON” can be applied to ThinQ On. FURON can combine LG Electronics’ smart home platform LG ThinQ with OpenAI’s latest massive language model (LLM), GPT-4 Omni (4o). FURON can be combined not only with GPT-4 Omni but also with various LLMs including LG ExaOne. FURON enhances the spatial sensing and customer understanding capabilities of generative AI and provides spatial solutions optimized for each individual based on learning about the customer’s lifestyle. For example, if a user says, “I studied well last week, so please set it up the same way,” the server (500) can remember the settings from that time and set up a personalized environment for home appliances.

[0185] Although it has been described that all components constituting an embodiment of the present invention are combined or operate as a single unit, the present invention is not necessarily limited to such an embodiment, and within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Furthermore, while all components may each be implemented as a single independent piece of hardware, some or all of the components may be selectively combined to be implemented as a computer program having a program module that performs some or all of the combined functions on one or more pieces of hardware. The codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. An embodiment of the present invention may be implemented by storing such a computer-readable storage medium and reading and executing it by a computer. The storage medium for the computer program includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording element. Additionally, a computer program implementing an embodiment of the present invention includes a program module that is transmitted in real time through an external device.

[0186] Although the present invention has been described above with reference to embodiments, various changes and modifications can be made by those skilled in the art. Therefore, it should be understood that such changes and modifications are included within the scope of the present invention as long as they do not depart from the scope of the invention.

[0187]

Claims

1. A communication unit that receives a first natural language command from a home appliance and transmits a response message generated in response to the first natural language command to the home appliance; A prompt generation unit that extracts first personal characteristic information or first personal preference information from the first natural language command and generates a prompt corresponding to the first natural language command based on the first personal characteristic information or the first personal preference information; and A server that provides personalized information services using generative AI, comprising a response generation unit that generates a response message for a result after inputting the generated prompt into a generative AI module.

2. In Paragraph 1, A server providing personalized information services using generative AI, further comprising a database storing the first personal characteristic information or the first personal preference information extracted by the prompt generation unit.

3. In Paragraph 1, The above prompt generation unit is a server that provides personalized information services using generative AI, which generates a prompt corresponding to the above natural language command based on second personal characteristic information or second personal preference information stored in a database.

4. In Paragraph 1, A server that provides personalized information services using generative AI, wherein the above-mentioned first personal characteristic information includes one or more of age, gender, region, and occupation.

5. In Paragraph 1, A server providing personalized information services using generative AI, wherein the first personal preference information includes either information about the character of the response message or persona information of the response message that is set or previously stored in the device that generated the first natural language command.

6. In Paragraph 1, A server providing personalized information services using generative AI, wherein when the communication unit receives a second natural language command containing feedback on the above response message, the prompt generator generates second personal characteristic information or second personal preference information corresponding to the feedback.

7. In Paragraph 6, The prompt generation unit is a server that provides personalized information services using generative AI, which determines the compatibility of the first personal characteristic information and the second personal characteristic information or the compatibility of the second personal preference information and the second personal preference information, and adjusts the weights of the information stored in the database.

8. In Paragraph 1, A server providing personalized information services using generative AI, wherein the first personal preference information includes one or more of field information for setting the range of the response message, the length of the response message, or the information level of the response message.

9. A communication unit of a server receives a first natural language command from a home appliance and transmits a response message generated in response to the first natural language command to the home appliance; A step in which a prompt generation unit of the server extracts first personal characteristic information or first personal preference information from the first natural language command and generates a prompt corresponding to the first natural language command based on the first personal characteristic information or the first personal preference information; and A method for providing personalized information services using generative AI, comprising the step of a response generation unit of the above-mentioned server inputting the generated prompt into a generative AI module and then generating a response message for the calculated result.

10. In Paragraph 9, A method for providing personalized information services using generative AI, further comprising the step of the database of the server storing the first personal characteristic information or the first personal preference information extracted by the prompt generation unit.

11. In Paragraph 9, A method for providing personalized information services using generative AI, further comprising the step of the prompt generation unit generating a prompt corresponding to the natural language command based on second personal characteristic information or second personal preference information stored in the database of the server.

12. In Paragraph 9, A method for providing personalized information services using generative AI, wherein the first personal characteristic information includes one or more of age, gender, region, and occupation.

13. In Paragraph 9, A method for providing personalized information services using generative AI, wherein the first personal preference information includes either information about the character of the response message or persona information of the response message that is set or previously stored in the device that generated the first natural language command.

14. In Paragraph 9, The step of the communication unit receiving a second natural language command containing feedback on the response message; A method for providing personalized information services using generative AI, further comprising the step of the prompt generating unit generating second personal characteristic information or second personal preference information corresponding to the feedback.

15. In Paragraph 14, A method for providing personalized information services using generative AI, further comprising the step of the prompt generation unit determining the compatibility of the first personal characteristic information and the second personal characteristic information or the compatibility of the second personal preference information and the second personal preference information, and adjusting the weights of the information stored in the database of the server.

16. In Paragraph 9, A method for providing personalized information services using generative AI, wherein the first personal preference information includes one or more of field information for setting the range of the response message, the length of the response message, or the information level of the response message.

Images