A method for changing the features of home appliances, and a server and device for implementing this.

Abstract

This invention relates to a method for modifying the features of home appliances, and to a server and device that implement this method. [Solution] A method according to one embodiment of the present invention includes: a first step in which a server or home appliance calculates a first feature set and a first accuracy corresponding to a feature instruction word; a second step in which, if the first accuracy is below a standard, the server or home appliance calculates a second feature set and a second accuracy corresponding to the feature instruction word using a theme module located on the server or home appliance or an external server; a third step in which, if the second accuracy is below a standard, the server or home appliance calculates a third feature set using personalization information and feature instruction words corresponding to the device information of the home appliance; and a fourth step in which one of the feature sets of the first feature set, the second feature set, or the third feature set is changed to a feature of the home appliance.

Description

Technical Field

[0004] , , ,

[0006] , , , , , , , , , ,

[0005] ,

[0003]

[0001] The present invention relates to a method for changing features of home appliances, and a server and a device for implementing the same.

Background Art

[0002] The control method for a device such as a home appliance can be classified into a method of direct human operation control and a method of control via natural language commands. In the case of direct operation, the user can control the operation of the home appliance by operating a remote control, buttons, dials, etc. of the home appliance. In the case of control via natural language commands, when the user inputs a natural language command word for the home appliance into the home appliance, the home appliance recognizes and operates in this way.

[0003] However, since the natural language command words spoken or input by the user are based on different natural languages for each person, it is very difficult to interpret them and change them into actual commands. In particular, the accuracy of the interpretation of natural language command words is required depending on each user's past command word input habits and the usage environment of the home appliance.

[0004] ​​​​​​​​​​​​​​​​​

[0007] Furthermore, this specification aims to improve accuracy in changing the color, functions, and voice of home appliances by providing various recognition methods for user commands.

[0008] The objectives of the present invention are not limited to those mentioned above. Other objectives and advantages of the present invention not mentioned can be understood from the following description and more clearly from the embodiments of the present invention. Furthermore, it is readily apparent that the objectives and advantages of the present invention can be achieved by the means and combinations thereof described in the claims. [Means for solving the problem]

[0009] A method for changing the features of a home appliance according to one embodiment of the present invention includes: a first step in which a server or home appliance calculates a first feature set and a first accuracy corresponding to a feature instruction word; a second step in which, if the first accuracy is below a certain threshold, the server or home appliance calculates a second feature set and a second accuracy corresponding to the feature instruction word using a theme module located on the server or home appliance or an external server; a third step in which, if the second accuracy is below a certain threshold, the server or home appliance calculates a third feature set using personalization information and a feature instruction word corresponding to the appliance information of the home appliance; and a fourth step in which the feature set of the first feature set, the second feature set, or the third feature set is changed to a feature of the home appliance.

[0010] A server according to one embodiment of the present invention includes a preprocessing module that receives a feature instruction word input to a home appliance and calculates a first feature set and a first accuracy corresponding to the feature instruction word; a theme module that calculates a second feature set and a second accuracy corresponding to the feature instruction word if the first accuracy is below a certain threshold; and a personalization module that calculates a third feature set using personalization information and a feature instruction word corresponding to the device information of the home appliance if the second accuracy is below a certain threshold. The home appliance then sets one of the feature sets from the first feature set, the second feature set, or the third feature set as a feature of the home appliance.

[0011] A server according to another embodiment of the present invention includes: a preprocessing module that receives feature instruction words input to a home appliance and calculates a first feature set and a first accuracy corresponding to the feature instruction words; a counter theme module that, if the first accuracy is below a standard, transmits the feature instruction words to a theme module of an external server of a generative AI and receives a second feature set and a second accuracy from the theme module; and a personalization module that, if the second accuracy is below a standard, calculates a third feature set using personalization information and feature instruction words corresponding to the device information of the home appliance, and the home appliance changes any of the feature sets of the first feature set, the second feature set, or the third feature set to a feature of the home appliance.

[0012] An apparatus according to one embodiment of the present invention includes: an instruction input module into which a feature instruction is input; a preprocessing module that receives the feature instruction and calculates a first feature set and a first accuracy corresponding to the feature instruction; a theme module that calculates a second feature set and a second accuracy corresponding to the feature instruction if the first accuracy is below a certain threshold; a personalization module that calculates a third feature set using stored personalization information and the feature instruction if the second accuracy is below a certain threshold; and a feature setting module that sets any of the first, second, or third feature sets to a feature of the apparatus.

[0013] A method for changing the features of a home appliance according to another embodiment of the present invention includes one or more processors that process information received from a home appliance and input it into a theme module, which is a generative AI model, to obtain a result, the processors comprising the steps of: receiving a feature instruction from a first or second home appliance that instructs a feature change of the first home appliance; generating a prompt that includes the feature instruction and configuration information of the first home appliance corresponding to the feature instruction; inputting the prompt into the theme module to obtain a result; and converting the result into a first feature set to be set on the first home appliance.

[0014] A storage medium according to one embodiment of the present invention is installed in a server or home appliance and includes a preprocessing module and a personalization module, and includes either a theme module or a counter theme module. The preprocessing module calculates a first feature set and a first accuracy corresponding to a feature instruction word. The theme module calculates a second feature set and a second accuracy corresponding to a feature instruction word if the first accuracy is below a certain threshold. The counter theme module transmits the feature instruction word to the theme module of an external server of a generative AI and receives a second feature set and a second accuracy from the theme module if the first accuracy is below a certain threshold. The personalization module calculates a third feature set using personalization information corresponding to the device information of the home appliance and the feature instruction word if the second accuracy is below a certain threshold. The storage medium controls the home appliance to set the features of the home appliance. [Effects of the Invention]

[0015] When applying the present invention, it is possible to realize a method and apparatus for processing voice commands to change the features of home appliances, which accurately reflect the user's intent in processing the commands.

[0016] When applying the present invention, it is possible to provide personalized services that reflect the user's preferences when processing the user's commands.

[0017] When applying the present invention, various recognition methods for the user's command words can be provided to improve the accuracy of changing the color, function, and sound of home appliances.

[0018] The effects of the present invention are not limited to those described above; various other effects of the present invention in the configuration of the present invention can be easily conceived. [Brief explanation of the drawing]

[0019] [Figure 1]A diagram showing the process by which a home appliance changes features when a feature command word according to an embodiment of the present invention is input. [Figure 2] A diagram showing the process by which a home appliance transmits a feature command word to a server and then receives a feature set from the server to change the features of the home appliance when a feature command word according to another embodiment of the present invention is input. [Figure 3] A diagram showing the process by which a server or a home appliance calculates a feature set according to an embodiment of the present invention. [Figure 4] A diagram showing the process by which a server processes a feature command according to an embodiment of the present invention. [Figure 5] A diagram showing the process of processing a color command word according to an embodiment of the present invention. [Figure 6] A diagram showing the configuration of a personalization module according to an embodiment of the present invention. [Figure 7] A diagram showing the process by which a personalization module recommends using personalization information according to an embodiment of the present invention. [Figure 8] A diagram showing the process of calculating a feature set based on a theme module 320 according to an embodiment of the present invention. [Figure 9] A diagram showing the processing process when a fine adjustment to a feature set occurs according to an embodiment of the present invention. [Figure 10] A diagram showing the configuration in which all components in a home appliance are included according to an embodiment of the present invention. [Figure 11] A diagram showing that a preprocessing module, a theme module, a personalization module, and a database are included in a server according to another embodiment of the present invention. [Figure 12] A diagram showing that a preprocessing module, a personalization module, and a database are included in a server according to another embodiment of the present invention. [Figure 13] A diagram showing the configuration of a home appliance, a home appliance for voice processing, and a server according to an embodiment of the present invention. [Figure 14]This figure shows an interface according to one embodiment of the present invention that changes the color based on the command words entered by the user. [Figure 15] This figure shows the process of changing the hue of numerous home appliances according to one embodiment of the present invention. [Figure 16] This figure shows the process of changing the features of a home appliance according to one embodiment of the present invention. [Modes for carrying out the invention]

[0020] Hereinafter, with reference to the drawings, embodiments of the present invention will be described in detail so that those with ordinary skill in the art to which the present invention pertains can easily implement them. The present invention can be embodied in various different forms and is not limited to the embodiments described herein.

[0021] To clearly explain the present invention, parts unrelated to the description have been omitted, and identical or similar components throughout the specification will be given the same reference numerals. Furthermore, some embodiments of the present invention will be described in detail with reference to illustrative drawings. When assigning reference numerals to components in each drawing, the same components may, as far as possible, have the same reference numeral even if they are shown in other drawings. In addition, when describing the present invention, if it is determined that a specific description of a related known configuration or function would obscure the gist of the present invention, such detailed description may be omitted.

[0022] In describing the components of the present invention, terms such as First, Second, A, B, (a), (b), etc., may be used. These terms are used to distinguish a component from other components, and the terms do not limit the nature, order, sequence, or number of the components. When it is stated that a component is “linked,” “joined,” or “connected” to another component, it should be understood that the component may be directly linked or connected to the other component, but other components may be “interposed” between each component, or each component may be “linked,” “joined,” or “connected” through other components.

[0023] Furthermore, while the components of the present invention can be subdivided and described for the sake of explanation, these components may be implemented within a single device or module, or a single component may be implemented across multiple devices or modules.

[0024] In this specification, household appliances refer to devices including electronic products. Household appliances may be placed in homes, offices, etc., and may be moved by people and placed in other locations.

[0025] Furthermore, the home appliances described herein may include electronic devices such as user terminals like smartphones, tablets, and iPads (registered trademarks). Additionally, hubs (e.g., AI speakers) that communicate with and control various electronic devices based on wired or wireless communication may also be considered home appliances.

[0026] Furthermore, the devices described herein, such as home appliances, can have certain feature sets applied to modify some of their appearance or some of their operation.

[0027] Sub-concepts of a feature set include color sets, speech sets (related to speech patterns and voices), and function sets (related to the functions of home appliances).

[0028] A feature set may contain a bundle of information, such as each feature and its corresponding area or function. For example, a feature set can be structured as follows:

[0029] {{First feature information, second feature information, ..., nth feature information}, {1st application area, 2nd application area, ..., nth application area}}

[0030] Alternatively, the feature set can be structured as follows:

[0031] {{First Feature Information, First Application Area}, {Second Feature Information, Second Application Area}, ..., {nth feature information, nth application area}

[0032] If the application area is predetermined by the order of the feature information, then the application area information may be omitted in the feature set. For example, if the generative AI module is incorporated inside a home appliance, such as on-device, then the information about the home appliance, including the application area of ​​the home appliance equipped with the generative AI, is known, so the information about the application area may be omitted.

[0033] First, let's consider the color sets.

[0034] The home appliances described herein can change the hue of their appearance by a specific color set. A color set refers to one or more color pieces of information and a specific area of ​​the home appliance's appearance to which the hue indicated by the color piece information is applied.

[0035] As one embodiment, the color set has the following structure of color information applied to n areas of a home appliance. The color information and area information can be bundled separately.

[0036] {{First color information, first region}, {Second color information, second region}, ...{nth color information, nth region}}

[0037] For example, if the color set applied to a refrigerator is {{#093337, LEFT}, {#093537, RIGHT}}, the left door of the refrigerator may be assigned the RGB color value "093337", and the right door may be assigned the RGB color value "093537". The color set of the present invention can be applied not only to refrigerators, which have light source modules such as LEDs that allow for color differentiation of the refrigerator door panels, but also to various home appliances (air conditioners, stylers, digital picture frames, etc.) that have light source modules such as LEDs. Furthermore, by coordinating between home appliances, a similar or similar hue theme can be applied to all home appliances.

[0038] While RGB values ​​(hexadecimal) are one example of color information, the present invention is not limited to this. For example, a three-digit hexadecimal color information can indicate 16 levels for each of R, G, and B. Alternatively, a specific color name can correspond to color information. For example, color information such as "RED1", "RED2", ..., "ORANGE1", ..., "BLUE1" can also constitute a color set.

[0039] Next, let's consider the speech set.

[0040] The home appliances described herein allow the voice guidance system to change its tone, manner of speaking, gender, and pitch through a specific speech set. A speech set refers to information that indicates the features of the voice guidance system.

[0041] As one example, the speech set can be configured to include the tone, manner of speaking, gender, etc., of the voice comments provided by the home appliance, based on the comments output by the home appliance.

[0042] {{First speech information, first speech comment}, {Second speech information, second speech comment}, ... {nth speech information, nth speech comment}}

[0043] For example, if the speech set applied to an air conditioner has a first speech ment that instructs the air conditioner to turn on / off (on-off ment) and a second speech ment that informs the temperature and airflow direction of the air conditioner (operation ment), it can be set up as follows:

[0044] {{Iron Man, On-Offment}, {Elsa, Activation}}

[0045] In this case, when turning the air conditioner on or off, the user may be guided by Iron Man's voice, and when the air conditioner is operating, the user may be guided by Elsa's voice to announce the temperature or change the fan speed.

[0046] Next, let's consider the feature set.

[0047] The home appliances described herein can be controlled by a specific set of functions. A function set is the information necessary for a user to configure the functions of one or more appliances.

[0048] {{First function information, first function}, {Second function information, second function}, ... {nth function information, nth function}

[0049] For example, to configure the input source for a TV, you can define a set of functions as follows:

[0050] {{YouTube, YouTube link}, {Netflix®, Netflix Link} {Cinebeam, projector connection} {Moving to a channel for pleasant content, a channel for pet animals}

[0051] When a user specifies a particular function, enters it as text, or selects a function displayed on the TV, the home appliance can provide that function.

[0052] In the various feature set configurations described above, users must input voice or text information into home appliances to configure a particular feature set, but this information may fall outside the scope of predefined and formatted information provided by the appliance. For example, when a user inputs voice or text commands to configure a specific function of an appliance, that input often falls outside the scope of standardized commands that the appliance is pre-configured to recognize. For instance, there may be standardized, intended, pre-programmed, or default voice commands for configuring a specific function (e.g., color, sound, or one or more functions), and users can use such standardized information, but they may also use information that goes beyond or differs from the standardized information. For example, a user may provide voice commands using words that differ from the standardized information.

[0053] In other words, it is necessary to configure the feature set so that the user's intent is reflected even when the user inputs voice / text in various forms.

[0054] Therefore, we will consider the process of combining the output of generative AI with the user's personal information in order to analyze the voice input by the user and provide a suitable feature set.

[0055] The feature commands described herein are commands used to specify or modify features, either by voice or text. However, for the sake of clarity, the explanation will focus primarily on voice commands.

[0056] The processing of feature instructions in this specification can be performed by either consumer electronics or server devices. Examples of feature instructions include color instructions, speech instructions, and function instructions. Consumer electronics or server devices (servers) or a system consisting of one or more devices can store hardware or software for processing feature instructions. Alternatively, consumer electronics or server devices (servers) or a system consisting of one or more devices can have software transmitted and executed from a remote third device for processing feature instructions. Processing feature instructions using hardware built into the device, software stored within the device, or software that can be transmitted and executed can be performed by a processor within each device. Alternatively, the hardware or software itself can act as a processor.

[0057] The language model described herein is an LLM (Large Language Model) as one embodiment, and the language model may be incorporated into a consumer electronics device or a server device. Alternatively, the language model may be incorporated into an external device, in which case the consumer electronics device or server device can obtain the results of the language model via the external device.

[0058] Personalized information as described herein includes information generated during the use of home appliances. Personalized information according to one embodiment of the present invention also includes information generated during the use of other home appliances used by the user of the said home appliance. Personalized information according to another embodiment of the present invention includes information generated during the use of other home appliances located in the same space as the said home appliance.

[0059] For example, the personalization information applied in the process of changing the hue of a refrigerator may include information generated during the use of the air conditioner in the space where the refrigerator is located. Similarly, the personalization information applied in the process of changing the hue of an air conditioner may include information generated during the use of a refrigerator used by the user of the air conditioner, or an air conditioner that the user has used in the past.

[0060] Furthermore, the personal information applied in the process of changing the voice guidance of a refrigerator includes information generated during the use of the air conditioner in the space where the refrigerator is located. Similarly, the personal information applied in the process of changing the voice guidance of an air conditioner includes information generated during the use of a refrigerator used by the user of the air conditioner or an air conditioner previously used by the user.

[0061] Similarly, if there are two air conditioners in a household, the personal information applied in the process of changing the function of the first air conditioner includes information generated during the use of the second air conditioner in the space where the first air conditioner is located. Similarly, the personal information applied in the process of changing the function of the second air conditioner includes information generated during the use of a refrigerator or the first air conditioner by the user of the first air conditioner.

[0062] Furthermore, a voice or text command can be directly input to a first home appliance that has been instructed to change a feature in the first or second home appliance, and this command can be processed or transmitted to a server. Alternatively, a voice or text command can be directly input to a separate second home appliance in addition to the first home appliance that has been instructed to change a feature, and this command can be processed or transmitted to a server. Home appliances can be understood as any (household) home appliance that includes one or more light source modules that perform one or more functions / operations or generate / produce light of one or more colors and / or variable brightness to illuminate one or more parts of the home appliance, or one or more sound source modules that generate / produce sound for making announcements (e.g., one or more types of voice and / or volume levels), or any combination thereof (i.e., it performs one or more functions / operations and may include one or both of a light source module and a sound source module).

[0063] Figure 1 shows the process by which a home appliance changes a feature when a feature instruction word according to one embodiment of the present invention is input.

[0064] User 1 inputs a predetermined command word (voice or text) into the home appliance 100 (S3). Alternatively, the user can input the predetermined command word via a hub that connects to and controls multiple electronic devices, such as a mobile terminal or AI speaker, in addition to the home appliance. The command word can be input by voice (VOICE) or by text. The home appliance (device) 100 converts the input feature command word into text and determines the feature set corresponding to the command word based on the pre-processed results (S5). The home appliance 100 can perform a feature set calculation process of up to three stages to determine the feature set. Once the feature set is determined through the feature set calculation process, the home appliance 100 sets the determined feature set in the home appliance 100 (S7).

[0065] Figure 2 shows the process by which, when a feature instruction word according to another embodiment of the present invention is input, the home appliance transmits the feature instruction word to the server, receives a feature set from the server, and then modifies the features of the home appliance.

[0066] For an explanation of S3, please refer to the content in Figure 1 mentioned above. The home appliance 100 transmits the input feature instruction word to the server 500 (S11). At this time, if the input instruction word is a voice command, the home appliance 100 can convert the voice command to text and transmit the feature instruction word, which is the result of pre-processing the text, to the server 500.

[0067] In this case, the server 500 can determine the feature set corresponding to the received feature instruction (S15). The preprocessing process can also be performed by the server 500. In this case, the home appliance 100 can transmit the input instruction directly to the server 500. For example, the home appliance sends a user command to the server, the server interprets the command to understand the user's intent, and determines the specific action or change in the function set that the home appliance should perform. The home appliance executes the command determined by the server, but the embodiments are not limited to this. In another embodiment, the decision process for determining the user's intent and generating the corresponding control command or change in the function set is performed locally by the home appliance 100.

[0068] To determine the feature set, the server 500 can perform a feature set calculation process with up to three stages. Once the feature set is determined through the feature set calculation process, the server 500 transmits information about the determined feature set to the home appliance 100 (S16), and the home appliance 100 sets the received feature set in the home appliance 100 (S17).

[0069] As discussed in Figures 1 and 2, the server 500 or the home appliance 100 can calculate a suitable feature set from the pre-processed feature instruction words. Therefore, we will examine the feature set calculation process performed by the server 500 or the home appliance 100.

[0070] Figure 3 shows the process by which a server or home appliance calculates a feature set according to one embodiment of the present invention. The process in Figure 3 can be performed by either the server 500 or the home appliance 100.

[0071] The server 500 or the home appliance 100 calculates the first feature set and the first accuracy corresponding to the feature instruction (S21). The first feature set corresponding to the feature instruction means the feature set in which the feature instruction corresponds to the name of a specific feature set or to the name of a feature.

[0072] In other words, a feature set includes one or more feature information and information about the area or function of the device to which the feature information is set. If the feature set is a color set, the color information and the area / function to which the color is applied can constitute the feature set. If the feature set is a speech set, the speech information and the area / function to which the speech is applied can constitute the feature set. If the feature set is a function set, the function information and the circumstances under which the function is applied can constitute the feature set.

[0073] For example, if a color set is one example of a feature set, refer to Table 1.

[0074] [Table 1]

[0075] In Table 1, if the color command spoken by the user includes the hue name of a color that constitutes the first color set, this means that the user has accurately spoken a specific feature set (color set), and therefore the server 500 or home appliance 100 can extract the color set corresponding to the color command. As a result, at stage S31, the server 500 or home appliance 100 determines that the result corresponding to the color command and the accuracy of that result (e.g., first accuracy) meet the criteria.

[0076] For example, if the server 500 or home appliance 100 has stored the contents of Table 1, and the user inputs a slightly modified color command such as "mandarin orange," "green," or "turquoise," the server 500 or home appliance 100 can compare the similarity between the color command and the names in Table 1 and calculate an accuracy of 90 or higher. Assuming the first accuracy criterion is 85, the server 500 or home appliance 100 can extract the color set corresponding to the color command from the stored color sets in Table 1.

[0077] For other examples, such as a speech set as one example of a feature set, please refer to Table 2.

[0078] [Table 2]

[0079] For other examples, please refer to Table 3 if the feature set is a functional set as one example of a feature set.

[0080] [Table 3]

[0081] In addition to the embodiments described above, the following set of functions can be configured.

[0082] As one example of a function set applied to a refrigerator, the name "Summer_Humid_Refrigerator" may be set to {Refrigerator_Temperature1, Freezer_Temperature2}.

[0083] As another example of a function set applied to a refrigerator, the name "Autumn_Optimal_Temperature_Refrigerator" may be set to {Refrigerator_Temperature3, Freezer_Temperature4}.

[0084] As one example of a function set applied to an air conditioner, {Air conditioner_temperature1, wind speed_1, time information_1} may be set to the name "Cool down before going to bed".

[0085] Another example of a function set applied to an air conditioner is that {Air Conditioner Temperature 2, Fan Speed ​​2, Time Information 2} may be set to the name "Wake Up in the Morning".

[0086] In Table 1, Color1, 2, ... etc., refer to specific color hues, for example, RGB values ​​are used as one example. Region1, 2, ... etc., refer to information about the regions to which Color1, Color2, etc., are applied. For example, a color set {{Color1, Color2, Color3, Color4}, {Region1, Region2, Region3, Region4}} indicates that Color1 is applied to Region1, Color2 to Region2, Color3 to Region3, and Color4 to Region4. If the order of the color information constituting the color set is applied to the order of regions, then information about the regions may be omitted. At stage S21, the first accuracy may be calculated based on the degree of correspondence between the feature instruction and the feature set / feature information.

[0087] For example, in the case of a color set, the first accuracy may be calculated based on the degree of correspondence between the color command and the color set / color information. If the color command is "set the hue of the refrigerator to orange" (Case 1) and if it is "set the hue of the refrigerator to mandarin orange" (Case 2), the server 500 or appliance 100 can similarly calculate a color set that is an orange-based theme hue for both color commands.

[0088] However, in Case 1, the color command and name are identical, whereas in Case 2, the color command "mandarin orange" is not exactly the same as the name in Table 1. Therefore, although the server 500 or the home appliance 100 calculates the first color set as the result value for the two color commands, the accuracy can be calculated differently. For example, it can be calculated as 100 in Case 1 and 80 in Case 2.

[0089] Similarly, in the case of speech sets, the first accuracy may be calculated based on the degree of correspondence between the speech command and the speech set / speech information. If the speech command is "Make the air conditioner's voice an adult woman's voice" (Case 1) and if it is "Make the air conditioner's voice a girl's voice" (Case 2), the server 500 or the home appliance 100 can calculate different speech sets for both speech commands.

[0090] In other words, in Case 1, the speech imperative and the name of the speech information are the same, but in Case 2, because the word is "girl," two cases can be calculated: "adult woman" and "boy," resulting in lower accuracy. For example, the calculation might be 100 in Case 1 and 80 in Case 2.

[0091] Similar to color / speech sets, the accuracy of function sets is calculated differently depending on whether the user accurately pronounces the corresponding voice or not. For example, this method could include an initial step of attempting to understand the user's function command by matching it against a library of predefined function sets. These function sets could be pre-packaged or predefined settings for various aspects of a device. For instance, a color function set called "orange" might contain a coordinated group of specific colors, defining which part of the device each color applies to. Similarly, a voice function set could define "adult female voice" for all voice prompts on an air conditioner. Another function set could create a "summer heat" preset that automatically adjusts temperature and fan speed to a specific level. Furthermore, in this example, upon receiving a command, an initial accuracy score can be calculated based on how well the user's words match the names in the predefined function set. A direct command like "set orange" might score 100%, while a similar but less accurate command like "set tangerine" would also be recognized, generating a high accuracy score, though slightly lower. However, if the command is ambiguous or does not exactly match any existing options, the accuracy score will be low. If this score falls below a pre-set threshold, it may indicate that the system is not confident in selecting a predefined setting. In such situations, the process is escalated by sending the command to a theme module, which can use a generative AI model to create a new custom set of features that meets the user's specific requirements.

[0092] Therefore, if the first accuracy is below a certain threshold, the server 500 or the home appliance 100 uses a theme module (e.g., a language model) to calculate a second feature set and a second accuracy corresponding to the feature instruction word (S31). The theme module may be located in the server 500 or the home appliance 100 or in an external language model server. Refer to Figure 4.

[0093] A first-order accuracy score below the threshold occurs when it is not possible to calculate a specific feature set within a predefined name-feature set, as shown in Tables 1, 2, and 3. In other words, if the initial confidence score (e.g., first precision) is too low, it may be because the user's command does not strictly match either a pre-programmed setting or a predetermined feature set. For example, the method and system may determine that none of the available presets adequately meet the user's requirements and that a more advanced process is needed to understand the user's requirements.

[0094] For example, in Table 1 and the examples of color commands, a user can input the command "Change the refrigerator's color to Iron Man." To construct a feature set for this, information about the hue represented by the character "Iron Man" is needed, but the server 500 or the home appliance 100 cannot calculate the result corresponding to the color command using the information in Table 1.

[0095] Therefore, in this case, the first accuracy is calculated to be 0 or a very low value, and in S31, the server 500 or home appliance 100 inputs a feature instruction (e.g., a color instruction) into a language model, e.g., a Large Language Model (LLM), and calculates the corresponding second feature set (e.g., a second color set) and the second accuracy (S31).

[0096] The second accuracy can be calculated for theme module 320, with LLM as one example. In this case, the second accuracy includes examples that include two types of results, such as success or failure, or three types of results, such as success, neutral, and failure.

[0097] The first accuracy may be a value used to determine whether the actual command word corresponds to the content stored in the database.

[0098] In the process of processing data by a theme module 320, which uses LLM as one embodiment, a result with a second accuracy of 0 (e.g., a failure result) may be calculated for a range of features where data is not output or cannot be applied to the device (e.g., in the case of a refrigerator, when an unoutputtable color is output). If LLM calculates an unoutputtable or unapplicable result in the process of processing prompts, this corresponds to the process of calculating the second accuracy in order not to use that result.

[0099] If the prompt instructs the LLM to output two values ​​for the accuracy of the result, either success or failure, the second accuracy may be output as one of the values ​​0 and 100 (or any number such as 1).

[0100] If the prompt instructs the LLM to output one of three accuracy values ​​for the result: success, neutral, or failure, the second accuracy may be output as one of the following values: 0 and 50 (or a number such as 0.5), or 100 (or a number such as 1).

[0101] Alternatively, a server 500 or home appliance 100, which is not a theme module, can use the results calculated by the theme module to determine the accuracy (success / failure).

[0102] Furthermore, in Table 2 and the examples of speech commands, the user can input the command, "Change the air conditioner's voice to the voice of an elderly professor." In order to construct the feature set for this, information about the voice represented by the character "elderly professor" is required, but the server 500 or the home appliance 100 cannot calculate the result corresponding to the speech command using the information in Table 2.

[0103] Therefore, in this case, the first accuracy is calculated to be 0 or a very low value, and in S31, the server 500 or home appliance 100 inputs feature commands (e.g., speech commands) into a language model, e.g., a Large Language Model (LLM), and calculates the corresponding second feature set (e.g., a second speech set) and the second accuracy (S31).

[0104] In the case of function sets, as with color sets and speech sets, if the user does not pronounce the sound corresponding to the name accurately, the accuracy will be calculated below the standard.

[0105] Here, the second accuracy score is one example of the accuracy information calculated by the LLM. For example, when inputting a prompt into the LLM and calculating the result, the prompt can instruct the system to also calculate the accuracy of the information it calculated. For example, if a user issues a creative or abstract command such as "change the refrigerator's color to Iron Man," the system may not be able to find a matching preset in the library, resulting in a very low initial accuracy score. This could be a trigger for using a theme module. This command is passed to a generating AI module, which can use its knowledge to interpret the request. The AI ​​module can understand the "Iron Man" theme and generate a completely new color set, such as a combination of red and gold, and how to apply it to the various lighting zones of the refrigerator. A similar process can be applied to a voice command such as "change the voice to the old professor's voice." This method may also involve generating a second accuracy score for the newly generated color set or feature set. This second accuracy score acts as a kind of confidence check, indicating whether the generated result is successful and applicable to the device. For example, if the AI ​​module successfully creates a valid color scheme, it can return a "success" score. On the other hand, if a command is unexecutable or generates a color that the device cannot display, a "failure" score may be returned. This verification step ensures that only logical and functional feature sets are applied, helping to prevent errors that may occur due to abstract or imaginative user commands.

[0106] The language model used in this invention includes a language model such as chatGPT as an example of an LLM. Alternatively, the language model used in this invention may be a language model specifically designed for generating feature sets. The language model may be implemented and embedded within the server device 500 or the home appliance 100. Alternatively, the language model may be configured to be implemented outside the server device 500 or the home appliance 100, and results can be calculated via prompt input.

[0107] When simply applying command words to a generative AI such as LLM without prompting, the desired output value may not be obtained depending on the number of appliance panels, the type of appliance, etc. Therefore, an embodiment of the present invention provides prompts to a generative AI such as LLM, and allows the prompts to include various types of information.

[0108] The various types of data that a prompt can contain, that is, the data included in a prompt, may include identification information about home appliances owned by the user, and configuration information about home appliances owned by the user. For example, identification information about home appliances refers to devices actually owned by the user, such as home appliances registered on the server or home appliances registered in the user account on the server.

[0109] Furthermore, to set an accurate feature set, important information specific to each appliance may be included in the prompts. The input data required to calculate the function set for the operation of the refrigerator may include the command words entered by the user, information about the refrigerator (the refrigerator model owned by the user, the installation location of the refrigerator, whether there is a refrigerator / freezer compartment, the number of refrigerator doors, etc.), and the current status of the refrigerator (power on / off status, currently set storage temperature, type of stored items, when the door was opened or closed in the past day, etc.). As a result, the temperature of the refrigerator can be controlled more accurately.

[0110] The necessary input data for calculating the function set for air conditioner operation may include user-entered commands, information about the air conditioner (user-owned air conditioner, installation location), and the air conditioner's current status (power on / off status, currently set temperature and airflow direction, information on air conditioner operations performed over the past day, etc.). As a result, the airflow and airflow direction of the air conditioner can be controlled more accurately.

[0111] On the other hand, detailed information about home appliances, information about their components, and status information can also be transmitted by the home appliances to the server, and after the home appliances store this information, it can be provided to the pre-processing module 310 or the theme module 320. In addition, home appliances can confirm the recognition of command words and transmit recognized command words between home appliances. For example, if a command word related to the refrigerator is input to the oven, the oven can transmit the command word input to the refrigerator.

[0112] In other words, the server or home appliance can cause the theme module 320 to input information such as the type of home appliance targeted by the feature instruction, the configuration information or state information of the home appliance, the range of features that can be set on the home appliance, and information about sample control instructions. As a result, the theme module 320 can calculate results that are suitable for multiple characteristics of the home appliance.

[0113] Personalization module 330, which is one example of a recommendation-type AI module, can store personal information such as the user's usage history of home appliances, and can calculate a personalized feature set using the stored personal information.

[0114] In process S31, if the second accuracy is below the standard, the server 500 or the home appliance 100 calculates a third feature set using the personalization information and feature instruction words corresponding to the home appliance's device information (S41). The device information may include at least one of the following: the identification information of the home appliance whose features are to be changed and the type of appliance (refrigerator, air conditioner, etc.). Personalization information may include, as one example, information about the feature set that was set on the home appliance. Alternatively, information calculated in the course of the user's use of other home appliances may also be included in the personalization information.

[0115] Next, the server 500 or the home appliance 100 changes one of the feature sets, either the first, second, or third feature set, to a feature of the home appliance (S51). As a result, the appearance color of the home appliance can be changed, for example, according to the user's intent. For example, this method may include applying a successfully generated feature set (e.g., a predefined one, one generated by AI, or a personalized option) to the home appliance. This includes directly changing the characteristics of the device, such as changing the appearance color of the device to match the user's original command and intent.

[0116] In one embodiment, a server 500 or a home appliance 100 can use a first accuracy and a second accuracy to determine whether to use a first or second feature set. If both the first and second accuracys are low, a third feature set can be used. For example, the system compares the confidence scores of the first two results (e.g., results from a predefined library and results from an AI model) to determine which is a better fit to the user's command. If neither result has a high confidence score, the system uses a third personalized feature set.

[0117] In another embodiment, the server 500 or home appliance 100 can, after performing S21, S31, and S41, select a feature set from the three calculated feature sets that is similar to a feature set previously used by the user.

[0118] When applying an embodiment of the color set of the present invention, in a home appliance capable of changing its appearance to various hues, the server 500 or the home appliance 100 first searches for a hue of a basic theme in response to a voice or text command input by the user (S21). If there is no hue that matches precisely, a color set (color information) can be obtained via a language model, such as generative AI (Artificial Intelligence) (S31). Information that may be included in the color set is output brightness change information in addition to RGB color information. For example, in addition to color changes, brightness changes may also be included in the color set.

[0119] However, since the language model and generative AI do not have the user's past hue results, if the language model (generative AI) cannot provide a color value in response to the user's voice / text command, the server 500 or the home appliance 100 can determine the AI ​​color via a recommendation AI server that includes and reflects the user's past color usage pattern data (S41). The recommendation AI server may be embedded in the home appliance and perform data processing there, or the data may be computed or performed in various environments such as a cloud server-based system, an on-device system, or a distributed server system (multiple servers).

[0120] When applying the embodiment shown in Figure 3, if a voice / text command is recognized to change the color theme of a home appliance that can change its appearance color, such as one fitted with an LED panel (e.g., a mood-enhancing refrigerator or air conditioner), even if the color theme included in the input command is not predefined, a generative AI, including a large language model such as ChatGPT, can be used to generate a color theme that matches the user's intent and change the color of the refrigerator.

[0121] Furthermore, when utilizing generative AI, in order to produce a feature set that matches the user's intent, data to be input to the generative AI (for example, prompts) can be generated or input to the generative AI, allowing for the accurate output of a feature set suitable for each situation in a short time or with fewer processes.

[0122] As one embodiment, home appliances and server devices can input information about the home appliance as additional input data to a generative AI in addition to the command words entered by the user, enabling the calculation of results that are suitable for the user's intentions quickly and with minimal computation. For this reason, the information about the home appliance that may be included in the additional input data includes information about feature command words. The additional input data may include information relating to the appearance of the home appliance, such as the arrangement of the display panel, the range of colors that the panel can generate, or the type of home appliance. The information that can be added may vary depending on the type of feature indicated by the feature command word. Home appliances and server devices can store additional input data or categories of additional input data corresponding to each type of feature and update them to reflect the results of the generative AI.

[0123] Because it combines a generative AI server or a recommendation AI server with an existing natural language processing (NLP) server for speech recognition, it can handle aspects that were impossible for existing natural language processing (NLP) servers alone, such as understanding the user's intent and generating color themes.

[0124] In Figure 3, the reference value compared to the accuracy can be set in various ways. In this specification, the accuracy can be compared to the reference value, such as being below or above the reference value, and if it does not meet these reference values, a process of acquiring a new feature set can be performed to improve the accuracy of the feature set.

[0125] In other words, it can handle color themes that are not predefined and user utterances, thus satisfying a wide range of user utterances and requests related to changing the color of home appliances such as refrigerators and air conditioners. For example, if a user inputs a command such as "I'm feeling down right now, so change the color" or "Recommend a color that suits me," the server 500 or the home appliance 100 can generate the optimal color for the user's command and apply the hue to the home appliance, such as a refrigerator or air conditioner.

[0126] The process shown in Figure 3 can also be applied to speech sets or function sets, similar to how it is applied to color sets.

[0127] When applying an embodiment of the speech set of the present invention, in a home appliance capable of changing voice guidance using various voice utterance characters, the server 500 or home appliance 100 first searches for a speech on a basic theme based on the voice or text command words input by the user (S21). If there is no speech that matches precisely, a speech set (speech information) can be secured via a language model, for example, generative AI (Artificial Intelligence) (S31).

[0128] However, if the language model (generative AI) does not have the speech results that the user has previously preferred, and therefore cannot provide specific speech information in response to the user's voice / text command, the server 500 or home appliance 100 can determine the AI ​​speech information via a recommendation AI that reflects the user's past speech selection patterns (S41).

[0129] When applying the embodiment shown in Figure 3, in a home appliance where the speech characteristics of the home appliance's voice guidance can be changed, such as by attaching an LED panel, if a voice / text command to change the speech theme is recognized, even if the speech theme included in the input command is not predefined, generative AI can be used to generate a speech theme that matches the user's intent, thereby changing the tone and wording of the home appliance's voice guidance.

[0130] By combining a generative AI server and a recommendation AI server with the existing natural language processing (NLP) server for speech recognition, it can handle aspects that were impossible for the existing natural language processing (NLP) server alone, such as understanding the user's intent and generating speech themes.

[0131] In other words, it can handle speech themes and user utterances that are not predefined, thus satisfying a variety of user utterances and requests related to changing the guidance messages of home appliances such as refrigerators and air conditioners. For example, if a user inputs a command such as "I'm feeling down right now, so change to a cheerful voice message" or "Recommend a voice message that matches the current music," the server 500 or the home appliance 100 can generate optimal speech information for the user's command and apply it to the guidance messages of the home appliances such as refrigerators and air conditioners.

[0132] Taking a refrigerator as one example, when a functional command such as "There are many new food items in the refrigerator, so please refrigerate them" is input, the server 500 or home appliance 100 can generate a function set containing optimal functional information for the user's command and apply it to the function settings of home appliances such as refrigerators and air conditioners. For example, by integrating generation AI and speech recognition, the present invention evolves beyond simple keyword matching to a stage where it can understand the user's intent more deeply. This makes it possible to appropriately interpret and execute abstract commands or undefined commands that cannot be handled by existing systems. In this way, the present invention realizes a more flexible and intuitive user experience by dynamically generating and applying the optimal feature set in response to diverse user inputs.

[0133] Figure 4 shows the process by which a server according to one embodiment of the present invention processes a feature instruction. A single server can perform the functions described later, or a group of servers can perform the functions described later. The embodiment in Figure 4 can be applied to any of the color instructions, speech instructions, or function instructions included in a feature instruction.

[0134] Therefore, according to one embodiment of the present invention, one server may include each of the modules. According to another embodiment of the present invention, each of the modules described later may become a server, and these servers may be included in a group of servers.

[0135] According to yet another embodiment of the present invention, two or more modules can constitute a single server.

[0136] The server's preprocessing module 310 receives voice / text instruction words as input (S21a). The input instruction words are then preprocessed to convert them into feature instruction words, and the corresponding first feature set and first accuracy are calculated (S21b).

[0137] The preprocessing module 310 uses an existing (legacy) NLP (Natural Language Processing) module as one embodiment to perform primary intent analysis on the voice or text command input by the user. If successful, it can control the features of the home appliance in the same way as the existing method.

[0138] For example, if the input color command is "Change the color to the spring theme," the preprocessing module 310 preprocesses the input command and extracts the color command "spring theme." If a color set corresponding to "spring theme" is already stored in the preprocessing module 310, the preprocessing module 310 calculates the color set and its accuracy.

[0139] If the input command is "Change the color to the Spring Breeze theme," the preprocessing module 310 preprocesses the input command and extracts the color command "Spring Breeze theme." However, unlike the previous embodiment, there is no color set corresponding to "Spring Breeze theme," so the preprocessing module 310 can calculate the closest color set corresponding to "Spring theme" and its accuracy (for example, 60).

[0140] For example, if the input speech command is "Change the voice to a refreshing voice," the preprocessing module 310 preprocesses the input command and extracts the speech command "refreshing theme." If a speech set corresponding to "refreshing theme" is already stored in the preprocessing module 310, the preprocessing module 310 calculates the speech set and its accuracy.

[0141] If the input command is "Change the speech to a refreshing and cheerful theme," the preprocessing module 310 preprocesses the input command and extracts the speech command "refreshing and cheerful theme." However, unlike the previous embodiment, there is no speech set corresponding to "refreshing and cheerful theme," so the preprocessing module 310 can calculate the accuracy (for example, 60) of the speech set that is closest to "refreshing theme."

[0142] If the accuracy of the feature set calculated by the preprocessing module 310 exceeds the standard, this is considered a success, and the feature setting module 340 can apply the feature set calculated in S21b to the home appliances 100a, 100b, and 100c (S51a, S51b, S51c).

[0143] On the other hand, if the accuracy of the feature set calculated by the preprocessing module 310 is below the standard, this corresponds to a failure in accuracy, and the theme module 320 can calculate the feature set and accuracy corresponding to the feature instruction word (S31b).

[0144] In other words, if the analysis of the user's intent to change a feature fails via the NLP module (or NLP server), which is one embodiment of the preprocessing module 310, the theme module 320, which is one embodiment of an LLM (Large Language Model) server, can process the user's feature command (e.g., a color command), perform a secondary intent analysis, and extract the intent analysis and the theme color Hex value.

[0145] In this case, if the theme module 320 is implemented in a different device than the preprocessing module 310 (for example, a device provided by an external third party), the server 500 can input a prompt to the theme module 320 in order to acquire information in a manner agreed upon in advance with the theme module 320.

[0146] Furthermore, even if the theme module 320 is an external or internal server, information necessary for calculating various feature sets (for example, the format of the color set, examples of color sets corresponding to color commands, etc.) may be input to the theme module 320.

[0147] For example, if the preprocessing module 310 generates the color command "The weather is hot. Change to a cool color" for the command "The weather is hot. Change to a cool color," but there is no predefined color set corresponding to a cool color, the color command may be input to an LLM, which is one embodiment of the theme module 320, and a color set may be calculated (S31b).

[0148] Then, based on the accuracy results calculated by the LLM, if the user's intention to change the color in the second step is successfully understood, the theme module 320, for example, extracts the color theme Hex value to apply the color theme calculated by the generative AI, and the feature setting module (e.g., color setting module) 340 can change the panel color of home appliances 100a, 100b, 100c such as refrigerators (S51a, S51b, S51c). Understanding the intention is exemplified by calculating a color set corresponding to the user's color command word. Understanding the intention for a feature command word entered by the user in this specification includes the process of calculating a feature set corresponding to the feature command word.

[0149] In this process, if the theme module 320 is an external server, the LG ThinQ server may include a feature setting module 340 that can set the feature set (e.g., color set) calculated by the theme module 320 to a home appliance.

[0150] On the other hand, if the accuracy of the feature set (e.g., color set) calculated by the theme module 320 is below a certain standard (S41a), this corresponds to a failure in accuracy, and therefore the personalization module 330 can calculate the feature set corresponding to the feature instruction word (S41b).

[0151] If the theme module 320 fails to analyze the user's feature change intent, for example, by a generative AI or LLM (i.e., fails to grasp the secondary feature intent), the personalization module 330, for example, a recommendation AI server, can recommend features. In this process, the theme module 320 can determine whether or not the user has a recommendation intent, and the theme module 320 can also calculate in the accuracy result whether or not the user wants feature recommendations from the recommendation AI server.

[0152] For example, if the input color command is "Recommend a color theme that suits me today," the accuracy of the color set calculated in S21b / S31b may be below the threshold and therefore a failure. Alternatively, the LLM may respond that it is unable to calculate a color set. In this case, the personalization module 330 can utilize the history information of the user's previous color settings for the home appliance.

[0153] The personalization module 330 extracts a color theme calculated to be preferred by the user based on the user's color usage patterns in their home appliances, and then changes the color theme of the transmitted product using the color settings module (one embodiment of the feature settings module 340) (for example, an LG ThinQ server).

[0154] In applying the embodiment shown in Figure 4, the pre-processing module 310 can perform different pre-processing depending on whether or not the type or name of a home appliance is included in the command. In one embodiment of the invention, if the type or name of a home appliance is included in the command, the pre-processing module 310 pre-processes the command based on that appliance. In another embodiment of the invention, if the name or type of multiple home appliances is included in the command, the pre-processing module 310 can pre-process the command based on a specific home appliance that recognizes the user's command utterance. In yet another embodiment of the invention, if the name or type of a specific home appliance is not included, the pre-processing module 310 can pre-process the command based on a specific home appliance that recognizes the user's command utterance, or it can identify a home appliance that matches the command and pre-process the command.

[0155] Furthermore, when processing the results calculated from theme module 320, which uses LLM as one example, the feature settings can be changed depending on whether or not the type and name of the home appliance are included in the results.

[0156] In one embodiment of the invention, if the command includes the type or name of a home appliance, the theme module 320 calculates the result based on that appliance. In another embodiment of the invention, if the command includes the names or types of multiple home appliances, the theme module 320 can calculate the result based on a specific home appliance that recognizes the user's command utterance. In yet another embodiment of the invention, if the command does not include the name or type of a specific home appliance, the theme module 320 can calculate the result based on a specific home appliance that recognizes the user's command utterance, or it can identify a home appliance through the command and calculate the result.

[0157] The above points can also be applied to various feature sets, such as speech sets or function sets.

[0158] Figure 5 shows the process of processing a color instruction word according to one embodiment of the present invention.

[0159] The user inputs a color command word via voice or text (S22). The preprocessing module 310 checks the color set based on logic (S23). That is, the preprocessing module 310 extracts keywords from the color command word (spoken command word or text command word) input by the user. Then, based on logic that compares the extracted keywords with the names and keywords of pre-stored color sets, the preprocessing module 310 calculates the color set that corresponds to the intent of the color command word input by the user. This includes the process by which the preprocessing module 310 grasps the intent of the color command word and calculates a color set appropriate for it.

[0160] If the color set is confirmed during the S23 process (i.e., the user's intention to change the color is understood) (S24 - Yes), the feature setting module 340 applies the confirmed color set to the home appliance (S52).

[0161] If, in the S23 process, the color set has not been confirmed (i.e., the user's color intent is not understood due to a color change) (S24 - No), the theme module 320 calculates the color set (S32). For example, the server 500 or home appliance 100 understands the user's color intent using color commands entered by the user, such as LLM (GPT) (S32). In one embodiment, the server 500 or home appliance 100 may include a theme module 320, and these theme modules 320 can be composed of software or hardware. The control unit, processor, CPU, etc. that control the server 500 or home appliance 100 can load the theme module 320, which is composed of software, into memory and execute it, or input predetermined information to a hardware chip that embodies the theme module 320 and obtain the result. The server 500 or home appliance 100 checks whether the accuracy of the calculated color set is above a certain standard (S33). For example, server 500 or home appliance 100 determines whether the color set calculated in S32 is suitable for the color intent of the color command word entered by the user. If the accuracy is above the standard, step S52 is performed.

[0162] On the other hand, if the accuracy in S33 is low, the server 500 or the home appliance 100 checks whether the entered color command is a recommendation based on personal information (S34). For example, if the entered color command is "Make it my favorite hue," the color command can be confirmed as a request for a color based on personal information.

[0163] On the other hand, if the input color command is "Hey, the refrigerator's color looks strange?", it is difficult to confirm that the color command is a request for a color change based on personal information.

[0164] One example of such a judgment is confirming that the theme module 320 has difficulty determining that a color command is a color change during the color set calculation process for the color command.

[0165] In S34, if it is confirmed that the user is a target for recommendation based on personal information, the personalization module 330 calculates a color set using the personal information (S42). Then, the server 500 or home appliance 100 checks whether the accuracy of the calculated color set is above the standard (S43). For example, the server 500 or home appliance 100 determines whether the color set calculated in S42 is suitable for the color intent of the color command word entered by the user. If the accuracy is above the standard as a result of the determination, step S52 is performed.

[0166] If the accuracy in S43 is not above the standard (S43-No), the home appliance 100 requests a re-speech (S45). For example, the home appliance 100 may request a new command to be uttered and re-inputted, such as "Please say it again to change the color."

[0167] Figure 6 shows the configuration of a personalization module according to one embodiment of the present invention.

[0168] The personalization module 330 may include a user database (DB) 350 that stores information about the user's color settings and a recommendation AI module 335. The personalization module 330 can extract a specific color set from the user database based on the user's past usage patterns for color commands. If the personalization module 330 is deployed on the server 500, the recommendation AI module 335 can be implemented on the recommendation AI server. On the other hand, if the personalization module 330 is deployed on the home appliance 100, the recommendation AI module 335 may consist of software components or hardware components within the home appliance 100.

[0169] Figure 7 shows the process by which a personalized module according to one embodiment of the present invention makes recommendations using personalized information. Process S42 will be examined in more detail.

[0170] The personalization module 330 checks the home appliance and time information, and the color set currently applied to the home appliance (S61). Then, the personalization module 330 requests n of the most frequently applied color sets (most frequent color sets) for the home appliance or other home appliances used by the user from the user database 350 (S62). n may be 2 or more.

[0171] If, as a result of the request, the extracted most frequent color set does not have more than m elements (S63 - No), the number of most frequent color sets is insufficient, so the personalization module 330 calculates a color set that reflects the seasonal information or current weather information to which the current date belongs and applies it to the home appliance (S67). For example, m can be 2 in one embodiment.

[0172] If there are m or more most frequent color sets (S63-Yes), the number of most frequent color sets secured by the personalization module 330 is sufficient, so each most frequent color set is compared with the color set currently set on the home appliance. For example, the personalization module 330 checks if most frequent color set 1 is the same as the color set currently set on the home appliance. If they are the same (S64-Yes), the personalization module 330 sets one of the m most frequent color sets, excluding most frequent color set 1, on the home appliance (S65). If they are not the same (S64-No), the personalization module 330 sets most frequent color set 1 on the home appliance (S66).

[0173] In the aforementioned embodiment, n or m can be set in various ways.

[0174] The example in Figure 7 can be applied to a process that recommends feature sets based on the user's usage patterns. Figure 7 shows an example in which the personalization module 330 extracts the top two most frequent values ​​from the list of color sets (color theme Hex values) that have been applied up to the present, and then applies them to a theme different from the currently applied color theme.

[0175] If the user database is insufficient and extracting the two most frequent values ​​is difficult, then a color theme can be recommended based on the current date, time, weather, etc., and applied to home appliances.

[0176] Next, we will consider the process of calculating the feature set using theme module 320.

[0177] When a user uses a predefined name in the product as a feature instruction word to use a specific feature of the device, the preprocessing module 310 included in the server 500 or home appliance 100 can provide a feature set corresponding to that feature.

[0178] On the other hand, if a user utters a feature command that is common in the user's social circle, the theme module 320 can use a language model to calculate a feature set. For example, the personalization module queries a database to identify the color set the user has used most frequently in the past. If there is sufficient data, it selects the color set that ranks highest among the user's preferences. To ensure diversity, if that color set is already active on the device, it can also apply the next most frequently used color set. If clear preferences cannot be determined from the user's history, the personalization module generates a new color set based on external contextual information such as the current season and weather, and recommends a highly relevant and appropriate color set.

[0179] Figure 8 shows the process of calculating a feature set based on a theme module 320 according to one embodiment of the present invention.

[0180] The server 500 or home appliance 100 extracts feature commands and feature sets from its internal storage unit that can be input into the theme module 320. These are called sample feature commands and sample feature sets. The server 500 or home appliance 100 then inputs the sample feature commands, sample feature sets, and feature commands into the theme module 320 (S71).

[0181] As a result, the theme module 320 calculates the second feature set and the second accuracy (S72). If the second accuracy is above the standard (S73), the server 500 or the home appliance 100 can apply the second feature set to the home appliance 100 (S74). On the other hand, if the second accuracy does not meet the standard, the server 500 or the home appliance 100 calculates the feature set using the personalization module 330.

[0182] Sample feature commands are examples of commands that a user can utter. Sample feature sets are feature sets that correspond to these sample feature commands.

[0183] For example, the sample color command and its corresponding sample feature command are as follows:

[0184] [Table 4]

[0185] When a sample color command and a sample feature set are input into the theme module 320, and a color command is input, the theme module 320 can apply the relationship between the sample feature command and the sample feature set to calculate the color set (second color set) corresponding to the color command.

[0186] In one embodiment, if the theme module 320 is configured based on a language model such as chatGPT, the server 500 or the home appliance 100 can input prompts to the theme module 320. In one embodiment, the server 500 or the home appliance 100 can include sample color commands and a sample feature set in a few shot.

[0187] The prompt input to the theme module may consist of instructions and sample feature commands. In one embodiment, the prompt may include instructions input to theme module 320 and messages provided to theme module 340 by server 500 or home appliance 100.

[0188] Furthermore, the message may include the sample feature command words (sample color command words) discussed in Table 4, and the corresponding sample feature set (sample color set). This allows the theme module 320 to calculate the color set corresponding to the color command word (user_request) uttered by the user.

[0189] To enable the theme module 320 to calculate results more accurately, the server 500 or home appliance 100 can input additional information into the theme module 320, such as the region or country to which the user who uttered the feature command belongs. For example, the color of an apple that a Korean person perceives may differ from the color of an apple that an American person perceives. Therefore, the server 500 or home appliance 100 can input information about the region or country where the feature command was uttered into the theme module 320, enabling it to accurately calculate a feature set, such as hues, sounds, or functions, that are commonly used by people belonging to a particular region or country.

[0190] In one embodiment, when the theme module 320 is GPT, the home appliance 100 or server 500 can be instructed to extract four theme colors as hex values ​​at the GPT prompt.

[0191] Furthermore, if the user's spoken color command indicates a desire for recommendation, the theme module 320 is instructed to return a "True" value (the area instructed to return whether a recommendation is desired). The home appliance 100 or server 500 can also add color therapy content as a prompt instruction to generate a color theme according to the situation. The home appliance 100 or server 500 can present several examples (sample color command words and sample color sets) in a few shots and include prompts to output a recommendation flag or RGB color hex values ​​that the home appliance 100 can set as a list of results.

[0192] For example, if applied to a 3-door kimchi refrigerator, the appliance 100 or server 500 can be modified to extract 3 color hex values ​​instead of 4.

[0193] Furthermore, when applying to different types of home appliances such as air conditioners and ovens, the color Hex values ​​output can be modified and set according to the number of panels (doors, sides, etc.) of the appliance.

[0194] On the other hand, the personalization module can use personalization information to calculate the feature set corresponding to the feature instruction word.

[0195] A personalization module 330 included in the server 500 or home appliance 100 acquires device information of the home appliance from which a feature instruction word has been input. The personalization module 330 then extracts personalization information corresponding to the device information. In one embodiment, the personalization information can calculate a third color set using information about the color sets previously set on the device corresponding to the device information, and information about the frequency of setting these color sets or the maintenance time of the color sets, whichever is greater than or equal to the above.

[0196] In another embodiment, the personalized information may include information about the home appliance corresponding to the device information, including the previously set function set and the frequency of the set function set or the duration of the function set, or more of these.

[0197] Therefore, the database can store identification information, time information, and device-specific identification information related to color sets. The database can also store information about the most recently applied color set.

[0198] The database stores the color sets configured for each device. Therefore, if the theme module 320 determines that a feature instruction is a recommendation based on the user's pattern, the personalization module 330 can recommend a feature set for each individual device through calculations. As a result, different feature sets may be calculated for each home appliance.

[0199] For example, if device_id is 1, the mode of the color set applied to device_id can be sorted, and then the hue of the home appliance can be set to the next color that is different from the current set color.

[0200] [Color1, Color2, Color3, Color4] If device_id is 2, the mode of the color set applied to device_id can be sorted, and then the hue of the home appliance can be set to the next color that is different from the current set color.

[0201] [Color13, Color14, Color15, Color16] For example, in the information stored in the database, if device_id is 1, then Iron Man has been applied twice, and currently the color set "Republic of Korea" is applied. Therefore, if the user's spoken command is "recommend," the personalization module 330 provides the hue corresponding to Iron Man to the recommendation color set.

[0202] Similarly, if device_id is 2, the "Autumn Colors" theme is applied twice, so the "Autumn Colors" theme will also be applied in the case of recommendations.

[0203] On the other hand, the server 500 or the home appliance 100 can store the feature set calculated by the theme module 320 in its personal information.

[0204] For example, if a device with device_id 1 is repeatedly set with the feature set calculated by theme module 320 (e.g., Iron Man hue), or if the setting is maintained for a long period of time, the server 500 or home appliance 100 stores the feature instruction word (Iron Man) and hue information ([Color1, Color2, Color3, Color4]) in the database.

[0205] Subsequently, when the user enters a new command, the server 500 or the home appliance 100 loads and applies the information stored in the database.

[0206] For example, when the preprocessing module 310 processes the color command "Make it Iron Man color," after extracting the keyword, it can calculate the first color set using the "Iron Man" hue and hue information ([Color1, Color2, Color3, Color4]) stored in the database.

[0207] Alternatively, when the personalization module 330 processes the color command "Make it Iron Man colors," after extracting the keyword, it can calculate a third color set using the "Iron Man" hue and hue information ([Color1, Color2, Color3, Color4]) stored in the database.

[0208] In this process, the server 500 or the home appliance 100 does not need to input any additional feature instructions to the theme module 320. That is, in the process of performing steps S21-S51 in Figure 3 once, we assume that the second feature set calculated in S31 is applied to the home appliance 100 and maintained for a long period of time.

[0209] If the user utters the same feature command, the pre-processing module 310 of the home appliance 100 or server 500 can process it, or the personalization module 330 can skip step S31 and load the stored feature set from the database in step S41.

[0210] In other words, if the user utters "Iron Man color" once and then applies and maintains it on the home appliance 100 for a certain period of time, or if the number of times it is repeated exceeds a certain threshold, the server 500 or the home appliance 100 can store the feature command and feature set in a database to increase the processing speed of the user's spoken command.

[0211] In summary, the following applies: The server 500 or home appliance 100 stores feature instructions and the second feature set in a database according to the maintenance time or application count of the second feature set calculated by the theme module 320.

[0212] After performing steps S21 to S51 in Figure 3, the server 500 or home appliance 100 can utilize the information stored in the database during the process in which the preprocessing module 310 calculates the first feature set or the personalization module 330 calculates the third feature set for the input feature instruction words.

[0213] More specifically, the preprocessing module 310 can retrieve information stored in the database and generate a feature set. Alternatively, the personalization module 330 can retrieve information stored in the database and generate a feature set.

[0214] The following describes an apparatus that embodies an embodiment of the present invention.

[0215] Embodiments of the present invention can be embodied in various devices. These devices include servers, home appliances, electronic devices, computer devices, and many others. In addition to physical devices, the devices of the present invention may also include hardware or software components that perform embodiments of the present invention. Furthermore, embodiments of the present invention include programs, hardware, chips, etc., that are stored or embodied in a form that enables them to perform predetermined tasks.

[0216] In the case of programs, software, etc., they may be stored permanently within the device, or they may be temporarily transmitted from an external source, stored in the device, and then executed. In the case of a permanent storage method, the device may include a non-transitory computer-readable medium.

[0217] In other words, embodiments of the present invention can be embodied in a computer-readable storage medium by one or more computer programs, or any combination of any or more of the above.

[0218] The functions of the elements disclosed herein can be embodied using circuits or processing circuits that include general-purpose processors, special-purpose processors, integrated circuits, ASICs ("Application-Specific Integrated Circuits"), existing circuits, and / or combinations thereof. These circuits may be processors configured or programmed to perform the disclosed functions. Since processors include transistors and other circuits, they can be considered processing circuits or circuits.

[0219] Circuits, units, or means in this specification may be hardware that performs or is programmed to perform the functions mentioned in the detailed description. Hardware may be the hardware disclosed herein or other known hardware, and may be hardware that is programmed or configured to perform the functions mentioned in the detailed description of the specification. If the hardware is a processor that is considered a type of circuit, then the circuit, means, or unit may be software used to constitute the hardware and / or processor in combination with the hardware and software. Furthermore, computer storage media may be non-transitory computer readable medium. For example, they may be executable by a cloud server-based system. Computer storage media may be located in the same single device or distributed across two or more different devices. Thus, logically one computer storage medium may physically include two or more computer storage media, and they may be located in one or more locations. Computer storage media include various storage media such as hard disks, CD / DVD discs, memory cards, and memory chips.

[0220] Furthermore, the data described in the specification can be computed or performed in various environments, including cloud server-based systems, on-device systems, and distributed server systems (multiple servers). Processing may be performed in a distributed manner on cloud servers or locally on an on-device processor, and the results processed in each environment can be stored in non-volatile memory.

[0221] Electronic devices such as home appliances and server computer equipment may be connected to one or more storage devices via a network. The storage devices may be a combination of volatile and non-volatile memory, and may or may not be located in the same physical location as the computer equipment.

[0222] A server computer system may include one or more processors and memory. The memory may store information accessible to the processor and may contain data capable of processing, storing, or modifying instruction words executed by the processor. This memory may also consist of volatile and non-volatile memory. The processor may include a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a tensor processing unit (TPU).

[0223] Instructions may be configured to perform specific actions when a processor executes an instructed task, and may be stored in the form of object code or an interpretable script. These instructions can be used to embody a system and can be executed on a local or remote processor. Data can be retrieved, stored, or modified by instructions and may consist of a database, JSON, YAML, or XML.

[0224] These commands may include executable files, source code files, or metadata.

[0225] Electronic devices, including home appliances and hubs, can be configured similarly to server computer devices. Electronic devices, including home appliances and hubs, may include a processor, memory, instructions, data, and user input and output devices. The server computer device can transmit data to the electronic devices, and the electronic devices can display a portion of the received data via a display. Furthermore, data transmission and communication between the server computer device and the electronic devices are possible via networks such as Bluetooth®, Wi-Fi, wired, and wireless networks, and various protocols and connection methods are supported. Direct and indirect communication between computer devices is possible, and various protocols and connection methods are supported.

[0226] Furthermore, the functions of the server computer device of the present invention can be performed by smartphones, tablets, etc.

[0227] The methods or processes described herein involve sequentially performing one or more tasks, and each task can be performed by either hardware, software, or a combination of both. For example, the first task may be performed by hardware and the second by software. Of course, either hardware or software can perform the entire task.

[0228] Figure 9 shows the processing steps when fine adjustments are made to the feature set according to one embodiment of the present invention.

[0229] The server 500 or home appliance 100 receives a fine-tuning instruction for either the first feature set calculated by the preprocessing module 310, the second feature set calculated by the theme module 320, or the third feature set calculated by the personalization module 330, and calculates a fourth feature set (S81).

[0230] In this case, the server 500 or the home appliance 100 stores the feature instruction and the fourth feature set in the database (S82). Then, the server 500 or the home appliance 100 processes the feature instruction that is subsequently input using the fourth feature set stored in the database (S83).

[0231] For example, after the process of steps S21 to S41 in Figure 3 is performed, the preprocessing module 310 calculates a feature set for the input feature instruction words, or the personalization module 330 loads a fourth feature set stored in the database and sets it as the feature set for the home appliance.

[0232] When applying the aforementioned embodiment, if a user wants to change the features (e.g., hue, voice, functions) of a home appliance such as a refrigerator or air conditioner, they can apply one of the following: a feature set provided by the appliance in response to the user's text or voice commands (e.g., a basic theme), a feature set provided by a theme module such as a generative AI, or a feature set provided by a personalization module equipped with a recommendation AI.

[0233] Furthermore, if the same voice / text commands accumulate for features (e.g., color values) calculated by the theme module 320, such as a generative AI, these can be stored in a database as a personalized feature set, improving efficiency in the process of modifying features. For example, after a feature set is generated by one of the main steps described above (NLP matching, AI theme generation, personalized generation, etc.), the user can adjust the results to their liking by issuing a fine-tuning command. The user-adjusted feature set is stored in the database and can be permanently associated with the original command. This allows the system to skip the initial generation step and directly apply the user-saved fine-tuning version when executing the same command in the future. This enables the system to learn and adapt to individual preferences over time, resulting in faster and more accurate responses.

[0234] The personalization module 330 can be implemented within a home appliance, or within a server 500, such as a cloud server. The theme module 320, which uses generative AI as one example, can use an external generative AI server and can be implemented within a home appliance 100 or its own server 500.

[0235] Each module discussed in Figure 4 can be implemented in either the home appliance 100 or the server 500. In the case of the theme module 320, it may also be placed on a separate external server of a generative AI distinct from the server 500.

[0236] Figure 10 shows a configuration in which all components are included within a home appliance according to one embodiment of the present invention.

[0237] One embodiment of the device, a home appliance 100, includes a pre-processing module 310, a theme module 320, a personalization module 330, and a feature setting module 340. The command input module 110 is a module into which feature command words such as voice and text are input. The pre-processing module 310, theme module 320, personalization module 330, feature setting module 340, and database 350 are described in the above description. The database 350 can store personalization information of the device.

[0238] The feature setting module 340 sets a feature set to a feature, which is either the first feature set, the second feature set, or the third feature set calculated by the preprocessing module 310, the theme module 320, or the personalization module 330, respectively.

[0239] Figure 11 shows a server containing a preprocessing module, a theme module, a personalization module, and a database according to another embodiment of the present invention. The server 500 and the home appliance 100 may also include a communication module (not shown in the drawing) for sending and receiving predetermined information.

[0240] The server in Figure 11 provides personalized services using generative AI. Looking at the detailed components of server 500, the preprocessing module 310 receives feature instructions input by a home appliance and calculates a first feature set and a first accuracy corresponding to the feature instructions.

[0241] The theme module 320 calculates a second feature set and a second accuracy corresponding to the feature instruction word if the first accuracy is below a certain threshold. The personalization module 330 calculates a third feature set using the personalization information corresponding to the device information of the home appliance and the feature instruction word if the second accuracy is below a certain threshold.

[0242] When the server 500 transmits one of the first to third feature sets calculated in this way to the home appliance 100, the home appliance 100 sets one of these feature sets as a feature of the home appliance. Therefore, the feature setting module 340 can set the received feature set to a specific area or function of the home appliance 100.

[0243] If the feature set is a color set, the feature instruction corresponds to a color instruction. For example, if the preprocessing color instruction includes the hue name of a color that constitutes the first color set, the server 500, for example, the preprocessing module 310, can determine that the first accuracy of the first color set corresponds to the criterion. This is the case when a specific hue name, such as orange or blue, is included in the color instruction.

[0244] If a feature set is a set of functions, then a feature instruction corresponds to a function instruction. For example, if a preprocessing function instruction includes the function name of a function that constitutes the first function set, the server 500, for example, the preprocessing module 310, can determine that the first accuracy of the first function set is the criterion. This is the case when a function instruction includes a specific function name, such as setting the temperature of the refrigerator or turning the air conditioner off / on.

[0245] The above-described embodiment can be applied to a server 500 or home appliance 100 that includes a pre-processing module 310.

[0246] Furthermore, in order to acquire a feature set using the theme module 320, the server 500 inputs a sample feature command and a sample feature set corresponding to the sample feature command into the theme module 320. The server 500 then checks the second feature set and second accuracy calculated by the theme module 320 and decides whether or not to input the feature command into the personalization module 330.

[0247] If the user enters a command such as "recommend me," theme module 320 can set the second accuracy to very low.

[0248] As described above, the personalization module 330 of server 500 can calculate a color set using information about the color set previously set for the home appliance corresponding to the device information, and information about the frequency of the set color set or the duration of the color set, or any of the above.

[0249] Furthermore, to improve the accuracy of the results calculated by the theme module 320, the server 500 can input additional information about the region or country to which the user who uttered the feature command belongs. In addition, the user's age, gender, and occupation information can also be input into the theme module 320 to improve the accuracy of the feature set calculated in response to the user's feature command.

[0250] The server 500 transmits one of the calculated feature sets to the home appliance 100, and the home appliance 100 can apply the transmitted feature set to the home appliance 100.

[0251] Figure 12 shows a server containing a preprocessing module, a personalization module, and a database according to another embodiment of the present invention. The server 500 and the home appliance 100 may also include a communication module (not shown in the drawing) for sending and receiving predetermined information.

[0252] Furthermore, the theme module 320 is located on a separate generative AI external server 700, distinct from server 500. In this case, unlike in Figure 11, server 500 includes a counter theme module 325. For a description of the other components, refer to Figure 11.

[0253] If the first accuracy calculated by the preprocessing module 310 is below the standard, the counter theme module 325 transmits the feature instruction to the theme module 320 of the generative AI's external server 700, and receives the second feature set and the second accuracy from the theme module 320.

[0254] The counter theme module 325 transmits a sample feature instruction and a sample feature set corresponding to the sample feature instruction to the theme module 320 in order to improve the accuracy of the results from the theme module 320. The counter theme module 325 then receives a second feature set and a second accuracy corresponding to the feature instruction from the theme module 320.

[0255] The above-described embodiment can be implemented not only with a remote controller or the main unit of the home appliance 100 for voice recognition and intent understanding, but also with a voice-recognition-capable device (e.g., an AI speaker or a hub). For example, a home appliance such as a refrigerator or air conditioner may include only the feature setting module 340, while a home appliance such as an AI speaker or a hub may include the command word input module 110.

[0256] Furthermore, home appliances such as AI speakers and hubs may include a pre-processing module 310, a theme module 320, and a personalization module 330. In this case, home appliances such as AI speakers and hubs can also transmit device control signals to related home appliances (or devices to be controlled).

[0257] As shown in the embodiment in Figure 12, if the theme module 320, for example, LLM(GPT), is deployed on an external AI server 700, the theme module 320 cannot access the customer's user database due to personal information and security concerns. Therefore, the embodiment in Figure 12 can be implemented by separating the generative AI, the server (theme module), and the recommendation AI server (personalization module).

[0258] The theme module 320 and the personalization module 330 may calculate a feature set for feature commands using a generative AI function (theme module) that reflects the linguistic characteristics of the society to which the user belongs, and a recommendation AI function (personalization module) based on personalization information. Furthermore, by applying the preprocessing module 310 and the personalization module 330, problems such as increased costs or network delays that occur during the use of the theme module 320 located on an external AI server 700 can be resolved.

[0259] Figure 13 shows the configuration of a home appliance, a home appliance for voice processing, and a server according to one embodiment of the present invention.

[0260] An AI speaker or hub is a home appliance 100b for voice processing, and includes a command word input module 110. In this case, the home appliance 100b for voice processing communicates with a server 500 to receive a feature set and provides it to home appliances 100a such as a refrigerator, air conditioner, or oven, thereby controlling the feature settings of the home appliances 100a.

[0261] Alternatively, when combining the embodiments shown in Figures 10 and 13, the AI ​​speaker or hub may include a pre-processing module 310, a theme module 320, a personalization module 330, and a command word input module 110 for voice processing. Furthermore, home appliances such as refrigerators, air conditioners, and ovens (100a) can be configured to include a feature setting module 340.

[0262] The present invention is not limited to specific home appliances, but can be applied to any home appliance that includes a feature setting module 340 for setting the hue, type of sound, and function, as described above.

[0263] In the embodiments shown in Figures 11 and 13, the preprocessing module 310, the theme module 320, and the personalization module 330 can constitute a single processor.

[0264] In the embodiment shown in Figure 12, the preprocessing module 310, the counter theme module 325, and the personalization module 330 can constitute a single processor.

[0265] Based on the embodiments shown in Figures 11 to 13, the server's processor can process information received from home appliances and input it into the theme module 320, which is a generative AI model, to obtain results. The processor can receive feature instructions from the first or second home appliance to instruct feature changes in the first home appliance.

[0266] Furthermore, the processor can generate a prompt in response to a feature instruction, which includes at least one of the following: configuration information for the first home appliance, state information for the first home appliance, and the feature instruction.

[0267] The processor can input the generated prompt into the theme module 320 to obtain the result, and convert the obtained result into a first feature set to be set in the first home appliance.

[0268] Here, the second home appliance may be a user terminal or any other home appliance besides the first home appliance. For example, a smart hub or a smart AI speaker may be the second home appliance, in which case the user's voice / text commands may be transmitted to the server 500 via the second home appliance to convert the feature set of the first home appliance (e.g., a refrigerator or air conditioner).

[0269] User commands may be entered via voice or text, and the device that receives and transmits these commands to the server may be a first consumer electronics device instructed to modify the feature set, or a separate second consumer electronics device (e.g., a user terminal, an AI speaker, etc.).

[0270] As described above, in one embodiment, the feature command is a command for instructing color change of the first household electrical appliance.

[0271] The feature set includes information regarding one or more colors set for the first household electrical appliance.

[0272] The configuration information is information regarding the area of the first household electrical appliance where one or more colors are set. For example, information that a specific color is set for the door of the refrigerator compartment of a refrigerator corresponds to the configuration information.

[0273] The status information includes information regarding the second feature set that is currently set or has been set in the past for the first household electrical appliance.

[0274] The result obtained from the theme module 320 may include two or more candidate feature sets.

[0275] The processor can apply any one or more of the configuration information, the status information, and the feature command to convert one of the two or more candidate feature sets into the first feature set.

[0276] In addition, the processor can transmit the first feature set to the first household electrical appliance or transmit a feature control command for applying the first feature set to the first household electrical appliance to the first household electrical appliance. The processor can also transmit the first feature set to the first household electrical appliance via the second household electrical appliance.

[0277] Therefore, the processor can store software or program code that can perform the above-described operations. These software or program code are received from another external device, stored in a storage medium used by the processor, and then the processor can execute the software and program code.

[0278] Furthermore, the processor may include hardware components such as a programmable chip, and the processor can store data and program code to be input to the hardware components in a predetermined storage medium before inputting them to the hardware components.

[0279] The hardware or software may be the processor itself. Alternatively, the hardware or software may work in conjunction with the processor to embody embodiments of the present invention.

[0280] In the embodiments shown in Figures 11 to 13, the server 500 or the home appliance 100 may include modules in various combinations. These modules can be embodied in hardware or software, and therefore each module can be embodied in one storage medium or multiple storage mediums. Furthermore, the storage mediums can be distributed and embodied in the server 500 and the home appliance 100.

[0281] Therefore, the storage medium installed in the server 500 or the home appliance 100 may include a pre-processing module 310 and a personalization module 330. Furthermore, the storage medium may include either a theme module 320 or a counter theme module 325.

[0282] The preprocessing module 310 calculates a first feature set and a first accuracy corresponding to the feature instruction word.

[0283] The theme module 320 can calculate the second feature set and second accuracy corresponding to the feature instruction word if the first accuracy is below a certain threshold.

[0284] If the first accuracy is below a certain threshold, the counter theme module 325 can transmit the feature instruction to the theme module 320 of the generative AI's external server 700 and receive the second feature set and the second accuracy from the theme module 320.

[0285] If the second accuracy is below a certain threshold, the personalization module 330 can calculate a third feature set using the personalization information and feature instruction words corresponding to the device information of the home appliance.

[0286] The storage medium can then control the home appliance to configure its features. The storage medium can provide the feature configuration module 340 of the home appliance 100 with the information necessary to configure the features.

[0287] Therefore, Figures 11 to 13 include embodiments of the configuration of a storage medium that can be read and executed by a computer. Figures 11 to 13 also include embodiments of the configuration of hardware or software embodied in a server 500 or a consumer electronics device 100.

[0288] When applying the embodiments shown in Figures 11-13, existing older home appliances without AI functionality (e.g., 100a) may also be connected to a server 500 that generates voice processing and feature sets, or a service (e.g., ThinkQ ON) that provides feature sets via home appliance equipment 100b for voice processing.

[0289] As a result, the generative AI provides a set of features that allow it to understand the context of the user's commands and operate smoothly, enabling the home appliance 100a to be upgraded to an AI home appliance. Consequently, users do not need to purchase all new AI home appliances, as they can combine ThinkQ ON with IoT devices to realize a rational AI home even with existing appliances.

[0290] For example, consider a scenario where a customer connects an existing air conditioner without AI capabilities and a room sensor that senses the surrounding space to a ThinkQ ON that provides a feature set. By generating a feature set corresponding to the feature command, ThinkQ ON can determine the user's location in real time and adjust the airflow and direction of the air conditioner to care for the customer with optimal airflow. Existing air conditioners can be upgraded to powerful AI air conditioners by combining them with LG ThinkQ ON, without the need to purchase expensive, state-of-the-art AI air conditioners.

[0291] The server 500 according to an embodiment of the present invention is a server that provides artificial intelligence services and may include a database 350 that stores past personalized information and routine information of customers.

[0292] By using a large language model like ChatGPT and a server containing user experience data such as the user's electronic device usage patterns, it is possible to provide personalized spatial solutions based on learning about the user's lifestyle. For example, if a user says "I studied well last week. Set it up the same way" as a feature command, the system can load the settings from that time stored in database 350 and create a personalized environment.

[0293] In one embodiment of the present invention, the server 500 includes embodiments in which the server is a group of servers, which is a collection of various servers. Therefore, the configuration of the server 500 can be easily modified at the level of those skilled in the art.

[0294] As one embodiment, the server 500 in the embodiments shown in Figures 11 to 13 can provide the functionality of an NLP (Natural Language Processing) server. For example, this could include a preprocessing module 310, a counter theme module 325, or a theme module 320. The server 500 can also perform the task of inputting predetermined prompt messages into the theme module 320. That is, in order to obtain accurate responses via an LLM model such as chatGPT, the server 500 can structure the content of the prompts and store them in an utterance database.

[0295] If the theme module 320 is implemented on an external AI server 700, as shown in Figure 12, the server 500 can efficiently generate prompt tokens and ensure the maximum performance of the theme module 320, with GPT as one example, through low cost and low network latency. The NLP server can be implemented internally on the server 500, or on a separate cloud server (for example, an AWS cloud server).

[0296] In the embodiment shown in Figure 12, server 500 may incur costs in the process of inputting predetermined prompt messages to the external AI server 700 and thereby obtaining results. Therefore, if server 500 ensures that the preprocessing module 310 and personalization module 330 perform fine-tuning to the maximum extent possible for feature instruction words, train the system, and store the results in a database, it can reduce the number of times it uses the external AI server 700.

[0297] For example, in the embodiment shown in Figure 12, the server 500 can process feature commands using its internal modules 310 and 330 without using the GPT API, by fine-tuning and training the speech data instead of using the GPT API, which incurs costs.

[0298] Fine-tuning according to an embodiment of the present invention includes developing and applying a dedicated model specialized for home appliances (e.g., specific home appliances such as refrigerators and air conditioners) using a sLM (small Language Model).

[0299] If the performance of the AI chip installed in the home appliance 100 is guaranteed, as shown in FIG. 10, the service can also be provided in the onDevice form. [[ID= 6]]

[0300] Also, a theme module 320 according to an embodiment of the present invention can apply an LLM. For example, the theme module 320 can interpret the meaning of speech using NLP and output corresponding intent and slots based on the input prompt and sample data.

[0301] Since an LLM such as GPT drives a high-performance data center, it can provide overwhelming performance in terms of processing speed and data volume. Therefore, the theme module 320 may be arranged in the server 500 and can be implemented in an external AI server 700 as shown in FIG. 12.

[0302] As an example, the external AI server 700 is an example of a server 700 that implements chatGPT, MS Azure GPT, GPT-4o mini, etc. provided by Open AI. For the operation of inputting a query language, e.g., a prompt, to these external AI servers 700 and receiving a result, an API (Application Protocol Interface) can be used. For example, the counter theme module 325 in FIG. 12 can obtain a feature set corresponding to the feature command language using the API provided by the theme module 320.

[0303] When applying embodiments of the present invention, a theme module 320 equipped with artificial intelligence that performs natural language processing, such as GPT, can understand the intent of the feature command words uttered by the user and provide results (feature sets) indicating whether the user desires recommendations or specific feature combinations.

[0304] Furthermore, when applying prompt engineering techniques to feature commands input to theme module 320, it is not necessary to fine-tune the entire GPT artificial intelligence model; the performance of fine-tuning can be achieved solely through prompt engineering.

[0305] Mood-enhancing home appliances, such as refrigerators and air conditioners, can be integrated with generative AI (GPT) to accurately understand the speech intentions of various users and set the features they desire (e.g., various color themes, voice tones, and sets of functions).

[0306] The preprocessing module 310 has predefined and learned to recognize feature sets corresponding to specific utterances of the user, so it can process command words for predefined features (e.g., a specific color theme) in a short amount of time.

[0307] On the other hand, if the user desires a feature set not defined in the preprocessing module 310, a theme module 320 such as LLM(GPT) can be applied. In this case, prompt engineering can be applied to undefined instructions (feature instructions) as well, allowing for the creation of numerous feature sets. For example, when a color instruction is input, the theme module 320 can recognize natural language related to various color theme changes and extract themes associated with color.

[0308] Theme module 320 may include one or more LLMs, such as Microsoft's GPT, an LLM module from another company, or an LLM developed by LG itself. Furthermore, the server 500 does not necessarily have to include theme module 320; it may also be included in a separate LLM server provided by an external service provider or LG.

[0309] By applying embodiments of the present invention, the server 500, in addition to a natural language processing (NLP) server (preprocessing module), has a structure that combines an LLM (GPT) server (theme module) and a recommendation AI server (personalization module), enabling it to provide an accurate feature set by understanding the intent behind the feature command words input by the user.

[0310] As an example, if the user says, "Please change it to a very cool color," the preprocessing module 310 can change the color set defined as the summer theme color to the summer theme color. Also, if the user says, "The weather is really hot and humid, so please change it to a very cool color," and the preprocessing module 310 does not have such a color set defined, the theme module 320 can understand the user's intent and generate a color theme.

[0311] In other words, when applying embodiments of the present invention, the preprocessing module 310 and the theme module 320 can be combined to understand the user's intent and calculate a feature set that reflects that intent. For this reason, by including sample feature commands and sample feature sets in the prompts via prompt engineering, the theme module 320 can calculate feature values ​​(e.g., color values) that reflect the user's intent.

[0312] Furthermore, the theme module 320 can secure information on a set of features that are commonly accepted by the society or nation in question. For example, the theme module 320 can provide a color therapy function, and when a user says, "I'm depressed. Change the color," it can generate a color theme that provides appropriate psychological stability.

[0313] On the other hand, if the preprocessing module 310 and the theme module 320 fail to calculate a feature set that corresponds to the user's intent, the personalization module 330 can use user information (e.g., information about the user's device usage) to provide a customized feature set based on the feature sets the user has previously or currently set on their home appliances. In this case, the appearance of the home appliance can be changed to a color that the user enjoys using or prefers.

[0314] Since the theme module 320 does not store information about the user, the personalization module 330 can use the information about the user to provide a feature set tailored to that user.

[0315] The personalization module 330 can generate a color theme suitable for the user when the user enters a feature command such as, "I'll leave it to you, just change the colors to suit me."

[0316] In one embodiment of the personalization module 330, the recommendation AI server uses the theme module 320, for example LLM(GPT), to determine whether the user's utterance indicates a desire for artificial intelligence recommendation.

[0317] If it is determined that a recommendation is desired (for example, if the result of a theme module such as LLM calculating the feature set does not meet the criteria), the recommendation AI server can recommend a color theme based on the user's usual usage patterns.

[0318] For example, if a user wants a refrigerator color theme recommended, the personalization module 330, for example, the recommendation AI server, can generate a color theme, thereby increasing the success rate of responses to the user's commands related to changing the color theme (color commands) and increasing satisfaction with the generated color theme.

[0319] Figure 14 shows an interface according to one embodiment of the present invention that changes the color of a command word entered by the user.

[0320] The user inputs a feature command via voice or text into a smartphone, tablet, or refrigerator 100a. The entered feature command (Feature_Command) may be displayed in the "User Request" field. For example, commands such as "I'm angry, change to a calming color," "Change to Iron Man's colors," or "Recommend a color that suits me today" may be entered.

[0321] The server 500 or the pre-processing module 310, theme module 320, or personalization module 330 of the home appliance 100a processes the input command words and calculates color results. "C_A1, C_A2, C_A3, C_A4" in Figure 14 are examples of color sets. Each color set may be applied to the exterior of the refrigerator 100a to which the color is applied. The order of application can be predetermined.

[0322] 91 may be a hue theme previously used by the user, or the name of a hue that the preprocessing module 310 can process.

[0323] The user can input information regarding whether to maintain or change the applied hue using the interface of the home appliance 100a or a smartphone / tablet. The input information (maintain or change) is stored in the personalization module 330.

[0324] Figure 15 shows the process of changing the hue of numerous home appliances according to one embodiment of the present invention.

[0325] The user inputs a voice command such as "The weather is really hot and humid. Change the color to a very cool theme!" using a home appliance (one of 100a, 100b, or 100c) or a smartphone / tablet (S92).

[0326] Server 500 or one of the home appliances (100a, 100b, or 100c) performs the process discussed in Figure 3 and calculates a feature set (e.g., a color set) corresponding to the feature instruction (S93). If the result is "[C_A1, C_A2, C_A3, C_A4]", the color set can be applied to multiple home appliances.

[0327] For example, all four colors are applied to refrigerator 100a. On the other hand, kimchi refrigerator 100b, which has three color panels, is applied to three colors, such as "[C_A1, C_A3, C_A4]". In the case of air conditioner 100c, two colors are applied, such as "[C_A1, C_A4]".

[0328] With a single voice command from the user, a consistent color theme can be applied to all household appliances 100a, 100b, and 100c linked to the user's account on the server 500, for example, a ThinQ server.

[0329] In summary, for a user's feature command, the home appliance 100 or server 500 can apply consistent features to home appliances owned or used by the user by applying one of the following to other home appliances linked to the user account of the home appliance 100: a first feature set calculated via the preprocessing module 310, a second feature set calculated via the theme module 320, or a third feature set calculated via the personalization module 330. For example, according to one embodiment, this method enables the synchronization of a consistent theme across multiple home appliances with a single command. Upon user request, this method can generate an integrated feature set and distribute it to all connected devices. Each home appliance can intelligently adjust this master feature set to its own specific functions, creating a harmonious aesthetic and functional experience throughout the user's smart home environment.

[0330] Figure 16 shows the process of changing the features of a home appliance according to one embodiment of the present invention.

[0331] The process shown in Figure 16 can be performed by either the home appliance 100 or the server 500 alone, or by the two devices working together.

[0332] The home appliance 100 or server 500 acquires information regarding the type of home appliance, configuration information or status information of the home appliance, the range of features to be set on the home appliance, and sample feature instructions (S95). Then, the home appliance 100 or server 500 generates a prompt based on the acquired information and the feature instruction words entered by the user (S96). Subsequently, the home appliance 100 or server 500 inputs the prompt into the theme module 320 to calculate the feature set (S97). Then, the home appliance 100 or server 500 changes the feature set into features of the home appliance (S98). The feature setting module 340 of the home appliance 100 can set the calculated feature set as features of the home appliance 100.

[0333] In one embodiment, when controlling the functions of an air conditioner, the home appliance 100 or server 500 can acquire various information about the air conditioner (type, configuration information, status information, range of controllable functions, and information about control commands for sample functions).

[0334] The home appliance 100 or server 500 can then generate a prompt based on the acquired information and the feature command words related to the function entered by the user.

[0335] Subsequently, the home appliance 100 or server 500 can obtain commands (control commands) to control its functions after prompting the theme module 320, which is an LLM in one embodiment. The theme module 320 can be implemented in the home appliance 100, server 500, or external server 700.

[0336] Subsequently, the home appliance 100 or server 500 can control the air conditioner using the acquired control instructions. In this process, the home appliance 100 or server 500 can respond to subsequent feature instructions by repeating the information acquisition process of S95.

[0337] One embodiment of the present invention provides a method for controlling home appliances that can dynamically interpret a wide range of natural language input, adapt to the speech patterns of individual users, and convert even ambiguous commands into precise device operations. Another embodiment of the present invention provides a method for controlling home appliances that can accurately interpret user intent from abstract or thematic descriptions, flexibly generate and apply corresponding new color schemes, and achieve more adaptable aesthetic customization.

[0338] For example, according to one embodiment, the method may include a three-tiered hierarchical process for interpreting and executing user commands. The initial stage is handled by a preprocessing module 310 that functions as a natural language processing (NLP) system. The preprocessing module 310 receives raw voice or text input from the user and attempts to match it against a library of predefined feature sets.

[0339] For example, if a user requests a "spring theme" and that theme exists as a preset, the preprocessing module 310 finds a direct match, generates a high-accuracy score, and applies the corresponding feature set. If the command is similar to but not identical to a predefined feature set, such as a "spring breeze theme," the preprocessing module 310 identifies the nearest existing option (e.g., "spring theme") but sets a lower accuracy score. If this score exceeds a predetermined threshold, the matched feature can be applied. If the score is too low, the command can be escalated to the next stage for more advanced analysis. Furthermore, in this example, if the initial processing by the preprocessing module 310 does not find a definite match, the command is passed to the secondary theme module 230. The secondary theme module 230 can be equipped with generative artificial intelligence, such as a Large-Scale Language Model (LLM). At this stage, it is configured to understand the user's intent behind abstract or creative commands that do not fit into predefined settings.

[0340] For example, if a user says, "It's hot. Please change the color to cool," the theme module can interpret the abstract concept of "cool" and dynamically generate entirely new color combinations, such as shades of blue and white. The theme module 230 can also evaluate its results to determine whether it has created a valid and applicable set of features (for example, by generating a second accuracy score). If successful, this newly generated set can be sent for application to the appliance. Furthermore, in this example, if neither predefined matching nor generative AI analysis yields a viable result, the method can proceed to the personalization module 330. The personalization module 330 acts as a kind of recommendation engine, leveraging the user's historical data and past preferences to generate an appropriate set of features. This is particularly useful for vague commands and direct recommendation requests, such as "Recommend a color theme that suits me today." By analyzing the user's past color choices and usage patterns, the personalization module 330 can suggest a customized set of features tailored to individual preferences. This personalized option is applied to the device, enabling it to provide an appropriate response even when the user's intent is not clearly indicated in the initial command.

[0341] For example, according to one embodiment, the method may include a hierarchical multi-stage process for interpreting a user's color command and ensuring accuracy to satisfy the user's intent. This process can begin with a preprocessing module that attempts to match the user's command with a predefined logic-based color set through keyword analysis. If a sufficiently accurate match is found, the color set can be applied to the home appliance (S23). If this initial attempt fails, the command is escalated to a theme module, which can use a generative AI model to interpret the user's abstract or creative intent and generate a corresponding new color set (S32).

[0342] Furthermore, if this AI-generated function set also fails to meet the required accuracy threshold, further attempts (S34 and S42) may be included in which the personalization module generates a color set based on the user's historical data and preferences to verify whether the personalization is appropriate. If a successful result is obtained in any of these three stages (e.g., pre-definition, AI generation, or personalization), the function can be applied to the home appliance. Otherwise, if all attempts fail, or if one or more attempts fail, this method may include prompting the user to re-enter the command to avoid misinterpretation (S45). Furthermore, if this AI-generated function set also fails to meet the required accuracy threshold, further attempts (S34 and S42) may be included in which the personalization module generates a color set based on the user's historical data and preferences to verify whether the personalization is appropriate. If a successful result is obtained in any of these three stages (e.g., pre-definition, AI generation, or personalization), the function can be applied to the home appliance. Otherwise, if all attempts fail, or if one or more attempts fail, this method may include prompting the user to re-enter the command to avoid misinterpretation (S45).

[0343] According to other embodiments, this method may include prompting the user to re-enter a command after any or all of the steps (for example, S45 may follow one or more of S24, S33, S34, and S43). For example, according to one embodiment, this method can improve the accuracy and relevance of the output of the generated AI by employing a contextual prompting process (e.g., one-shot learning or multi-shot learning). When the theme module interprets the user's command, it may provide a detailed prompt that includes, in addition to the user's request, one or more examples of different commands and their corresponding successful feature sets. This allows the AI ​​module to obtain a clear model of the expected output format and logic. To further refine the results, the prompt can be supplemented with additional context, such as the user's geographical region to consider cultural color associations, or device-specific parameters, such as the number of color panels on a refrigerator, to ensure that the generated feature set is contextually appropriate and physically compatible with the target device.

[0344] Furthermore, according to one embodiment, the method can incorporate a dynamic learning mechanism that improves efficiency and personalization over time. If the theme module successfully generates a new set of features that the user applies and maintains, this can be recognized as the user's preference. The newly created association between an abstract command (e.g., Iron Man colors) and a specific set of features (e.g., a list of hexadecimal color codes) can be stored in a user-specific database. In this way, the next time the user issues the same command, the system can bypass the generation AI process. Instead, a preprocessing module or personalization module can immediately retrieve the stored set of features from the database. The set of features can include a set of colors, a set of sounds, or a set of functions. If the set of features is a set of colors, and the feature command is a color command, and the color command includes the color names of the colors that make up the first set of colors, the first operation may include the server or home appliance determining that the first precision corresponds to a baseline value.

[0345] Here, the second operation may include the server or home appliance inputting a sample feature command and a sample feature set corresponding to the sample feature command into the theme module to generate a second feature set and a second accuracy. If the feature set is a color set, the feature command is a color command, and the personalization information corresponding to the device information may include information about one or more of the following for the home appliance corresponding to the device information: a pre-configured color set, the frequency of the configured color set, or the maintenance time for the configured color set.

[0346] This method may further include the server or home appliance storing function commands and the second function set in a database according to the maintenance time or the number of times the second function set has been applied, and the server or home appliance setting the information stored in the database as the first function set or the third function set during the process of generating the first function set or the third function set in response to the input of a function command after the fourth operation.

[0347] This method may further include the server or home appliance receiving a fine-tuning command for any one of the first, second, or third function sets to generate a fourth function set, storing the function command and the fourth function set in a database, and, while generating a function set in response to a function command entered after the fourth operation, the server or home appliance setting the fourth function set stored in the database to the home appliance.

[0348] This method may further include the server or appliance applying one of the first, second, or third function sets to other appliances connected to the appliance's user account. This method may further include the server or appliance inputting at least one of the following into a theme module: the type of appliance targeted by the function command, the appliance's configuration or status information, the range of functions configurable on the appliance, and a sample control command.

[0349] A function command is a command that instructs a first appliance to change its color, and the first function set contains information about one or more colors to be set on the first appliance. The setting information contains information about the area of ​​the first appliance to be set with one or more colors, and the status information contains information about the second function set currently or previously set on the first appliance. The result contains two or more candidate function sets, and the method includes the processor applying one or more of the setting information, status information, and function commands to convert one of the two or more candidate function sets into the first function set.

[0350] This method may further include the processor sending a first set of functions to a first household appliance, or sending a function control command to the first household appliance to apply the first set of functions to the first household appliance.

[0351] To summarize an exemplary embodiment, when applying (or updating, changing, or implementing) a set of features (e.g., color, sound, function) to a home appliance, the user provides / inputs a feature command (i.e., a voice command and / or text command) to the appliance. The feature command includes information about the set of features to apply (e.g., different words). If the feature command entered / provided by the user is a voice command, the words in the voice command may optionally be converted / processed into text. This step is optional even if the feature command entered / provided by the user is a voice command.

[0352] The home appliance 100 and / or server 500 include one or more standardized function commands and a set or database of function sets corresponding to each standardized function command (e.g., multiple standardized function commands and multiple corresponding function sets). Each standardized function command includes a pre-configured, standardized, default, or predefined word. The home appliance / server compares the function command entered / provided by the user with the standardized function command. Specifically, it compares the word in the function command entered / provided by the user with the standardized word in the standardized function command. The result of the comparison is either a match / duplicate (partial or complete) or no match. In other words, there is a match between the function command entered / provided by the user and one of the standardized function commands based on the above comparison. The function set (of the home appliance 100 and / or server 500 or database) corresponding to the matched (partial or complete) standardized function command is determined as the first function set.

[0353] If a match is found, the degree / level of the match, i.e., the match accuracy, is determined. This is called the initial accuracy, or the initial accuracy corresponding to the initial feature set. For example, if all words match a standardized word specified in one of the standardized feature commands, the initial accuracy is considered to be 100% (scales other than percentages, such as a scale from 1 to 5, or scales using quantifiers / modifiers / parameters such as "low," "medium," and "high," can also be used). If some words match a standardized word, the initial accuracy is considered to be 60%, 80%, etc. (depending on the degree of match / overlap). In this way, the first feature set and the first accuracy are determined based on a comparison between the function commands entered / provided by the user and one of the standardized function commands of the home appliance or server. This is called the first operation. If the first accuracy of the first feature set is higher than a baseline or threshold (e.g., the first baseline), the home appliance or server implements or applies the first feature set to the home appliance. This is called the fourth operation, the final operation, or the implementation operation.

[0354] On the other hand, if the first precision of the first feature set is below a baseline or threshold (for example, the first baseline is 50 percent or more), or if there is no match, i.e., the first feature set is not determined or recognized, a second operation is performed. In the second operation, the function command entered / provided by the user is provided to a theme module (e.g., LLM), and the theme module generates a corresponding function command. The theme module is stored locally (on appliance 100) or on server 500 or another server. In other words, the theme module generates a corresponding function command based on the words in the function command entered / provided by the user. The corresponding function command generated by the theme module may include alternative, similar, or suggested words to the words in the function command entered / provided by the user.

[0355] The function commands generated by the theme module are compared to the function commands standardized by the appliance / server. Specifically, the words of the function commands generated by the theme module (e.g., generated words) are compared to the standardized words of the standardized function commands. The result of the comparison is either a match / duplicate (partial or complete) or no match. In other words, based on the above comparison, one of the standardized function commands matches the function command generated by the theme module. The function set (a set or database of appliance 100 and / or server 500) corresponding to the matched (partial or complete) standardized function command is determined as the second function set.

[0356] If a match is found, the degree / level of the match, i.e., the accuracy of the match, is determined. This is called the second-order accuracy, or second-order accuracy corresponding to a second set of features. For example, if all words (such as generated words) match one standardized word in a standardized feature command, the second-order accuracy can be said to be 100 percent (other scales can be used besides percentages, such as a scale from 1 to 5, or scales using quantifiers / modifiers / parameters such as "low," "medium," and "high"). Also, if some words (such as generated words) match a standardized word, the second-order accuracy can be said to be 60 percent or 80 percent, etc. (depending on the degree of match / overlap).

[0357] In this way, a second set of functions and a second precision are determined based on a comparison between the function commands generated by the theme module and one of the standardized function commands of the home appliance or server. This is called the second operation. If the second precision of the second set of functions is higher than a reference value or threshold (e.g., second reference value), the home appliance or server implements or applies the second set of functions to the home appliance. This is called the fourth operation, final operation, or implementation operation.

[0358] On the other hand, if the second accuracy of the second feature set is below a baseline or threshold (for example, the second baseline is 50 percent or more), or if there is no match, i.e., the second feature set is not determined or recognized, a third operation is performed. In the third operation, the function commands input / provided by the user are provided to a personalization module, such as a predictive model or recommendation AI, which is stored locally (i.e., on the home appliance 100) or on server 500 or another server, and the corresponding function commands are generated by the personalization module. The personalization module includes personalization information such as the user's preferences previously stored / provided by the user, or the user's past usage history of the home appliance or other home appliances, and / or the user's personal data, and can generate corresponding function commands based on this information. The personalization module is also called a user preference prediction module.

[0359] In other words, the personalization module generates corresponding function commands based on the words in the function commands entered / provided by the user, and / or the words in the function commands generated by the theme module, as well as personalization information. The corresponding function commands generated by the personalization module may include alternative, similar, or suggested words to the words in the function commands entered / provided by the user. The function commands generated by the personalization module are generated so that one of the standardized function commands contains words that match standardized words.

[0360] The function commands generated by the personalization module are compared with the function commands standardized by the appliance / server. As a result of this comparison, a third set of functions, i.e., a set of functions (for appliance 100 and / or server 500 or the database), is determined that corresponds to the matching standardized function commands. Thus, the third set of functions is determined based on a comparison between the function commands generated by the personalization module and either the standardized function commands of the appliance or the server. This is called the third operation. The third set of functions is then implemented or applied to the appliance by the appliance or the server. This is called the fourth operation, final operation, or implementation operation.

[0361] Although it has been stated that all components constituting embodiments of the present invention are combined into one or operate in combination, the present invention is not necessarily limited to these embodiments, and any components within the scope of the present invention can also be selectively combined into one or more components and operate. Furthermore, although each component can be embodied in a single independent piece of hardware, some or all of the components can be selectively combined to be embodied as a computer program having program modules that perform some or all of the combined functions in one or more pieces of hardware. The code and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. These computer programs can embody embodiments of the present invention by being stored on a computer-readable medium and read and executed by a computer. Storage media for computer programs include magnetic recording media, optical recording media, and storage media containing semiconductor recording elements. Furthermore, computer programs embodying embodiments of the present invention include program modules that are transmitted in real time via an external device.

[0362] The above description has focused on embodiments of the present invention, but various modifications and variations can be made at the level of an ordinary engineer. Therefore, as long as these modifications and variations do not deviate from the scope of the present invention, they can be understood to be included within the scope of the present invention.

Claims

1. A server or consumer electronics device calculates a first feature set and a first accuracy corresponding to a feature instruction word in a first stage. If the first accuracy is below a certain standard, the server or the home appliance performs a second step of calculating a second feature set and a second accuracy corresponding to the feature instruction word using a theme module located in at least one of the server, the home appliance, or an external server. If the server or the home appliance is below the second accuracy threshold, the third step involves calculating a third feature set using the personal information corresponding to the appliance information and the feature instruction words, and A fourth step is to change any of the feature sets of the first, second, or third feature sets to features of the home appliance. including, How to change the features of home appliances.

2. The aforementioned feature set includes either a color set, a speech set, or a function set. A method for changing the features of a home appliance as described in claim 1.

3. If the feature set is a color set, and the feature instruction is a color instruction, and the color instruction includes the hue name of a color constituting the first color set, the first step includes the step in which the server or the home appliance determines that the first accuracy meets the criteria. A method for changing the features of a home appliance as described in claim 2.

4. The second stage described above is, The server or home appliance includes the step of inputting a sample feature instruction and a sample feature set corresponding to the sample feature instruction into the theme module, and calculating the second feature set and the second accuracy corresponding to the feature instruction, A method for changing the features of a home appliance as described in claim 1.

5. If the feature set is a color set, the feature instruction is a color instruction, The personal information corresponding to the device information includes information about the color set previously set on the home appliance corresponding to the device information, and information about the frequency of the set or the duration of the set, or any of the above. A method for changing the features of a home appliance as described in claim 2.

6. The server or home appliance further includes the step of inputting information about the region or country to which the user who uttered the feature command belongs into the theme module. A method for changing the features of a home appliance as described in claim 1.

7. The feature set includes one or more feature information and information about the area or function of the device in which the feature information is set. A method for changing the features of a home appliance as described in claim 1.

8. The server or home appliance further includes the step of storing the feature instruction words and the second feature set in a database according to the maintenance time or number of times the second feature set is applied, The server or home appliance, in the process of calculating the first feature set or the third feature set for a feature instruction word input after the fourth step, further includes the step of setting the information stored in the database into the first feature set or the third feature set. A method for changing the features of a home appliance as described in claim 1.

9. The server or the home appliance receives a fine adjustment command for any of the first feature set, the second feature set, or the third feature set and calculates a fourth feature set. The steps include storing the aforementioned feature instruction and the fourth feature set in a database, and The server or the home appliance, in the process of calculating a feature set for a feature instruction word input after the fourth stage, sets the fourth feature set stored in the database to the home appliance. This also includes, A method for changing the features of a home appliance as described in claim 1.

10. The server or the home appliance further includes the step of applying any of the first feature set, the second feature set, or the third feature set to other home appliances linked to the user account of the home appliance. A method for changing the features of a home appliance as described in claim 1.

11. The server or home appliance further includes the step of causing the theme module to input one or more of the following: the type of home appliance targeted by the feature instruction word, the configuration information or state information of the home appliance, the range of features that can be set on the home appliance, and information regarding sample control instructions. A method for changing the features of a home appliance as described in claim 1.

12. It includes one or more processors that process information received from home appliances, input it into a theme module which is a generative AI model, and obtain the result. The aforementioned processor, In the step of receiving a feature instruction word from the first home appliance or the second home appliance that instructs a feature change of the first home appliance, A step of generating a prompt that includes the feature instruction and configuration information of the first home appliance corresponding to the feature instruction, The steps include: inputting the aforementioned prompt into the theme module to obtain the result, and The process includes converting the above result into a first feature set to be configured in the first home appliance, How to change the features of home appliances.

13. The prompt further includes status information of the first home appliance, A method for changing the features of a home appliance as described in claim 12.

14. The second home appliance is characterized by being either a user terminal or a home appliance other than the first home appliance. A method for changing the features of a home appliance as described in claim 12.

15. The aforementioned feature instruction is an instruction that instructs the first home appliance to change its color. The first feature set includes information about one or more colors to be set on the first consumer electronics device. A method for changing the features of a home appliance as described in claim 12.

16. The configuration information is information relating to the area of ​​the first home appliance in which one or more colors are set. The status information includes information about a second feature set that is currently set or has been set in the first home appliance. A method for changing the features of a home appliance as described in claim 15.

17. The above results include two or more candidate feature sets, The processor includes the step of applying one or more of the configuration information, the state information, and the feature instruction words to convert one of the two or more candidate feature sets into the first feature set, A method for changing the features of a home appliance as described in claim 12.

18. The processor includes the steps of transmitting the first feature set to the first consumer electronics device or transmitting a feature control instruction to the first consumer electronics device to apply the first feature set to the first consumer electronics device. A method for changing the features of a home appliance as described in claim 12.

19. In a storage medium installed in a server or home appliance, The storage medium includes a pre-processing module and a personalization module, and includes either a theme module or a counter theme module. The preprocessing module calculates a first feature set and a first accuracy corresponding to the feature instruction word. If the first accuracy is below a certain threshold, the theme module calculates a second feature set and a second accuracy corresponding to the feature instruction word. If the first accuracy is below a certain threshold, the counter theme module transmits the feature instruction to the theme module of the external server of the generative AI, and receives the second feature set and the second accuracy from the theme module. If the second accuracy is below a certain threshold, the personalization module calculates a third feature set using the personalization information corresponding to the device information of the home appliance and the feature instruction words. The storage medium controls the home appliance to set the features of the home appliance. Storage medium.

Images