Method for controlling home appliance on basis of instruction and apparatus implementing same

Abstract

The present invention relates to a method for controlling a home appliance on the basis of an instruction and an apparatus implementing same. The method for controlling a home appliance on the basis of an instruction, according to an embodiment of the present invention, comprises: a first step of a server or a home appliance checking whether an input instruction corresponds to direct control; a second step of, when the input instruction does not correspond to direct control in the first step, the server or the home appliance extracting, from a database, an example sentence having a similarity to the input instruction greater than or equal to a certain criterion, and inputting same into a generative AI module; a third step of, when the input instruction corresponds to post-interpretational control on the basis of a result produced by the generative AI module, the server or the home appliance generating a control command by using the produced result; and a fourth step of the home appliance providing a function according to the generated control command.

Description

Method for controlling home appliances based on commands and device implementing the same

[0001] The present invention relates to a method for controlling a home appliance based on a command and a device for implementing the same.

[0002] Control methods for devices like home appliances can be divided into direct control through human intervention and natural language command control. In direct control, the user can control the operation of the appliance by operating the remote control, buttons on the appliance, or dial. In natural language control, the user inputs natural language commands to the appliance, which are then recognized and operated.

[0003] However, natural language commands spoken or entered by users are based on individual differences in natural language, making interpretation and translation into actual commands challenging. In particular, accuracy in interpreting natural language commands is required, depending on each user's past command input habits and the device's usage environment.

[0004] Accordingly, this specification describes a method and device for interpreting the intent of a command uttered by a user, generating a corresponding control command, and controlling a home appliance using the same.

[0005] This specification aims to solve the aforementioned problems and to flexibly recognize user commands by improving speech recognition errors.

[0006] Additionally, this specification allows for the simultaneous operation of various controls when processing user commands.

[0007] Additionally, the present specification allows a user to control home appliances through voice recognition even when the user does not use a set of commands.

[0008] The objectives of the present invention are not limited to those mentioned above. Other objectives and advantages of the present invention not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present invention. Furthermore, it will be readily apparent that the objectives and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0009] A method for controlling a home appliance based on a command according to one embodiment of the present invention includes a first step in which a server or a home appliance determines whether an input command corresponds to direct control; a second step in which, if the input command in the first step does not correspond to direct control, the server or the home appliance extracts an example sentence having a similarity level higher than a certain standard with the input command from a database and inputs the example sentence into a generative AI module; a third step in which, if the input command corresponds to post-interpretational control in the result produced by the generative AI module, the server or the home appliance generates a control command using the produced result; and a fourth step in which the home appliance provides a function according to the generated control command.

[0010] A server for controlling a home appliance based on a command according to one embodiment of the present invention includes a server control unit for determining an input command and generating a corresponding control command, a database in which a plurality of example sentences are stored, and a server communication unit for communicating with the home appliance, wherein after the server communication unit receives a command input into the home appliance from the home appliance, the server control unit determines whether the command input into the home appliance corresponds to direct control, and if the input command does not correspond to direct control as a result of the determination, extracts example sentences having a similarity level higher than a certain standard with the input command from the database and inputs the example sentences to a generative AI module, and if the input command corresponds to post-interpretational control from the result produced by the generative AI module, the server control unit generates a control command using the produced result, and the server communication unit transmits the control command to the home appliance so that the home appliance can provide a function according to the generated control command.

[0011] When the present invention is applied, speech recognition errors can be improved, and user commands can be flexibly recognized.

[0012] When the present invention is applied, various controls can be operated simultaneously when processing a user's command.

[0013] When the present invention is applied, home appliances can be controlled through voice recognition even when the user does not use a set command.

[0014] The effects of the present invention are not limited to the effects described above, and various effects of the present invention can be easily derived from the configuration of the present invention.

[0015] FIG. 1 is a drawing showing a process in which a home appliance operates according to a command input according to one embodiment of the present invention.

[0016] FIG. 2 is a drawing showing a process in which a home appliance transmits a command to a server when a command is input according to another embodiment of the present invention, and then receives a control command from the server to operate the home appliance.

[0017] Figure 3 is a drawing showing the configuration of a server according to one embodiment of the present invention.

[0018] FIG. 4 is a drawing showing the classification of input commands according to one embodiment of the present invention.

[0019] FIG. 5 is a diagram showing a process in which a server processes an input command according to one embodiment of the present invention.

[0020] Fig. 6 is a drawing showing the structure of a prompt according to one embodiment of the present invention.

[0021] Fig. 7 is a drawing showing exemplary contents of a base according to one embodiment of the present invention.

[0022] FIG. 8 is a diagram showing a flow for interpreting an input command using a generative AI module according to one embodiment of the present invention.

[0023] FIG. 9 is a drawing detailing the operation process of a server according to one embodiment of the present invention.

[0024] Fig. 10 is a drawing showing the configuration of a home appliance according to one embodiment of the present invention.

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present invention. The present invention may be implemented in various different forms and is not limited to the embodiments described herein.

[0026] In order to clearly explain the present invention, parts that are not related to the description have been omitted, and the same or similar components are designated by the same reference numerals throughout the specification. In addition, some embodiments of the present invention will be described in detail with reference to exemplary drawings. When adding reference numerals to components in each drawing, the same components may have the same numerals as much as possible even if they are shown in different drawings. In addition, when explaining the present invention, if it is determined that a specific description of a related known configuration or function may obscure the gist of the present invention, the detailed description may be omitted.

[0027] When describing components of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only intended to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by the terms. When it is described that a component is "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that other components may also be "interposed" between each component, or that each component may be "connected," "coupled," or "connected" through another component.

[0028] In addition, in implementing the present invention, components may be described in detail for convenience of explanation, but these components may be implemented in one device or module, or one component may be implemented by being divided into multiple devices or modules.

[0029] Hereinafter, the home appliances described herein are devices containing electronic products. Home appliances can be placed in homes, offices, and other locations, and can be moved and placed in other locations by people.

[0030] The device described herein, for example, a home appliance or server device, preprocesses a voice command or text command input by a user, classifies the input command into a specific category, interprets the input command according to the classified category, and controls the home appliance so that the home appliance can operate in response to the interpreted result.

[0031] A home appliance or server device (server) or a system comprising one or more devices may store hardware or software for processing feature commands. Furthermore, a home appliance or server device (server) or a system comprising one or more devices may receive and execute software for processing feature commands from a remote third-party device. Processing of feature commands using hardware embedded in the device, software stored within the device, or software that can be received and executed may be performed by a processor within each device. Alternatively, these hardware or software themselves may operate as processors.

[0032] To this end, the processor may store software or program code capable of performing the aforementioned tasks. Such software or program code may be received from another external device and stored on a storage medium used by the processor, after which the processor may execute the software or program code.

[0033] Additionally, the processor may include hardware components such as programmable chips, and the processor may store data or program codes to be input into the hardware components in a predetermined storage medium and then input them into the hardware components.

[0034] The hardware or software may be the processor itself. Alternatively, the hardware or software may work in conjunction with the processor to implement embodiments of the present invention.

[0035]

[0036] FIG. 1 is a drawing showing a process in which a home appliance operates according to a command input according to one embodiment of the present invention.

[0037] A user (1) inputs a predetermined command (voice or text) into a home appliance (100) (S3). The command can be input by voice (VOICE) or text. The home appliance (device) (100) determines a control command corresponding to the command based on the result of preprocessing by performing speech-to-text conversion on the input command (or input command) (S5). Once the control command is determined, the home appliance (100) executes the determined control command to perform a predetermined function (S7).

[0038]

[0039] FIG. 2 is a drawing showing a process in which a home appliance transmits a command to a server when a command is input according to another embodiment of the present invention, and then receives a control command from the server to operate the home appliance.

[0040] S3 refers to FIG. 1. The home appliance (100) transmits the input command to the server (500) (S11). At this time, if the input command is a voice command, the home appliance (100) can convert the voice command into text and transmit the command, which is the result of preprocessing the text, to the server (500).

[0041] In this case, the server (500) can determine a control command corresponding to the received command (S15). The preprocessing process may also be performed in the server (500). In this case, the home appliance (100) can transmit the input command as is to the server (500).

[0042] When a control command is determined, the server (500) transmits information about the determined control command to the home appliance (100) (S16), and the home appliance (100) executes the received control command to perform a predetermined function (S17).

[0043] As seen in FIGS. 1 and 2, the server (500) or the home appliance (100) can produce an appropriate control command for the preprocessed command.

[0044] Hereinafter, the description will focus on the server (500) performing tasks for command recognition, but the present invention is not limited thereto, and the home appliance (100) may also provide some or all of the functions provided by the server (500). Furthermore, for convenience of explanation, the description will focus on an air conditioner as an example of a home appliance (100), but the present invention is not limited thereto.

[0045] Below, a device for implementing embodiments of the present invention will be described.

[0046] Embodiments of the present invention can be implemented in various devices. Devices include servers, home appliances, electronic devices, computing devices, and the like. Furthermore, in addition to physical devices, the devices of the present invention may include hardware or software components that perform embodiments of the present invention. Furthermore, embodiments of the present invention include programs, hardware, chips, and the like, implemented in a form capable of storing or executing certain tasks.

[0047] In the case of programs, software, etc., they may be stored permanently within the device, or they may be temporarily transmitted from an external source, stored in the device, and then executed. In the case of a fixed storage method, the device may include a non-transitory computer-readable medium.

[0048] That is, embodiments of the present invention may be implemented as computer-readable storage media as one or more computer programs, or a combination of one or more of the above.

[0049] The functions of the elements disclosed herein may be implemented using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, ASICs ("application-specific integrated circuits"), conventional circuits, and / or combinations thereof. The circuits may be processors configured or programmed to perform the disclosed functions. A processor may be considered a processing circuit or circuits, as it includes transistors and other circuits.

[0050] In this specification, a circuit, unit, or means may be hardware that performs or is programmed to perform the functions mentioned in the detailed description. The hardware may be hardware disclosed herein or other known hardware, and may be hardware programmed or configured to perform the functions mentioned in the detailed description of the specification. If the hardware is a processor that can be considered a type of circuit, the circuit, means, or unit may be a combination of hardware and software used to configure the hardware and / or the processor. In addition, the computer storage medium may be a non-transitory computer readable medium. For example, it may be executable by a cloud server-based system. The computer storage medium may be located within the same device or may be distributed across two or more different devices. Therefore, a logical computer storage medium may physically include two or more computer storage media, and the locations where they are located may also be one or more locations. The computer storage medium includes various storage media such as hard disks, CD / DVD disks, memory cards, and memory chips.

[0051] Furthermore, the data described in this specification can be computed or performed in various environments, including cloud server-based systems, on-device systems, and distributed server systems (multiple servers). Processing can be distributed on cloud servers or executed locally on on-device processors, and the results of processing in each environment can be stored in non-volatile memory.

[0052]

[0053] Electronic devices, such as home appliances and server computing devices, may be connected to one or more storage devices via a network. The storage devices may be a combination of volatile and non-volatile memory, and may or may not be located in the same physical location as the computing device.

[0054] A server computing device may include one or more processors and memory. The memory stores information that can be accessed by the processor and may include data that can be processed, stored, or modified by instructions that can be executed by the processor. The memory may also be comprised of volatile and non-volatile memory. The processor may include a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a tensor processing unit (TPU).

[0055]

[0056] Instructions can be configured to cause a processor to perform specific actions when performing a directed task, and can be stored in object code or interpretable script format. These instructions can be used to implement a system and can be executed on a local or remote processor. Data can be retrieved, stored, or modified according to the instructions, and can be organized in database, JSON, YAML, or XML formats.

[0057] These commands may contain executable files, source code files, or metadata.

[0058]

[0059] An electronic device, including a home appliance or hub, may be configured similarly to a server computing device. The electronic device, including a home appliance or hub, may include a processor, memory, instructions, data, and user input and output devices. The server computing device may transmit data to the electronic device, and the electronic device may display a portion of the received data through a display. In addition, data transmission and communication between the server computing device and the electronic device is possible through a network such as Bluetooth, Wi-Fi, a wired network, or a wireless network, and various protocols and connection methods are supported. Direct and indirect communication between computing devices is possible, and various protocols and connection methods can be supported.

[0060] Additionally, the functions of the server computing device of the present invention can be performed by a smartphone, tablet, etc.

[0061] The methods or processes in the embodiments of this specification sequentially perform one or more tasks, and each task may be performed through the collaboration of either hardware or software, or both. For example, hardware may perform the first task, and software may perform the second task. Of course, hardware may perform the entire task, or software may perform the entire task.

[0062]

[0063] Figure 3 is a drawing showing the configuration of a server according to one embodiment of the present invention.

[0064] The server (500) includes a server control unit (550), a control example database (550), a situation example database (520), a control command database (530), and a server communication unit (590). Optionally, the server includes a generative AI module (300). That is, the generative AI module (300) may be implemented in the server (500) or in an external device.

[0065] The server control unit (550) determines which category or control command the input command corresponds to and generates a corresponding control command. During the determination process, the server control unit (550) may input certain information into the generative AI module (300).

[0066] The database (510, 520) can store a number of example sentences. In addition, among the databases, the situation example database (520) can store predefined routines.

[0067] The server communication unit (590) communicates with the home appliance (100). In addition, when the generative AI module (300) is placed in an external device (external server), the server communication unit (590) also communicates with the external device.

[0068] When the server communication unit (590) receives a command input to the home appliance (100) from the home appliance (100), the server control unit (550) checks whether the command input to the home appliance (100) corresponds to direct control.

[0069] If the verification result shows that the entered command does not correspond to direct control, the server control unit (550) extracts an example sentence whose similarity to the entered command is higher than a certain standard from the database (520) and inputs it into the generative AI module (300).

[0070] If the input command corresponds to direct control, the server control unit (550) can generate a control command corresponding to the input command without inputting the input command to the generative AI module (300).

[0071] The server control unit (550) generates a control command using the output result of the generative AI module (300) when the input command corresponds to post-interpretational control. Refer to the category description in Fig. 4 for this.

[0072] And, the server communication unit (590) transmits a control command to the home appliance so that the home appliance (100) can provide a function according to the generated control command, thereby allowing the server (500) to control the home appliance (100) based on the command.

[0073]

[0074] FIG. 4 is a drawing showing the classification of input commands according to one embodiment of the present invention.

[0075] Hereinafter, input voice commands, input text commands, or preprocessed utterances thereof are collectively referred to as input commands.

[0076] The input commands fall broadly into categories that fall under direct control (Category 1) or non-direct control (Categories 2, 3, and 4). That is, the input commands can be divided into those that require interpretation through a generative AI module (Categories 2, 3, and 4) and those that do not (Category 1).

[0077] Direct control refers to a case where the input command is directly matched to the control command required to control the home appliance. This refers to a case where the server (500) does not apply a separate interpretation process to the input command. An input command corresponding to direct control may correspond to one or more control commands.

[0078] If the input command belongs to the direct control category, the server (500) generates a control command necessary to control the home appliance (100) in response to the input command. For example, in case of responding to one control command such as “Turn on the air conditioner” or “Temperature 25 degrees” (consisting of a single utterance sentence), or in case of responding to two control commands such as “Cooling, 4-level wind speed” (consisting of a composite utterance sentence), the server (500) can control the home appliance (100) using the corresponding control command. That is, the server (500) can generate a control command in response to the input command (input command) corresponding to direct control and provide the control command to the home appliance (100).

[0079] The server (500) only determines whether the input command corresponds to direct control, and if it does not correspond to direct control, inputs it to the generative AI module (300). At this time, the server (500) includes a base, a guide, and an example sentence in the information (prompt) to be input to the generative AI module (300).

[0080] The base includes information that indicates the role that the generative AI module (300) must perform or the result that must be produced in order to interpret the input command.

[0081] The Guide includes information for distinguishing which category the input command belongs to (i.e., which of categories 2, 3, and 4 of FIG. 4 it belongs to) and components of a control command that the generative AI module (300) must produce in response to the information.

[0082] A sample text is an example sentence whose similarity to the input command is higher than a certain standard. The server (500) can extract N example sentences (N is a natural number greater than or equal to 1) whose similarity to the input command is higher than a certain standard from the database (510, 520) by utilizing the BM25 or Cosine similarity technique. In other words, the sample text is an example sentence whose similarity to the input command is higher than the standard based on the BM25 or Cosine similarity among the sentences stored in the database (510, 520).

[0083] The generative AI module (300) can produce a predetermined result based on a prompt input into the generative AI module (300), and the server (500) determines that the input command corresponds to one of categories 2 / 3 / 4 based on the produced result.

[0084] If the input command belongs to the second category, post-interpretational control, the server (500) can generate a control command necessary to control the home appliance using the produced result.

[0085] Meanwhile, if the input command is a post-interpretational routine, which is a third category, the server (500) can generate a control command corresponding to the routine. In this case, the routine can be stored in the situation example database (520), and the routine includes a common routine applied to all home appliances and a personalized routine applied to home appliances used by a specific user.

[0086] If the input command is a post-interpretational chat (category 4), the server (500) outputs the resulting result in voice or text. To enable the home appliance to output this, the server (500) can generate a control command so that the speaker or screen of the home appliance can output voice or text.

[0087] Categories 1, 2, and 3 all control the home appliance to perform specific actions. Therefore, input commands corresponding to categories 1, 2, and 3 are control utterance commands. However, there are cases where the user explicitly instructs the home appliance to operate and cases where the user implicitly instructs. If the server (500) cannot produce a control command matching the input command, the input command, base, guide, and example sentences can be input to the generative AI module (300) to confirm the user's intention.

[0088] That is, even if the user explicitly instructs a functional operation, if words or sentences unrelated to control are mixed in the input command or if sentences containing errors are included in the input command, the server (500) may not be able to produce a corresponding control command.

[0089] In this case, the server (500) can increase the accuracy of command processing by using the generative AI module (300).

[0090] When a user implicitly inputs a voice / text command to operate a home appliance, the server (500) can process this as a context-related command. The context-related commands can be divided into those defined / stored in the server (500) and those not defined.

[0091] That is, the server (500) stores routines, which are a bundle of functions provided by the home appliance (100), and even when a user instructs execution of such routines, if the user cannot clearly utter the name of the routine, the accuracy of command processing can be increased by using the generative AI module (300).

[0092] Additionally, the server (500) can interpret the user's intention even if it is not specified in the routine.

[0093] This specification performs a process in which a home appliance operates in response to an input command after the home appliance or server preprocesses a command inputted through the home appliance and the preprocessed command provides the information necessary for controlling the home appliance (Category 1). On the other hand, if the preprocessed command does not provide the information necessary for controlling the home appliance, an interpretation process for the input command is performed and then a process in which the home appliance operates using the information produced after the interpretation is performed (Categories 2, 3, 4).

[0094] In order for the generative AI module (300) to produce accurate results, the server (500) can extract example sentences that have a similarity level with the input command above a certain standard from the control example database (510) or the situation example database (520) and input them together with the input command into the generative AI module (300).

[0095] As a result, the generative AI module (300) interprets the input command using an example and produces a result accordingly. The server (500) produces a control command corresponding to the produced result and provides it to the home appliance (100).

[0096] The server (500) of the present invention can analyze and extract the user's intention when a voice / text command for a home appliance (100) such as an air conditioner is input using a generative AI module (300) and control the home appliance (100) so that it operates accordingly.

[0097] If the sentence spoken by the user (input command) corresponds to the previously set format, the server (500) can control the home appliance (100) in response to the input command without using a separate generative AI module (300).

[0098] Meanwhile, if the input command is different from the format set in the server (500), the generative AI module (300) is used to interpret it and obtain the corresponding result, but the intent / control for the result is provided as predetermined information (base and guide, etc.) to prevent errors in the generative AI module (300).

[0099] Additionally, you can control your appliances in a customized way using your personalized information about which commands are routine and which are not.

[0100] The server (500) selects similar commands as example sentences during the voice command processing process and provides them to a generative AI module (300) such as chatgpt, thereby increasing the accuracy of the results. The output of the generative AI module (300) also applies routine and non-routine types, so that the user's intention can be sufficiently reflected.

[0101]

[0102] FIG. 5 is a diagram showing a process in which a server processes an input command according to one embodiment of the present invention.

[0103] A home appliance (100) or a user terminal or remote control connected to the home appliance (100) receives a command in the form of voice or text. Then, the home appliance (100) or the user terminal transmits the input command (input command) to the server (500) (S21).

[0104] The server (500) checks whether there is a control command corresponding to the input command, and if there is a corresponding control command, it generates a control command (S22) and transmits it to the home appliance (100) (S23). As a result, the home appliance (100) operates according to the control command (S24).

[0105] On the other hand, if there is no control command corresponding to the input command (S31), the server (500) extracts example sentences similar to the input command from the database and generates a prompt including the base, side, example sentence, and input command (S32). For example, the example sentences are stored in the control example database (510) or the situation example database (520), and the server (500) can extract example sentences whose similarity to the input command is higher than a standard from each database (510, 520).

[0106] The server (500) can generate a prompt including the base, guide, example sentences, and input commands discussed above. The configuration of the prompt will be described later.

[0107] And the server (500) inputs the prompt to the generative AI module (300) (S33). If the generative AI module (300) is implemented in an external server that is a generative AI service provider, the server (500) can input the prompt to the generative AI module (300) through an API agreed upon in advance with the generative AI service provider. One embodiment in which the generative AI module (300) is implemented in the generative AI service provider includes ChatGPT. Alternatively, a generative AI module such as ChatGPT can be deployed within the server (500).

[0108] In response to the prompt input of S33, the generative AI module (300) generates and provides a result. That is, the generative AI module (300) returns a result in response to the prompt input from S33 (S34).

[0109] The generative AI module (300) may be implemented in a server (500) or a separate external server (generative AI service provider). The server (500) may include guidance in the prompt to interpret which category the input command belongs to.

[0110] The server (500) reviews the validity of the result (S35). If the result includes a control command or can generate a control command, i.e., if it is category 2, the server (500) generates a control command (S36a).

[0111] Meanwhile, if the result of the review does not include a control command and includes a routine, i.e., if it is category 3, the server (500) extracts information from the situation example database (520) in response to the result and generates a control command (S36b).

[0112] The server (500) transmits the control command generated in S36a or S36b to the home appliance (100) (S37). As a result, the home appliance (100) operates according to the control command (S38).

[0113] The function of the server (500) of Fig. 5 can be provided by a home appliance (100).

[0114] The process of Fig. 5 is briefly summarized as follows. The server (500) or home appliance (100) checks whether the entered command corresponds to direct control (S22, S31).

[0115] If the command entered in S22, S31 does not correspond to direct control (category 1), the server (500) or home appliance (100) extracts an example sentence having a similarity level higher than a certain standard with the entered command from the database (510, 520) and inputs it into the generative AI module (300) (S32, S33).

[0116] If the input command corresponds to post-interpretational control (category 2) in the result (S34) produced by the generative AI module (300), the server (500) or the home appliance (100) generates a control command using the produced result (S35a). Then, the home appliance (100) provides (operates) a function according to the generated control command (S37, S38).

[0117] If the input command, such as S22, corresponds to direct control (category 1), the server (500) or the home appliance (100) generates a control command corresponding to the input command, and the home appliance (100) provides a function according to the control command corresponding to the input command (S23).

[0118] If it does not fall under category 1 in S32, the server (500) or the home appliance (100) may input example sentences and input commands into the generative AI module (300). At this time, the server (500) or the home appliance (100) may extract example sentences having a similarity level higher than a standard with the input command from the database (510, 520) and input them into the generative AI module (300). For example sentences, refer to Tables 3 to 7 described below.

[0119] And, if the input command belongs to category 3, that is, if it corresponds to the execution of a routine stored in the server (500) or home appliance (100) as shown in Table 6 described later, the server (500) or home appliance (100) obtains a control command corresponding to the stored routine and controls the home appliance (100) through this.

[0120] That is, if the input command in the result produced by the generative AI module (300) corresponds to a post-interpretational routine (category 3), the server (500) or the home appliance (100) can generate a control command corresponding to the routine included in the produced result. To this end, the server (500) or the home appliance (100) can extract the control command corresponding to the pre-stored routine from memory or a database, etc.

[0121] The server (500) or home appliance (100) of the present invention improves voice recognition so that even if a user inputs a command containing a control function or a sentence related to a specific situation (=non-controlled utterance), the server (500) or home appliance (100) implements a GPT prompt so that the utterance text can be interpreted. In addition, since representative utterance learning data and a sample extraction algorithm, which are examples of example sentences, are used, the server (500) or home appliance (100) can interpret the intent of the command (or sentence) input by the user and recommend an appropriate function.

[0122] In particular, since the generative AI module (300) interprets the user's input commands, the user's intention to control the home appliance can be extracted even from complex speech sentences without a set format or form. In addition, the user can give a voice command related to his / her situation, and in this case, the home appliance can perform customized operation according to the user's situation. In addition, when a specific routine operation is set for the home appliance, the server (500) or the home appliance (100) can load a control command related to the routine corresponding to a sentence containing the user's situation and use it to control the operation of the home appliance (100).

[0123] In an embodiment of the present invention, if a sentence spoken or entered by a user corresponds directly to a control command for controlling a home appliance, the home appliance is controlled based on the corresponding control command. Otherwise, the home appliance is controlled based on the result interpreted by the generative AI module (300). As a result, the server or home appliance can quickly determine whether the user's command corresponds to direct control under Category 1 of FIG. 4, so voice / text control of the home appliance can be performed quickly.

[0124] In addition, if the user's command does not correspond to direct control of category 1, the server or home appliance obtains a control command corresponding to the input command by collaborating with the generative AI module (300), so that voice / text control of the home appliance can be performed accurately.

[0125] When using a generative AI module (300), the server (500) or home appliance (100) can process both utterances with control functions and uncontrolled utterances.

[0126] In the case of control function utterances, it can process complex utterance sentences that indicate various control commands, sentences mixed with non-control words, and short utterances that contain errors or mispronunciations during the recognition process (Tables 3 to 5).

[0127] Additionally, in the case of situation-aware speech, the generative AI module (300) can provide descriptions or interpretations of situations in predefined situations (set routines), situations registered in ThinQ routines, and undefined situations. Furthermore, speech for everyday chat can also be processed by the server (500) or home appliances (Table 6 or Table 7).

[0128] That is, by dividing the control commands for controlling the home appliance into intent control commands and slot control commands, the server (500) or the home appliance (100) can flexibly process the sentence-type commands entered by the user, extract the user's device control intention even from complex speech commands, and control the functions of the home appliance (100). The intent control commands and slot control commands will be described later.

[0129] In addition, by inputting example sentences and guides for undefined routines or situations into the generative AI module (300), the server (500) or home appliance (100) can combine two to three control commands for functions that the home appliance (e.g., air conditioner) can provide.

[0130] By interpreting the user's speech, it is possible to execute a pre-defined routine (e.g., a routine stored on the ThinQ server or app).

[0131] A control command for controlling a home appliance (100) is a command that controls the operation of the home appliance, i.e., the home appliance to provide a specific function. The control commands for home appliances are divided into intent control commands and slot control commands.

[0132] The intent control command indicates the upper concept of the function of the home appliance (100), and the slot control command indicates the detailed function of the intent control command. In order for the home appliance to operate, both the intent control command and the slot control command must be provided. In other words, the set of intent control commands and slot control commands indicates the function to be performed by the home appliance.

[0133] That is, intent control commands are high-level control commands, such as on / off, temperature control, and wind volume control, among the functions of home appliances. Slot control commands correspond to the detailed settings of these intent control commands. The detailed control commands for on / off include on (turn on) and off (turn off), and the slot control command that constitutes the intent control command called temperature control can be a specific temperature value. Home appliances perform their functions through a set of intent control commands and slot control commands.

[0134] For air conditioners, some of the intent control commands and slot control commands are listed in Table 1. The appliance can be turned on according to the intent control command called AIR_POWER_REQUEST and the slot control command called ON.

[0135] Intent control command Slot control command Power on / off (AIR_POWER_REQUEST) On or off Temperature control (AIR_TEMPERATURE_REQUEST) 15 - 30 Hour control (AIR_TIME_REQUEST) 10 minutes - 3 hours Wind volume / speed control (AIR_WIND_STRENGTH_REQUEST) 1 - 5 Wind direction control (AIR_WIND_DIRECTION_REQUEST) Left / right / up / down function (AIR_OPERATION_REQUEST) Cooling / blowing / dehumidifying / sleeping wind characteristic setting (AIR_WIND_SETTING_REQUEST) Indirect wind

[0136] For the oven, some of the intent control commands and slot control commands are listed in Table 2.

[0137] Intent control command Slot control command On_Off (OVN_POWER_REQUEST) On or Off Temperature control (OVN_TEMPERATURE_REQUEST) 180 - 250 Time control (OVN_TIME_REQUEST) 10 seconds - 1 hour Function (OVN_OPERATION_REQUEST) Warming / cooking / defrosting

[0138]

[0139] That is, the intent control command is a command corresponding to the user's intention to control the device, and the slot control command is the detailed content of this intention. Therefore, when the intent control command and the slot control command are combined, the home appliance can operate with a specific function. When the contents of Table 1 or Table 2 are input into the generative AI module (300) depending on the home appliance, the generative AI module (300) can produce the intent control command and the slot control command corresponding to the input command.

[0140] And the interpretation of the intent of the voice / text command input by the user can be performed by the generative AI module (300).

[0141] And, an example sentence can be input to the generative AI module (300) so that the generative AI module (300) can produce an intent control command and a slot control command corresponding to the input command.

[0142] In order to obtain accurate results through the generative AI module (300), it is necessary to generate a prompt.

[0143] FIG. 6 is a diagram showing the structure of a prompt according to one embodiment of the present invention. The name of information to be input into the generative AI module (300) is referred to as a prompt, but the present invention is not limited thereto. The server (500) or the home appliance (100) may extract intent control commands (Intent) and slot control commands (Slot) from the input commands by inputting text classification and few-shot learning based on prompting of the GPT LLM (Generative Pre-trained Transformer Large Language Model) to obtain necessary information from the generative AI module (300).

[0144] As previously discussed, the prompt (400) may be composed of a base, guide, example sentences, and input commands required for controlling a home appliance, and may include only some of these depending on the embodiment.

[0145] The base (410) includes information indicating the role to be performed or the output to be produced by the generative AI module (300) to interpret the input command. For example, a server (500) or a home appliance (100) can generate the base as follows. The base includes basic rules, specifically, the type of home appliance to be applied, the type of output to be produced (411), a description of each output to be produced (412), and limitations on the output (413).

[0146] The generative AI module (300) is instructed to generate results in the order of [1], [2], and [3] by specifying three fields for the results to be produced. See Fig. 7.

[0147] The Guide includes information for distinguishing which category an input command belongs to (i.e., which of categories 2, 3, and 4 in FIG. 4 it belongs to) and components of a control command that the generative AI module (300) should output in response to the information. In one embodiment, the Guide includes content defining a text classification method and intents / slots. The Guide may include descriptions or definitions of intent control commands and slot control commands. The Guide (420) may be written in various ways depending on the characteristics or implementation method of the generative AI module (300).

[0148]

[0149] Figure 7 is a diagram illustrating exemplary contents of a base according to one embodiment of the present invention. Each corresponding element is indicated by a reference number. Reference number 411 indicates the type of result to be produced and that the applicable device is an air conditioner. Reference number 412 defines three control-related results (whether a control command is present, an intent, and a slot). Reference number 413 instructs that the three contents must be included in the output.

[0150]

[0151] For example, a guide might describe the rules for each outcome, listing the intents and slots that must be filled in, and specifying what rules must be followed to fill in the content.

[0152] The structure of the guide can be composed of three areas. The three areas are, for example, a guide for outputs, a guide for outputs of intent control commands, and a guide for outputs of slot control commands.

[0153] The output guide provides guidance on whether the input command is intended for control or not. For appliances related to air conditioning, weather, situations, and emotions are considered "non-control," while power, temperature settings, and wind speed, which are related to air conditioning control, are considered "control."

[0154] The intent control commands and their descriptions presented in Table 1 form a set. Intent control commands are a superset of slots and can be instructed to select one of a specific category for the input command.

[0155]

[0156] The area guiding the content of slot control commands can indicate that different intent control commands should be processed separately. If the output is "control," slot control commands related to air conditioning control can be listed. These can be individually guided by the intent control command.

[0157] Intent Control Commands – Slot Control Command 1, Slot Control Command 2, etc.

[0158]

[0159] The guide may include sample text (Few-Shot learning). The sample text (430) is an example sentence that has a similarity level higher than a certain standard with the input command, and the server (500) can extract N example sentences (N is a natural number greater than or equal to 1) that have a similarity level higher than a certain standard with the input command from the database (510, 520) using the BM25 or Cos similarity technique.

[0160] The example sentences that become learning data are various utterances (representative utterances), and these can be stored in a control example database (510) or a situation example database (520).

[0161] The example sentences stored in the control example database (510) and the corresponding intent control commands and slot control commands are referenced in the following tables. Table 3 shows example sentences of complex utterances with two or more controlling functions (intent control commands). Table 4 shows example sentences containing words or sentences unrelated to control. Table 5 shows example sentences with some errors.

[0162]

[0163] Example sentence (speech sentence)INTENTSLOTLower the temperature to 26 degrees and increase the wind speed by 2 degreesAIR_TEMPERATURE_REQUEST[{'temperature': '26'}]AIR_WIND_STRENGTH_REQUEST[{'wind_speed': 'up'},{'option': '2'}]4-stage cooling wind speed operationAIR_OPERATION_REQUEST[{'mode_name': 'cool'},{'onoff': 'on'}]AIR_WIND_STRENGTH_REQUEST[{'wind_speed': '4'}]

[0164]

[0165] Example sentence (sentence)INTENTSLOTTurn on the air conditioner. It feels suffocating. Cool it down.AIR_POWER_REQUEST[{'onoff': 'on'}]AIR_OPERATION_REQUEST[{'mode_name': 'cool'},{'onoff': 'on'}]Say, set the temperature to 26 degrees.AIR_TEMPERATURE_REQUEST[{'temperature': '26'}]

[0166]

[0167] Sentence INTENTSLOTPlease set it to 24 degrees AIR_TEMPERATURE_REQUEST[{'temperature': '24'}]Set the wind strength to 1 AIR_WIND_STRENGTH_REQUEST[{'wind_speed': '1'}]Set the target temperature to 22 degrees AIR_TEMPERATURE_REQUEST[{'temperature': '22'}]Turn on air cleaning AIR_OPERATION_REQUEST[{'mode_name': 'airclean'},{'onoff': 'on'}]

[0168]

[0169] When example sentences such as those in Tables 3 to 5 are input and the aforementioned base and guide are input, the generative AI module (100) produces a result including an intent control command and a slot control command.

[0170] For example, if a user inputs a command such as “Make the temperature 23 degrees and make the wind strong,” the generative AI module (100) can produce the following result based on an example sentence such as Table 3 above.

[0171] [{"intent":"AIR_TEMPERATURE_REQUEST","slots":[{"temperature":"23"}]}]

[0172] [{"intent":"AIR_WIND_STRENGTH_REQUEST","slots":[{'wind_speed': 'up'}]}]

[0173] Meanwhile, example sentences that indicate routines in relation to specific situations are also divided into two types: predefined routines and non-predefined routines.

[0174] Predefined routines are predefined situations for home appliances, such as exercising, studying, getting a job, or going out. Users can instruct these routines. Table 6 provides examples of routines.

[0175]

[0176] Example situation: Customized driving (example) Exercise "I'm going to exercise at home" Direct wind On Desired temperature 22 degrees [{"intent":"AIR_EXERCISE_REQUEST", "slots":[{"wind_setting":"direct"},{"temperature":"22"}]}] "I'm going to do some home training" "I'm going to exercise because my body is sore" Study "Please help me focus on my studies" Low noise mode Indirect wind On [{"intent":"AIR_STUDY_REQUEST", "slots":[{"wind_setting":"indirect"},{"additional_one":"low_noise"}]}] "I'm going to study hard from now on" "Please help me focus on my studies" Sleep "I'm starting to feel sleepy. I'm going to sleep now" Mood light Off Sleep reservation [{"intent":"AIR_SLEEP_REQUEST", "slots":[{"additional_two":"light_off"},{"sleep_reservation":"6"}]}]"I'm tired so I can sleep, please help me sleep well""I'm sleepy"Server-specified routine"Turn on the outing routine"Acquire a specific routine action from the server

[0177]

[0178] Next, for an unspecified routine, the generative AI module (300) can output corresponding information.

[0179] Example sentence: Custom driving (example) "Keep the temperature and humidity appropriate for the cat" Desired air conditioning temperature: 24 degrees Wind speed: Low wind [{"intent":"AIR_GPT_RECOMMEND","slots":[{"mode_name":"cool"},{"temperature":"24"},{"wind_speed":"low"}] "There are a lot of people at home, so it's a bit stuffy" Cooling direct wind On Wind speed: Strong wind [{"intent":"AIR_GPT_RECOMMEND","slots":[{'mode_name': 'cool'},{'wind_setting': 'direct'},{'wind_speed': 'high'}]}]

[0180]

[0181] If the next input command is completely unrelated to controlling the home appliance, this falls under Category 4, "Chat after Interpretation," and the generative AI module (300) can output a result accordingly. For example, if the user inputs a command asking, "Who is the author who won the Nobel Prize in Physics this time?", the generative AI module (300) can output a result as follows.

[0182] [{'intent': 'CHAT_REQUEST', 'slots': [{'answer': 'The person who won the Nobel Prize in Physics this time is A'}]}]

[0183] That is, if the result produced by the generative AI module (300) for the input command corresponds to post-interpretational chat, the server (500) or the home appliance (100) generates a control command to output the produced result in voice or text. Thereafter, the home appliance (100) can output some of the produced results in voice or text.

[0184] In this case, the user can ask questions and receive answers as if talking to the home appliance (100).

[0185] As shown in Tables 3 to 7, example sentences stored in the database (510, 520) may be composed of a set of sentences for controlling home appliances and corresponding intent control commands and slot control commands.

[0186] Among these, Tables 3 to 5 can be stored in the control example database (510), and Tables 6 and 6 can be stored in the situation example database (520).

[0187] Alternatively, the example sentences in Tables 3 to 7 and the corresponding sets of intent control commands and slot control commands can be stored in a single database without such distinction.

[0188] Using various example sentences and routines stored on the server, the server (500) can generate control commands appropriate to the user's intent. While a user may instruct a home appliance to perform a specific function using voice or text commands, these commands may not always directly correspond to the control commands that control the home appliance (100).

[0189] A direct match corresponds to the direct control described in Figure 4 above. For example, to start an air conditioner, a user can enter a command including "turn on / turn on / turn on / turn on / start / start." However, a user can also input a command to the appliance to start the air conditioner, including words other than these.

[0190] For example, if a user types "Air conditioner, it's hot," the command doesn't include any of the various verbs we've discussed before, such as "turn on / turn on / turn on / turn on / start / start," and thus doesn't indicate a specific function of the air conditioner. However, since the user's intention is to turn on the air conditioner or increase the airflow, the user's intention is to direct the operation of the air conditioner's function.

[0191] The server (500) inputs example sentences into the generative AI module (300) so that the generative AI module (300) can interpret the commands even if the commands do not correspond to direct control. That is, the generative AI module (300) that receives example sentences corresponding to intent control commands and slot control commands that indicate functions provided by home appliances can also output intent control commands and slot control commands for the input commands.

[0192] Additionally, home appliances can operate according to various routines. For example, air conditioners may have study routines, exercise routines, etc., and wind speed, air volume, and target temperatures can be set accordingly. Similarly, ovens may have routines for reheating instant rice, boiling ramen, etc., and heating temperatures and times can be set accordingly.

[0193] And even if the command input by the user does not exactly correspond to the routine, when the server (500) inputs a situation example sentence into the generative AI module (300), the generative AI module (300) can produce a result by reflecting the situation example sentence, and the server (500) can generate a control command based on the routine if the produced result corresponds to a specific routine.

[0194] That is, the server (500) extracts example utterances that have a similarity level higher than a certain standard with the input command from the situational utterance database (510) and inputs them together with the input command into the generative AI module (300). As a result, the generative AI module (300) interprets the input command using the example utterances and produces a result accordingly. The server (500) produces a control command corresponding to a routine indicated by the produced result and provides it to the home appliance (100).

[0195] If the input command does not correspond to a predefined routine, the server (500) guides the generative AI module (300) to produce a result related to the undefined routine, so that the generative AI module (300) can produce a result accordingly.

[0196] The server (500) according to the embodiment of the present invention is a server that provides artificial intelligence services and may include a database (310, 320) that stores a customer's past personalized information or routine information.

[0197] Based on learning about a user's lifestyle, we can provide personalized space solutions. For example, if a user uses voice or text commands to say, "I studied well last week. Set up the same settings for me," the system can load the settings stored in the database (510, 520) and create a personalized environment.

[0198] In one embodiment of the present invention, the server (500) includes an embodiment of a server group that is a collection of various servers. Accordingly, the configuration of the server (500) can be easily modified by those skilled in the art.

[0199]

[0200] Fig. 8 is a diagram showing a flow for interpreting an input command using a generative AI module according to one embodiment of the present invention. It is a detailed embodiment of the embodiment of Fig. 5.

[0201] When a command is input to a home appliance (100), the server (500) receives it (S21). Then, the server (500) checks whether the control command corresponds to it as is (S31 / S32). If it does not correspond as is, the server generates a prompt as seen in FIG. 5 and inputs it to the generative AI module (300) (S33). The server (500), which receives the result produced by the generative AI module (300) (S34), reviews the validity of the result, performs S36a / S36b depending on the type of the result, and then transmits the control command to the home appliance (100). Then, the home appliance (100) provides (operates) a function according to the control command (S38).

[0202]

[0203] Figure 9 is a drawing detailing the operation process of a server according to one embodiment of the present invention. The process of S35 will be examined in detail.

[0204] The server control unit (550) of the server (500) checks whether the intent control command and slot control command included in the result produced by the generative AI module (300) are valid (S41). This can be checked using the intent control command and the corresponding slot control command as shown in Table 1 or Table 2. For example, the server control unit (550) determines that the result is valid if the intent control command produced in relation to the air conditioner is AIR_POWER_REQUEST and the slot control command is "ON."

[0205] On the other hand, if the intent control command generated in relation to the air conditioner is AIR_POWER_REQUEST and the slot control command is “5 minutes,” the generated result is incorrect, so the server control unit (550) can instruct the home appliance (100) to request the command to be input again.

[0206] Alternatively, the server control unit (550) can check whether the produced result indicates a routine, and if it indicates a routine, can perform a validity check by checking whether it corresponds to a stored routine.

[0207] After completing the validity check of S41, the server control unit (550) verifies whether the generated result is a predefined routine (S42). If it is a predefined routine, the server control unit (550) obtains the routine stored in the database (S43) and controls the home appliance (100) according to the stored routine.

[0208] Meanwhile, if it is not a predefined routine, the server control unit (550) generates a control command in response to the intent control command and the slot control command and transmits the control command to the home appliance (100) to control the home appliance (100) (S45).

[0209] The command (Voice, Text) entered by the user goes through a predetermined preprocessing process in the server (500) or home appliance (100). If the entered command corresponds directly to a word or sentence that has been predefined and stored to control the home appliance, the server (500) or home appliance (100) converts the entered command into a control command.

[0210] Meanwhile, if interpretation is required for the input command, the server (500) or the home appliance (100) performs a command interpretation process. In this process, generative AI can be utilized, and the server (500) or the home appliance (100) can include a generative AI module. Alternatively, the server (500) or the home appliance (100) can interpret the input command using a generative AI module deployed on an external server.

[0211] The generative AI module (300) first interprets the input command to determine whether 1) it is a command related to controlling a home appliance or not, and 2) if it is a command related to controlling a home appliance, it can use example sentences or guides provided by the server (500) to produce an intent control command and a slot control command. Alternatively, the generative AI module (300) produces a result instructing the execution of a specific routine stored in the server (500).

[0212] A specific routine is a case where an input command is a command that indicates a predefined situation, i.e., a routineized command, and the server (500) or home appliance (100) can obtain intent control commands and slot control commands related to the predefined situation from a database.

[0213] For example, if the predefined routine is “exercise,” even if the user says the word “home training” or “running” or “cycle,” the generative AI module (300) interprets it as “exercise,” and the server (500) or home appliance (100) can execute the function of the exercise routine.

[0214] Additionally, personalized routines can also be stored. Even if a user sets up their own routine on the appliance (100) and enters the words for that routine, these routines can be stored on the appliance (100) or the server (500). Furthermore, if a user speaks a specific word to invoke this personalized routine, the generative AI module (300) can interpret it.

[0215] If the command is a command that indicates a situation that is not predefined, i.e., a non-routine command, the generative AI module (300) can extract the most similar command among the pre-stored routines, or generate an intent control command and a slot control command based on the interpretation result of the input command.

[0216]

[0217] Fig. 10 is a drawing showing the configuration of a home appliance according to one embodiment of the present invention.

[0218] It is similar to the structure of the server (500) discussed above. However, the home appliance (100) differs from the server (500) in that it provides a specific function.

[0219] The home appliance (100) includes a device control unit (150), a control example database (150), a situation example database (120), a control command database (130), and a device communication unit (590). Optionally, the home appliance includes a generative AI module (300). That is, the generative AI module (300) may be implemented within the home appliance (100) or may be implemented in an external device (e.g., a server (500) or an external provider).

[0220] The function module (180) provides the unique functions of the home appliance (100). For example, if the home appliance (100) is an air conditioner, the function module (180) includes both an indoor unit and an outdoor unit. If the home appliance (100) is an oven, the function module (180) includes components for heating. If the home appliance (100) is a refrigerator, the function module (180) includes components for providing freezing and refrigeration functions.

[0221] Depending on the implementation method of the home appliance (100), the function module (180) may be configured separately, and other components (110, 120, 130, 190, and optionally 300) including the device control unit (150) may be configured with one or more software or hardware. According to one embodiment of the present invention, other components (110, 120, 130, 190, and optionally 300) including the device control unit (150) may be implemented with a single chip.

[0222] The device control unit (150) determines which category or control command the input command corresponds to and generates a corresponding control command. During the determination process, the device control unit (150) may input certain information into the generative AI module (300).

[0223] The database (510, 520) can store a number of example sentences. In addition, among the databases, the situation example database (520) can store predefined routines.

[0224] The device communication unit (190) communicates with other devices (e.g., other servers or mobile terminals). If the generative AI module (300) is placed in an external device (external server), the device communication unit (190) also communicates with the external device.

[0225] The device control unit (150) checks whether the command entered into the home appliance (100) corresponds to direct control.

[0226] If the verification result shows that the entered command does not correspond to direct control, the device control unit (150) extracts an example sentence having a similarity level with the entered command that is higher than a certain standard from the database (520) and inputs it into the generative AI module (300).

[0227] If the input command corresponds to direct control, the device control unit (150) can generate a control command corresponding to the input command without inputting the input command to the generative AI module (300).

[0228] The device control unit (150) generates a control command using the output from the generative AI module (300) when the input command corresponds to post-interpretational control. Refer to the category description in Fig. 4 for this.

[0229] And the home appliance (100) controls the function module (180) to provide a function according to the generated control command.

[0230] By applying embodiments of the present invention, speech recognition errors can be improved, enabling flexible recognition of user commands. Furthermore, through situation-specific customized control, multiple functions of a home appliance can be controlled and operated with a single speech command. Furthermore, the server (500) can recognize and control home appliances when a user inputs not only defined keywords or control phrases but also undefined keywords or non-control phrases. Furthermore, routines pre-stored on the server (e.g., ThinQ routines) can be executed as voice / text command utterances.

[0231] Additionally, a separate generative AI module (300) (e.g., Chat GPT API) can be utilized to improve the voice recognition function of home appliances, enabling recognition of utterances that would not be recognized based on existing prescribed commands. Furthermore, if the utterance invokes a ThinQ routine, which is a customer-customized mode, the routine stored on the server (e.g., a routine stored on the ThinQ Cloud server) can be utilized to control the home appliance.

[0232] Although all components constituting the embodiments of the present invention have been described as being combined or operating in combination, the present invention is not necessarily limited to such embodiments, and within the scope of the present invention, all components may be selectively combined and operated one or more times. In addition, although all of the components may be implemented as individual independent hardware, some or all of the components may be selectively combined and implemented as a computer program having program modules that perform some or all of the functions combined in one or more hardware pieces. The codes and code segments constituting the computer program may be easily inferred by those skilled in the art of the present invention. Such a computer program may be stored in a computer-readable storage medium and read and executed by a computer, thereby implementing the embodiments of the present invention. Storage media for computer programs include magnetic recording media, optical recording media, and storage media including semiconductor recording devices. In addition, a computer program implementing an embodiment of the present invention includes a program module that is transmitted in real time through an external device.

[0233] While the above description focuses on specific embodiments of the present invention, various modifications and variations can be made within the scope of those skilled in the art. Therefore, it should be understood that such modifications and variations are within the scope of the present invention, as long as they do not depart from its scope.

[0234]

Claims

1. The first step is to check whether the command entered by the server or home appliance corresponds to direct control; In the first step, if the input command does not correspond to direct control, the second step is to extract example sentences having a similarity level higher than a certain standard with the input command from the database and input them into the generative AI module; A third step in which, if the input command corresponds to post-interpretational control in the result produced by the generative AI module, the server or the home appliance generates a control command using the produced result; and A method for controlling a home appliance based on a command, comprising a fourth step in which the home appliance provides a function according to the generated control command.

2. In paragraph 1, If the command entered in the above first step corresponds to direct control, A method for controlling a home appliance based on a command, wherein the home appliance further includes a step of providing a function according to a control command corresponding to the input command.

3. In paragraph 1, A method for controlling a home appliance based on a command, characterized in that the above example sentence has a similarity with the input command that is higher than a standard by BM25 or cosine similarity.

4. In paragraph 1, In the above third step, A method for controlling a home appliance based on a command, further comprising a step of generating a control command corresponding to a routine included in the generated result by the server or the home appliance when the input command corresponds to a post-interpretational routine in the result produced by the generative AI module.

5. In paragraph 1, In the above third step, A method for controlling a home appliance based on a command, further comprising a step of generating a control command for outputting the output result as voice or text, when the input command corresponds to post-interpretational chat in the result produced by the generative AI module.

6. In paragraph 1, The third step above is A method for controlling a home appliance based on a command, wherein the server or the home appliance further includes a step of generating the control command using an intent control command and a slot control command included in the calculated result.

7. In paragraph 6, The above second step is A method for controlling a home appliance based on a command, comprising a step of inputting a guide including one or more intent control commands and one or more slot control commands necessary for controlling the home appliance, the example sentence, and the input command into the generative AI module.

8. In paragraph 7, The above example sentence is a method for controlling a home appliance based on a command, which includes a sentence for controlling the home appliance and a set of corresponding intent control commands and slot control commands.

9. In paragraph 7, A method for controlling a home appliance based on a command, wherein the set of the intent control command and the slot control command instructs the home appliance to perform a function.

10. A server control unit that judges the input command and generates a corresponding control command; A database containing many example sentences; and Includes a server communication unit that communicates with home appliances, After the above server communication unit receives a command entered into the home appliance from the home appliance, The above server control unit checks whether the command entered into the home appliance corresponds to direct control, and if the result of the check indicates that the entered command does not correspond to direct control, it extracts an example sentence having a similarity level higher than a certain standard with the entered command from the database and inputs it into the generative AI module. The server control unit generates a control command using the calculated result when the input command corresponds to post-interpretational control in the result calculated by the generative AI module. A server that controls a home appliance based on commands, wherein the server communication unit transmits the control commands to the home appliance so that the home appliance can provide functions according to the generated control commands.

11. In paragraph 10, If the above entered command corresponds to direct control, A server that controls home appliances based on commands, characterized in that the above server control unit does not input the above-mentioned commands into the generative AI module.

12. In paragraph 10, The above example statement is a server that controls home appliances based on commands, characterized in that the similarity between the above input command and BM25 or cosine similarity is greater than or equal to a standard.

13. In paragraph 10, A server that controls home appliances based on commands, wherein if the input command corresponds to a post-interpretational routine in the result produced by the generative AI module, the server control unit generates a control command corresponding to the routine included in the produced result.

14. In paragraph 10, A server that controls a home appliance based on a command, wherein if the input command corresponds to post-interpretational chat in the result produced by the generative AI module, the server control unit generates a control command that outputs the produced result as voice or text.

15. In paragraph 10, The above server control unit is a server that controls home appliances based on commands, which generates the control commands using the Intent control command and Slot control command included in the above calculated result.

16. In paragraph 15, The server control unit above is a server that controls a home appliance based on commands, which inputs a guide including one or more intent control commands and one or more slot control commands required for controlling the home appliance, the example statement, and the input commands into the generative AI module.

17. In paragraph 16, The above example statement is a server that controls a home appliance based on commands, comprising a set of statements that control the home appliance and corresponding intent control commands and slot control commands.

18. In Paragraph 16, A server that controls a home appliance based on commands, wherein the set of intent control commands and slot control commands above directs the home appliance to perform a function.

Images