Household appliance and method of controlling the same

Abstract

A home appliance is disclosed. It includes a sensor, a microphone, a speaker, and a processor, and if one of a first event in which a user behavior is sensed through the sensor and a second event in which a trigger command language for starting a voice recognition mode is input through the microphone occurs, the processor operates in the voice recognition mode, and controls the speaker to output an audio signal corresponding to the occurred event, the audio signal being differently set for each of the first and second events.

Description

Technical Field

[0001] This invention relates to household appliances and their control methods, and more particularly to household appliances and their control methods that operate in a voice recognition mode if an event of user behavior is sensed. Background Technology

[0002] Home appliances can perform a variety of functions based on user control commands. Recent home appliances not only use input devices such as keyboards and remote controls to input control commands, but also employ voice recognition technology to input control commands via the user's voice.

[0003] In particular, more and more home appliances are sensing the trigger command language (e.g., Bixby) to activate the voice recognition function, thus operating in voice recognition mode. Summary of the Invention

[0004] The purpose of this invention is to provide a home appliance and its control method that operate in voice recognition mode if a user behavior event is detected.

[0005] A household appliance according to an embodiment of the present invention may include a sensor, a microphone, a speaker, and a processor. If one of a first event, in which the sensor senses a user's behavior, and a second event, in which the microphone inputs a trigger command language for activating a voice recognition mode, occurs, the processor operates in the voice recognition mode and controls the speaker to output an audio signal corresponding to the event that has occurred. The audio signal may be an audio signal that is set differently for each of the first and second events.

[0006] On one hand, a method for controlling a household appliance according to an embodiment of the present invention may include the following steps: if one of a first event of sensing user behavior and a second event of inputting a trigger command language for activating a voice recognition mode occurs, outputting an audio signal corresponding to the event that has occurred; and operating in the voice recognition mode, wherein the audio signal may be an audio signal that is set differently for each of the first event and the second event.

[0007] On one hand, a storage medium stores a program for executing a control method for a household appliance according to an embodiment of the present invention, the control method including the following steps: if one of a first event of sensing user behavior and a second event of inputting a trigger command language for activating a voice recognition mode occurs, outputting an audio signal corresponding to the event that has occurred; and operating in the voice recognition mode, the audio signal being an audio signal that is set differently for each of the first event and the second event. Attached Figure Description

[0008] Figure 1 This is a drawing illustrating a speech recognition system according to an embodiment of the present invention.

[0009] Figure 2 This is a block diagram illustrating a simple structure of a household appliance according to an embodiment of the present invention.

[0010] Figure 3 This is a block diagram illustrating the specific structure of a household appliance according to an embodiment of the present invention.

[0011] Figure 4 This is a block diagram illustrating a speech recognition mode according to an embodiment of the present invention.

[0012] Figure 5 This is a drawing illustrating a method for activating a speech recognition mode according to a first embodiment of the present invention.

[0013] Figure 6 This is a drawing illustrating a method for activating a speech recognition mode according to a second embodiment of the present invention.

[0014] Figure 7 This is a drawing illustrating a method for switching to standby mode according to a first embodiment of the present invention.

[0015] Figure 8 This is a drawing illustrating a method for switching to standby mode according to a second embodiment of the present invention.

[0016] Figure 9 The attached diagram shows an example of permission information for multiple users.

[0017] Figure 10 This is an accompanying diagram illustrating the execution of actions based on the permission information of multiple users.

[0018] Figure 11 The accompanying drawings illustrate an algorithm for a speech recognition pattern according to an embodiment of the present invention.

[0019] Figure 12 This is a sequence diagram illustrating the execution of actions based on the permission information of multiple users according to the first embodiment of the present invention.

[0020] Figure 13 This is a sequence diagram illustrating the execution of actions based on the permission information of multiple users according to a second embodiment of the present invention.

[0021] Figure 14 This is a flowchart illustrating a method for controlling a household appliance according to an embodiment of the present invention. Detailed Implementation

[0022] The terminology used in this specification is briefly explained, and the invention is described in detail.

[0023] The terminology used in embodiments of this invention is selected, as far as possible, from commonly used terms while taking into account the functionality of the invention. However, it may be changed based on the intent of those skilled in the art, precedents, or the emergence of new technologies. Furthermore, in certain cases, terms may be arbitrarily chosen by the applicant, and their meanings will be detailed in the description of the invention in such cases. Therefore, the terminology used in this invention should be defined based on its contained meaning and the entire content of the invention, rather than simply the name of the term.

[0024] The present invention can have various modifications and embodiments; therefore, specific embodiments are shown in the accompanying drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and should be understood to include all modifications, equivalents, and substitutions within the disclosed technical spirit and scope. In the description of embodiments, detailed descriptions of the functions of related well-known technologies will be omitted when such descriptions are deemed to obscure the subject matter.

[0025] Terms such as "first" and "second" can be used to describe multiple constituent elements, but a constituent element should not be limited by the term. A term is used only for the purpose of distinguishing a constituent element from other constituent elements.

[0026] Unless the context clearly indicates otherwise, singular representations include plural representations. In this invention, terms such as "comprising" or "constituting" are intended to indicate the presence of features, numbers, steps, actions, constituent elements, components, or combinations thereof described in the specification, and are to be understood as not precluding the presence or additional possibilities of one or more other features, numbers, steps, actions, constituent elements, components, or combinations thereof.

[0027] To facilitate implementation by those skilled in the art, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Furthermore, for the sake of clarity, parts unrelated to the description have been omitted from the drawings.

[0028] The invention will now be described in more detail with reference to the accompanying drawings.

[0029] Figure 1 This is a drawing illustrating a speech recognition system according to an embodiment of the present invention.

[0030] refer to Figure 1 The voice recognition system 1000 includes a home appliance 100 and a server 200.

[0031] The household appliance 100 can be a refrigerator, washing machine, oven, microwave oven, etc., and can be any other appliance besides those mentioned above, as long as it supports voice recognition mode.

[0032] Home appliance 100 can sense wake-up behavior①. Here, wake-up behavior means an event in which the user may issue a control command, ask a question, or respond. For example, the user issuing a trigger command or the opening and closing of the door of home appliance 100 can be considered wake-up behavior.

[0033] Furthermore, the home appliance 100 can be operated in voice recognition mode②. Also, the home appliance 100 can provide feedback indicating that voice recognition mode is enabled③. Specifically, the home appliance 100 can output an audio signal indicating that voice recognition mode is enabled.

[0034] At this point, the audio signal can be set to different signals based on the events that have occurred. Specifically, if an event triggering a command language is detected, a simple notification sound can be output indicating that voice recognition mode has been enabled. If a user action is detected, a specific voice message announcing that voice recognition mode has been enabled can be output.

[0035] Furthermore, the home appliance 100 can accept user voice input ④. Additionally, the home appliance 100 can transmit the input user voice to the server 200 ⑤.

[0036] Server 200 means a computer (server computer) or program (server program) that provides information or services to clients over a network.

[0037] Furthermore, server 200 can perform speech recognition on the input speech⑥. Specifically, server 200 can use a speech recognition module to perform speech recognition on the input speech. Furthermore, server 200 can send the speech recognition results to home appliance 100⑦.

[0038] Furthermore, the home appliance 100 can execute actions corresponding to the voice recognition results. On one hand, before executing an action, the home appliance 100 checks with the user to determine if they have permission for the action; once permission is granted, the action can be executed. (Reference) Figure 3 The specific actions to be taken in response will be described later.

[0039] On the one hand, regarding the target Figure 1 The illustrations and descriptions show that the voice recognition system includes a home appliance and a server, but in practice it can be implemented to include multiple home appliances or multiple servers.

[0040] In addition, for those targeting Figure 1 The illustrations and descriptions provided are intended to depict a voice recognition system including a server. However, in practice, if a home appliance is designed to perform all the functions of a server, the voice recognition system may be implemented without a server.

[0041] Figure 2 This is a block diagram illustrating a simple structure of a household appliance according to an embodiment of the present invention.

[0042] refer to Figure 2 The household appliance 100 includes a sensor 110, a microphone 120, a processor 140, and a speaker 130.

[0043] Sensor 110 senses user behavior. Specifically, sensor 110 can sense user behavior related to home appliance 100.

[0044] For example, if sensor 110 is a sensor that senses the opening and closing of a door (not shown) installed on the main body of household appliance 100, sensor 110 can sense whether the door is opened or closed by the user and output a corresponding value. In this case, sensor 110 can be implemented as a switch, outputting a value corresponding to a closed door if the switch is on, and outputting a value corresponding to an open door if the switch is off.

[0045] Furthermore, if sensor 110 is a motion sensor, it can sense the approach of a user to the home appliance 100. In this case, sensor 110 can be implemented as an IR sensor, proximity sensor, thermal sensor, or camera, etc. Additionally, if sensor 110 is a touch sensor, it can sense touch from a user to the home appliance 100.

[0046] On the other hand, the type of sensor 110 is not limited to this, and can vary depending on the type or model of the household appliance 100. Furthermore, multiple sensors 110 can be provided.

[0047] The microphone 120 can receive voice input. Specifically, the microphone 120 can receive a user's spoken voice and convert it into voice data that can be processed on the home appliance 100. Furthermore, the microphone 120 can send the converted voice data to the processor 140.

[0048] On the one hand, although the illustrated example shows that the microphone 120 is installed inside the household appliance 100, it can be implemented as an external device of the household appliance. That is, it can be implemented as a device that receives and uses voice signals from an external microphone.

[0049] The speaker 130 can be installed inside the household appliance 100 to output sound to the outside of the household appliance 100. Specifically, the speaker 130 can convert audio signals into sound and output them according to the control of the processor 140.

[0050] The processor 140 controls the household appliance 100. Specifically, the processor 140 can control various components of the household appliance 100 according to user control commands. For example, if the household appliance 100 is a washing machine, and the processor 140 receives a spin-dry command, it can control the operation of the motor to provide rotational force to the drum containing the laundry.

[0051] The processor 140 can operate in a speech recognition mode. Here, speech recognition mode means that the processor 140 processes user questions and responses input via the microphone 120 into a natural language action pattern. In this case, the processor 140 can be implemented either by directly processing the user's questions and responses into natural language using a speech recognition module, or by sending the speech input via the microphone 120 to a server and receiving the speech recognition result from the server. On the one hand, referring to… Figure 4 The following section provides a detailed explanation of the speech recognition module.

[0052] Specifically, if a preset event occurs, the processor 140 can operate in voice recognition mode. Here, a preset event means an event that is set as potentially allowing a user to issue a control command, ask a question, or respond.

[0053] For example, a preset event could be an event where the user speaks a trigger command language to activate the voice recognition mode, and the microphone 120 inputs voice containing the trigger command language. In this case, the processor 140 uses a trigger command language sensing model to determine whether the voice data obtained through the microphone 120 includes the trigger command language. If it is determined that the trigger command language is included, operation can proceed in voice recognition mode. On one hand, the trigger command language can be referred to as a wake-up command language or an activation command language, etc.

[0054] Additionally, the preset event can be an event that triggers user behavior detected by sensor 110. In this case, if user behavior is detected by sensor 110, processor 140 can operate in voice recognition mode. Here, the user behavior can be set differently depending on the type of sensor 110.

[0055] If sensor 110 is a sensor for sensing the opening and closing of a door installed on the main body of household appliance 100, then processor 140 can operate in voice recognition mode when sensor 110 senses an open or closed door. On one hand, processor 140 can be implemented by operating in voice recognition mode only when sensor 110 senses an open door, or only when sensor 110 senses a closed door. (See reference...) Figure 5 and Figure 6 The specific actions to be taken in response will be described later.

[0056] Conversely, if sensor 110 is a motion sensor, then processor 140 can operate in voice recognition mode when sensor 110 senses the approach of a user to home appliance 100.

[0057] Similarly, the processor 140 can operate in voice recognition mode if it is input with a trigger command language to activate the voice recognition mode via the microphone 120, or if user behavior is sensed by the sensor 110.

[0058] Furthermore, when the processor 140 operates in voice recognition mode, it can control the speaker 130 to output an audio signal notifying that the operation is in voice recognition mode.

[0059] Specifically, when a preset event occurs and the processor 140 operates in voice recognition mode, it can control the speaker 130 to output audio signals corresponding to each preset event.

[0060] More specifically, if the processor 140 is triggered by a command language input via the microphone 120 and operates in voice recognition mode, it can control the speaker 130 to output a first audio signal. Furthermore, if the processor 140 senses user behavior via the sensor 110 and operates in voice recognition mode, it can control the speaker 130 to output a second audio signal different from the first audio signal.

[0061] Here, the first audio signal can be a simple notification sound. This is so that when the user triggers a command by speaking, since the user is interested in the voice recognition mode of the home appliance 100, the simple notification guides the user to immediately speak a control command or ask a question without waiting.

[0062] On the one hand, the second audio signal can be a voice message notifying the activation of voice recognition mode (e.g., "What help do you need?"). This is because even if user behavior is sensed, it is uncertain whether the user intends to use the voice recognition mode of the home appliance 100. Therefore, the specific notification guides the user to speak control commands or ask questions while operating in voice recognition mode.

[0063] Furthermore, the processor 140 can perform speech recognition on speech input via the microphone 120. Specifically, the processor 140 can perform speech recognition on speech input via the microphone 120 after starting operation in speech recognition mode.

[0064] At this time, the processor 140 can perform voice recognition using a voice recognition module installed in the home appliance 100. Alternatively, the processor 140 can also perform voice recognition by sending voice data corresponding to the voice input to an external server (not shown) and receiving the voice recognition result for the sent voice data.

[0065] Furthermore, the processor 140 can execute actions corresponding to the voice recognition results. For example, if the household appliance 100 is a washing machine and the voice recognition result is "wash for 30 minutes", the processor 140 can control the household appliance 100 to perform a 30-minute washing operation.

[0066] The above only illustrates the simple composition of household appliance 100, but household appliance 100 may also include, for example... Figure 3 The following describes the configuration. For the specific configuration of household appliance 100, please refer to the following. Figure 3 Please provide an explanation.

[0067] Figure 3 This is a block diagram illustrating the specific structure of a household appliance according to an embodiment of the present invention.

[0068] refer to Figure 3 According to this embodiment, the household appliance 100 may be composed of a sensor 110, a microphone 120, a speaker 130, a processor 140, a user interface device 150, a memory 160, a communication device 170, and a display 180.

[0069] The configuration of sensor 110, microphone 120 and speaker 130 and Figure 2 The configurations are the same, therefore repeated descriptions are omitted. Furthermore, for processor 140, reference has been made... Figure 2 To provide an explanation, it will not be repeated here. Figure 2 The content already explained in the text is only explained below in relation to the content already explained in the text. Figure 3 The additional content related to the composition.

[0070] Processor 140 may include a plurality of processors 141, 143. Specifically, processor 140 may include a first processor 141 for sensing trigger command language and a second processor 143 for operating in a voice recognition mode.

[0071] The first processor 141 can remain powered on in a standby state. Here, standby state means a state where a preset event is sensed in order to initiate operation in voice recognition mode. Furthermore, if the first processor 141 receives user voice input, including a trigger command language, via the microphone 120, it can output a notification signal to the second processor 143.

[0072] A first processor 141 like this can be included in an audio chipset (not shown) that converts analog audio signals to digital audio signals or converts digital audio signals to analog audio signals.

[0073] On the one hand, the second processor 143 can remain in a turned-off state in standby mode. Specifically, if the standby state is maintained for a preset time or longer, the second processor 143 can switch from an on state to a turned-off state. Furthermore, if the second processor 143 receives an output notification signal from the first processor 141, it can be turned on and operate in voice recognition mode.

[0074] The user interface device 150 is configured to receive user interactions such as input user operations. Specifically, the user interface device 150 can receive various control commands from the user for the home appliance 100. For example, in the case of a washing machine, the user interface device 150 can receive a spin-dry command from the user.

[0075] The user interface device 150 may include buttons formed on any area of ​​the main body of the home appliance 100, such as the front, side, or back, or a light receiving device that receives light signals from a remote control device corresponding to user input (e.g., touch, press, touch gesture, voice, or action). On the other hand, if the display 180 is a touch screen, the display 180 may also operate as the user interface device 150.

[0076] The user interface device 150 can receive input from the user regarding the activation or deactivation of various preset events.

[0077] Specifically, the user interface device 150 can receive input regarding the activation or deactivation of various events, which refer to events where a trigger command language for activating the voice recognition mode is input via the microphone 120 or events where user behavior is sensed by the sensor 110.

[0078] Furthermore, if a preset event occurs, the processor 140 can operate in voice recognition mode based on the input result. Specifically, if the preset event is activated by an input preset event, the processor 140 can operate in voice recognition mode. However, if the preset event is deactivated by an input preset event, the processor 140 may not operate in voice recognition mode even if the preset event occurs.

[0079] For example, if the user interface device 150 receives a deactivation input via the microphone 120 to trigger a command language for activating the voice recognition mode, the processor 140 may operate without using the voice recognition mode even if a trigger command language is input via the microphone 120.

[0080] Furthermore, when the user interface device 150 receives a deactivation input indicating that a user action has been sensed by the sensor 110, the processor 140 may operate without using voice recognition mode, even if the user action has been sensed by the sensor 110.

[0081] Furthermore, if the processor 140 is disabled by a preset event, it can perform additional actions based on what the disabled preset event is.

[0082] Specifically, when a deactivation input is received via microphone 120 to trigger a command language for activating voice recognition mode, processor 140 does not need to perform sensing of the trigger command language. Therefore, processor 140 can turn off microphone 120 in the standby state of home appliance 100, thereby saving power consumption.

[0083] At this time, if the event of user behavior is activated by the sensor 110, the processor 140 can turn on the microphone again and execute the voice recognition module if the user behavior is detected by the sensor 110.

[0084] Conversely, when a deactivation input is received via sensor 110 to detect user behavior, processor 140 does not need to sense user behavior. Therefore, processor 140 can save power consumption by turning off sensor 110 to sense user behavior in standby mode.

[0085] At this time, if an event triggering a command language is activated by inputting through the microphone 120, the processor 140 can keep the microphone 120 open in standby mode in order to trigger the sensing of the command language.

[0086] The memory 160 stores programs for processing or control by the processor 140, as well as various data for the overall operation of the home appliance 100. Specifically, the memory 160 may store multiple applications driven by the home appliance 100, data for the operation of the home appliance 100, and command languages.

[0087] Furthermore, the memory 160 can be accessed by the processor 140 and can perform operations such as reading, recording, modifying, deleting, and updating data according to the processor 140. This memory 160 can be implemented not only as a storage medium in the home appliance 100, but also as an external storage medium, including a removable disk with USB memory, a network server, etc.

[0088] Furthermore, the memory 160 can store the speech recognition module. The processor 140 can also use the speech recognition module to generate natural language responses to user speech. The speech recognition module can be executed using an existing general-purpose processor (e.g., CPU) or a dedicated AI processor (e.g., GPU, etc.). For further details, please refer to [reference needed]. Figure 4 This will be discussed later.

[0089] Furthermore, the memory 160 can store a speaker recognition model. Here, the speaker recognition model means a model that identifies the user who is the current speaker from among the pre-registered users by using voiceprint information included in the speech input through the microphone 120 and voiceprint information of pre-registered users.

[0090] Furthermore, the processor 140 can use a speaker recognition model stored in the memory 160 to perform a speaker recognition function that identifies the user corresponding to the voice input through the microphone 120.

[0091] The memory 160 can store the permission information of each user. Specifically, the memory 160 can store the permission information of multiple users for multiple actions of each household appliance 100.

[0092] Furthermore, if the processor 140 receives user voice input via the microphone 120, it can identify the user by recognizing the user's voice, and if the action of the home appliance 100 corresponding to the user's voice matches the permission information of the identified user, the action of the home appliance 100 corresponding to the user's voice can be executed.

[0093] More specifically, the processor 140 can identify the user using the user's voice input through the microphone 120 in voice recognition mode and the speaker recognition model stored in the memory 160. Furthermore, the processor 140 uses the user's permission information stored in the memory 160 to determine whether the action of the home appliance 100 corresponding to the user's voice (e.g., a washing action) conforms to the identified user's permission information. If the permission is met, the action of the home appliance 100 corresponding to the user's voice can be executed.

[0094] On the one hand, if the user whose voice is input through the microphone 120 is not recognized, or if the recognized user does not have permission to perform an action on the home appliance 100 corresponding to the user's voice, the processor 140 may not perform an action on the home appliance 100 corresponding to the user's voice.

[0095] refer to Figure 9 and Figure 10 The specific actions to be taken in response will be described later.

[0096] The communication device 170 communicates with various types of external devices using various communication methods. Specifically, the communication device 170 can communicate with other household appliances or with external networks (e.g., the Internet, servers, etc.). For this purpose, the communication device 170 may include a WiFi chip and a Bluetooth chip.

[0097] For example, a Bluetooth chip can be used to communicate with other household appliances in the vicinity, while a WiFi chip can be used to communicate with an external server.

[0098] If the home appliance 100 uses an external server to perform the voice recognition function, the communication device 170 can transmit the voice data of the voice input through the microphone 120 to the external server and receive the voice recognition result from the external server.

[0099] On the one hand, when the home appliance 100 uses an external server to perform the speaker recognition function, the communication device 170 can send voice data for the voice input through the microphone 120 to the external server and receive the speaker recognition result from the external server.

[0100] Furthermore, the processor 140 uses the received speaker recognition result (i.e., user information corresponding to the voice data) and pre-stored permission information of multiple users for multiple actions of each household appliance 100 to determine whether there is permission. If the permission is determined, the function corresponding to the voice recognition result can be executed.

[0101] The display 180 can display various information provided on the home appliance 100. Specifically, the display 180 can display the operating status of the home appliance 100, or it can display a user interface window for users to select actions and options.

[0102] For example, the display 180 can show the results of the voice recognition function, or it can display an interface window that allows the user to change the settings for the voice recognition function.

[0103] It has always operated in speech recognition mode only when the triggering command language is in the user's voice input through the microphone, thus presenting a technical problem that the triggering command language must be spoken before the user speaks a voice command.

[0104] Conversely, the household appliance according to this embodiment, as described above, operates in voice recognition mode even if the user does not speak to trigger the command language, as long as the user behavior predicted to be a voice command is sensed, thereby solving the technical effect of solving the inconvenience of the user having to speak to activate the command language.

[0105] Furthermore, according to this embodiment, when the home appliance senses user behavior, it outputs a voice message to activate the voice recognition mode. Even if it is uncertain whether the user intends to use the voice recognition mode of the home appliance, it specifically notifies the user to operate in the voice recognition mode, thereby guiding the user to speak control commands or ask questions.

[0106] Furthermore, according to this embodiment, if the sensing of an event triggering a command language is disabled, the microphone is turned off in standby mode; if the sensing of a user behavior event is disabled, the sensor is turned off in standby mode, thereby saving power consumed by components that are not needed in standby mode.

[0107] Figure 4 This is a block diagram illustrating a speech recognition mode according to an embodiment of the present invention.

[0108] According to an embodiment of the present invention, the action of the voice recognition mode can be performed by using, for example Figure 4 The voice recognition module 300 shown is executed. According to one embodiment of the present invention, the voice recognition module 300 may be stored in the memory 160 of the home appliance 100. However, this is merely one embodiment, and at least one component of the voice recognition module 300 may be included in at least one external server.

[0109] 300 voice recognition modules Figure 4As shown, it may include an automatic speech recognition (ASR) module 310, a natural language understanding (NLU) module 320, a dialogue manager (DM) module 330, a natural language generator (NLG) module 340, and a text-to-speech (TTS) module 350.

[0110] The automatic speech recognition (ASR) module 310 can convert user speech received from the home appliance 100 into text data. Furthermore, the ASR module 310 may include a speech recognition module, which may include an acoustic model and a language model. Specifically, the acoustic model can obtain acoustic feature information of the user's speech.

[0111] The Natural Language Understanding module 320 performs syntactic analysis or semantic analysis to understand the user's intent.

[0112] The Natural Language Understanding module 320 can divide user input into grammatical units (e.g., words, sentences, morphemes, etc.), understand what grammatical elements the divided units have, and then perform grammatical analysis.

[0113] The natural language understanding module 320 can perform semantic analysis by using semantic pairing, rule pairing, formula pairing, etc. In this way, the natural language understanding module 320 can obtain the parameters (or slots) necessary for the user's input to represent a domain, intent, or purpose.

[0114] The natural language understanding module 320 can use pairing rules that divide users into domains, intents, or parameters (or slots) necessary for understanding their intentions to determine the user's intent and parameters.

[0115] For example, a domain name (e.g., "alert") can include multiple intents (e.g., "turn on alerts", "turn off alerts"), and an intent can include multiple parameters (e.g., "time", "repetition count", "alert sound"), etc. Furthermore, multiple rules can include more than one required element parameter. On the one hand, pairing rules can be stored in a Natural Language Understanding Database (NLU DB)

[323] .

[0116] The Natural Language Understanding module 320 can use linguistic features such as morphemes and sentences (e.g., grammatical elements) to understand the meaning of words extracted from user input, match the understood meanings of words to domain names and intents, and thus determine the user's intent.

[0117] For example, the natural language understanding module 320 can determine the user's intent by calculating how many words extracted from the user input are included in each domain name and intent. According to one embodiment, the natural language understanding module 320 can use words that form the basis for understanding the intent to determine the parameters of the user input.

[0118] Furthermore, the natural language understanding module 320 can use a natural language recognition database 323, which stores language features for understanding the user's input intent, to determine the user's intent.

[0119] The natural language understanding module 320 can use a private knowledge database 325 to understand user queries. In this case, the private knowledge database 325 can learn relationships based on at least one of the following: user interactions input on the home appliance 100, sensor information sensed by the home appliance 100, and user information received from external devices. The private knowledge database 325 can then store the relationships between these knowledge information in an ontological form.

[0120] The dialogue management module 330 can determine whether the user's intent, as understood by the natural language understanding module 320, is clear. For example, the dialogue management module 330 can determine whether the user's intent is clear based on whether the parameter information is sufficient. The dialogue management module 330 can also determine whether the parameters understood by the natural language understanding module 320 are sufficient to perform the task.

[0121] According to one embodiment, when the user's intent is unclear, the dialogue management module 330 can request feedback on information needed by the user. For example, the dialogue management module 330 can request feedback on information regarding parameters used to understand the user's intent. Furthermore, the dialogue management module 330 can generate and output a message used to determine a user question that includes text modified by the natural language understanding module 320.

[0122] The Natural Language Generation (NLG) module 340 can convert specified information into text form. The information converted into text form can be the form of natural language speech. Here, the specified information can be information for additional input, information guiding the completion of an action corresponding to user input, or information guiding additional user input (e.g., feedback information for user input).

[0123] The text-to-speech (TTS) module 350 can convert text-based information into speech-based information. The TTS module 350 can receive text-based information from the natural language generation module 340, convert it into speech-based information, and output it through a speaker. Furthermore, the text-based information can be displayed on the display 180 of the home appliance 100.

[0124] Figure 5 and Figure 6 The accompanying diagram illustrates various publicly available methods for speech recognition patterns.

[0125] If the home appliance 100 senses user behavior through the sensor 110, it can be operated in voice recognition mode. For example, if the home appliance 100 senses the opening or closing of a door through the sensor 110, it can be operated in voice recognition mode.

[0126] For ease of explanation, the following assumes that the household appliance 100 is a washing machine. The operation of the household appliance 100 will be explained when the user's action is to open or close the door.

[0127] Figure 5 This is a drawing illustrating a method for activating a voice recognition mode according to a first embodiment of the present invention. In this embodiment, it is assumed that the user behavior is set to the case of opening a door.

[0128] If a user approaches the washing machine and speaks a command (e.g., "Hello Bixby"), the home appliance 100 can be operated in voice recognition mode. Furthermore, the home appliance 100 can output an audio signal corresponding to the sensed command event. In this case, the output audio signal is a simple notification sound indicating that voice recognition mode has been activated. (See reference) Figure 5 The audio signal that triggers the command language sensing event can be a simple notification sound like "ding-dong"①.

[0129] On one hand, a user may open the door to put laundry into the washing machine drum. In this case, the appliance 100 can sense the door opening and operate it using voice recognition mode. Furthermore, the appliance 100 can output an audio signal corresponding to the door opening event. At this time, the output audio signal is a voice message specifically notifying that voice recognition mode has been activated. (Reference) Figure 5 The audio signal corresponding to the door opening event can be a voice message like "What help do you need?"②. On the other hand, the voice message is not limited to the above example; it can be a voice message like "Turn on voice recognition mode".

[0130] Furthermore, the home appliance 100 can perform voice recognition on the user's voice input when operating in voice recognition mode, and can perform actions corresponding to the voice recognition results.

[0131] On the one hand, in addition to being able to operate in voice recognition mode when a door opening event is detected, the home appliance 100 can also be operated in voice recognition mode when a door closing event is detected. The following explains the relevant content.

[0132] Figure 6 This is a drawing illustrating a method for activating a voice recognition mode according to a second embodiment of the present invention. In this embodiment, it is assumed that the user behavior is set to close the door.

[0133] refer to Figure 6 Users can open the door to put laundry into the washing machine drum. However, opening the door is not set as a user action, so the appliance 100 does not operate in voice recognition mode and can remain in standby mode.

[0134] Furthermore, if the user closes the washing machine door again, the appliance 100 can sense the door closing and operate it in voice recognition mode. The appliance 100 can also output an audio signal corresponding to the door closing event. At this time, the output audio signal is a voice message specifically notifying that voice recognition mode has been activated.

[0135] In this way, the home appliance 100 can set different methods to enable the voice recognition mode based on whether the action set as the user behavior is opening or closing the door.

[0136] On the one hand, user behavior can be set during manufacturing. Furthermore, user behavior can be set to open or close a door according to user settings. Specifically, the home appliance 100 can receive input from the user via the user interface device 150 of at least one action as a user behavior: opening or closing a door. In this case, the home appliance 100 sets the input action as a user behavior, and if the input action is sensed, it can be operated in voice recognition mode.

[0137] On the one hand, after the home appliance 100 is operated in voice recognition mode, it can be switched back to standby mode if certain conditions are met. The following is a detailed explanation of this.

[0138] Figure 7 This is a drawing illustrating a method for switching to standby mode according to a first embodiment of the present invention. In this embodiment, it is assumed that the user behavior is set to the case where the door is open.

[0139] refer to Figure 7 It can determine whether the user has closed the washing machine door. When the appliance 100 senses that the door is open, it operates in voice recognition mode, and then switches back to standby mode if it senses that the door is closed.

[0140] Specifically, if the home appliance 100 senses that the door is closed when it performs voice recognition by opening the door, it assumes that the user is no longer issuing voice commands and can switch from voice recognition mode to standby mode.

[0141] On the one hand, even if the home appliance 100 senses that the door is closed, it can still maintain voice recognition mode. (For reference...) Figure 8 The following is an explanation.

[0142] Figure 8 This is a drawing illustrating a method for switching to standby mode according to a second embodiment of the present invention.

[0143] The home appliance 100 can maintain the voice recognition mode based on whether the voice input through the microphone 120 when operating in voice recognition mode includes the user's control commands or questions.

[0144] If the voice input through microphone 120 during operation in voice recognition mode includes the user's control commands or questions, the home appliance 100 can maintain voice recognition mode and perform interaction with the user.

[0145] For example, refer to Figure 8When the user opens the washing machine door, they can issue a control command such as "Please wash" and then close the door. In this case, when operating in voice recognition mode, the user's control command is included in the voice input through the microphone 120, and the appliance 100 can maintain the voice input. Furthermore, the appliance 100 can output a sound such as "How many minutes do you want to wash?" and perform interaction with the user.

[0146] Conversely, if the voice input through the microphone 120 during operation in voice recognition mode does not include the user's control commands or questions, the home appliance 100 assumes that the user intends not to use the voice recognition function, and thus switches from voice recognition mode to standby mode.

[0147] On the one hand, for illustration and explanation Figure 7 and Figure 8 The user action is illustrated and described as opening the door, but the same applies when the user action is closing the door.

[0148] Specifically, when the user action is to close the door, the home appliance 100 can switch from voice recognition mode to standby mode based on whether a preset time has been reached after operation in voice recognition mode.

[0149] On the one hand, the home appliance 100 can maintain the voice recognition mode based on whether the voice input through the microphone 120 within a preset time includes the user's control commands or questions.

[0150] If, within a preset time, the voice input via microphone 120 includes a user's control command or question, the home appliance 100 can maintain voice recognition mode and perform interaction with the user.

[0151] Conversely, if the voice input via microphone 120 within a preset time does not include the user's control commands or questions, the home appliance 100 assumes that the user intends not to use the voice recognition function, and thus switches from voice recognition mode to standby mode.

[0152] In this way, when the home appliance 100 senses a preset event, it operates in voice recognition mode, and when the preset conditions are met, it switches from voice recognition mode to standby mode, thereby preventing incorrect recognition and power consumption caused by continuously maintaining voice recognition mode.

[0153] On the one hand, for illustration and explanation Figures 5 to 8 Assuming the household appliance is a washing machine, but the types of household appliances are not limited to washing machines; any household appliance with a door and a sensor to detect the opening and closing of the door is acceptable.

[0154] On the one hand, even if the home appliance 100 is operated in voice recognition mode as described above and a user inputs a control command, the user can execute the action based on whether the user has the permission to perform the action corresponding to the control command. The specific actions for this are explained below.

[0155] Figure 9 The attached diagram illustrates an example of permission information for multiple users. If a user's voice is input via the microphone 120, the processor 140 can recognize the user's voice and identify the user. If the action of the home appliance 100 corresponding to the user's voice matches the permission information of the identified user, the processor 140 can execute the action of the home appliance 100 corresponding to the user's voice.

[0156] More specifically, the processor 140 can use the user's voice input through the microphone 120 in a voice recognition mode and the speaker recognition model stored in the memory 160 to identify the user.

[0157] Furthermore, the processor 140 uses the user's permission information stored in the memory 160 to determine whether the action of the home appliance 100 corresponding to the user's voice (e.g., washing action) conforms to the permission information of the identified user. If the permission is met, the action of the home appliance 100 corresponding to the user's voice can be executed.

[0158] For example, refer to Figure 9 In the case of a home appliance 100 being an oven, the appliance 100 can store permission information for multiple users, namely, a father, a mother, and a child. Based on the permission information shown in the diagram, it can be determined that the father and mother have permissions for preheating and baking actions, but the child does not have such permissions.

[0159] Therefore, the processor 140 can determine in the voice recognition mode whether the current speaker is the father, mother, or child, and whether the determined user has the permission to perform the action corresponding to the control command. If the user has the permission, the action is executed; otherwise, the action is not executed.

[0160] Figure 10 This is an accompanying diagram illustrating the execution of actions based on the permission information of multiple users.

[0161] According to one embodiment of the present invention, the execution of an action based on the user's permission information can only be performed if the control command determined by the voice recognition result is a preset action. Here, the preset action can be an action that requires safety when performing the action. For example, in the case of a household appliance 100 being an oven, the preset action could be a baking action or a preheating action.

[0162] Specifically, when the action corresponding to the user's voice input through the microphone 120 is a preset action, the processor 140 can identify which user among the pre-registered users the current speaker is, and determine whether the identified user has the permission for the preset action.

[0163] For example, refer to Figure 10 It can be determined that the child says "Please bake for 1 hour"①. In this case, the processor 140 can determine that the control command included in the spoken voice is a baking action, and that it is a safe action. Furthermore, the processor 140 can determine that the current speaker is a child by recognizing the input voice, and determine whether the child has permission to perform the baking action.

[0164] However, since the child does not have permission to perform the baking action, the processor 140 may choose not to perform the baking action. Furthermore, to prevent the baking action from being performed, the speaker 130 can be controlled to output a sound similar to "danger"②. Additionally, the processor 140 can display a message on the display 180 indicating that the baking action should not be performed.

[0165] On one hand, the processor 140 can receive input regarding permissions for various actions of the home appliance 100 for each user who has pre-registered with the user. For example, the processor 140 can receive input of a command granting permission or deletion permission to a "child" of a user registered through the user interface device 150 for the baking action of the home appliance 100.

[0166] On the one hand, the actions of the household appliance 100 are not limited to the baking and preheating actions mentioned above; various actions based on the characteristics of the household appliance 100 can be applied.

[0167] In this way, the processor uses the permission information of multiple users to execute actions corresponding to control commands, thereby achieving the effect of safe use of home appliances and security maintenance.

[0168] Figure 11 The accompanying drawings illustrate an algorithm for a speech recognition pattern according to an embodiment of the present invention.

[0169] refer to Figure 11 In standby mode, the processor 140 of the home appliance 100 can sense whether voice, including the trigger command language, is input through the microphone 120 S1110.

[0170] If a trigger command language S1110-Y is sensed, the processor 140 can output an audio signal S1130 corresponding to the event of sensing the trigger command. Here, the audio signal corresponding to the event of sensing the trigger command can be a simple notification sound.

[0171] Conversely, if no trigger command language S1110-N is sensed, the processor 140 can determine whether user behavior S1120 has been sensed by the sensor 110.

[0172] If user behavior S1120-Y is sensed, the processor 140 can output an audio signal S1135 corresponding to the event of the sensed user behavior. Here, the audio signal corresponding to the event of the sensed user behavior can be a voice message notifying that the voice recognition mode has been turned on.

[0173] Conversely, if no user behavior S1120-N is sensed, the processor 140 can continue to remain in standby mode and sense whether voice input including the trigger command language S1110 is received through the microphone 120.

[0174] On the one hand, after outputting a notification sound or voice message, the processor 140 can operate in voice recognition mode S1140. Specifically, the processor 140 can use the voice recognition module to perform voice recognition on voice input through the microphone 120.

[0175] Furthermore, the processor 140 can determine whether the action corresponding to the speech recognition result is a preset action S1150.

[0176] If the action corresponding to the speech recognition result is not a preset action (S1150-N), since this function does not require judging the user's permissions, the processor 140 can execute the action corresponding to the speech recognition result (S1180).

[0177] Conversely, when the action corresponding to the speech recognition result is a preset action (S1150-Y), the processor 140 can identify the user of the speech input through the microphone 120. Specifically, the processor 140 can use a speaker recognition model to recognize the input speech, thereby identifying the user.

[0178] Furthermore, it can be determined whether the identified user is a pre-registered user S1160.

[0179] If the identified user is a pre-registered user, S1160-Y, it is possible to determine whether the identified user has permission for an action corresponding to the speech recognition result, S1170.

[0180] If the identified user has permission for an action corresponding to the speech recognition result, the processor 140 can execute the action S1180 corresponding to the speech recognition result.

[0181] If the identified user is not a pre-registered user S1160-N, or if the identified user does not have permission for the action corresponding to the speech recognition result S1170-N, the processor 140 may not execute the action corresponding to the speech recognition result and switch to standby mode.

[0182] On the other hand, regarding the target Figure 11 The illustrations and explanations show that the user behavior is detected after the trigger command language is detected, but in practice, it can also be implemented by detecting whether the trigger command language is detected after the user behavior is detected.

[0183] In addition, for those targeting Figure 11 Although the illustrations and explanations show that the operation is performed in voice recognition mode after outputting a notification sound or voice message, it can also be implemented by operating in voice recognition mode and then outputting a notification sound or voice message.

[0184] In addition, for those targeting Figure 11 The illustrations and descriptions provided illustrate that a processor in a household appliance performs all the described actions, but in practice, this can also be achieved by a server performing some of the processor's actions. See the following references. Figure 12 and Figure 13 This section provides a detailed explanation of the specific details.

[0185] Figure 12 This is a sequence diagram illustrating the execution of actions based on the permission information of multiple users according to the first embodiment of the present invention.

[0186] First, the home appliance 100 can sense a preset event S1210. Specifically, the home appliance 100 can sense events triggered by inputting command language through the microphone 120 or events sensed by user behavior through the sensor 110.

[0187] Furthermore, if the home appliance 100 senses a preset event, it can operate in voice recognition mode S1220. Specifically, the home appliance 100 can output audio signals corresponding to each preset event and receive user voice input via microphone 120.

[0188] Furthermore, the home appliance 100 can send voice data S1230 corresponding to the input voice to the server 200.

[0189] Furthermore, server 200 can use the received voice data to identify the user S1240 corresponding to the speaker. Specifically, server 200 can use a speaker recognition model to identify the user corresponding to the speaker.

[0190] More specifically, server 200 can extract voiceprints from the received voice data. Furthermore, server 200 can compare the extracted voiceprint information with the voiceprint information of pre-registered users to determine whether the speaker is equivalent to a pre-registered user.

[0191] Furthermore, the server 200 can send information S1250 about the identified user to the home appliance 100.

[0192] Furthermore, the home appliance 100 can determine the action S1260 corresponding to the voice input through the microphone 120. Specifically, the home appliance 100 can use a voice recognition module to perform voice recognition on the voice input through the microphone 120, thereby determining the control command and the action of the home appliance 100 corresponding to the control command.

[0193] Furthermore, the home appliance 100 can determine, S1270, whether the identified user has the authority to perform actions corresponding to the determined control command. Specifically, the home appliance 100 can use the permission information of multiple users to determine whether the identified user has the authority to perform actions corresponding to the determined control command.

[0194] Furthermore, the home appliance 100 can provide feedback S1280 based on whether the user has the necessary permissions. Specifically, if the identified user has the necessary permissions, the home appliance 100 can execute an action corresponding to the control command.

[0195] Conversely, if the identified user lacks authorization or is not a registered user, the action corresponding to the control command may not be executed. In this case, the home appliance 100 can provide guidance to the user that the action cannot be performed. For example, the home appliance 100 can output a preset guidance message sound or display a preset guidance message on the display 180.

[0196] Therefore, if a household appliance does not have a speaker recognition model, a server capable of performing speaker recognition can be used to identify the user corresponding to the input voice.

[0197] On the one hand, depending on their type or application, home appliances may not only lack a speaker recognition model, but also a speech recognition module. In this case, a server can be used to perform speaker recognition and speech recognition, as explained below.

[0198] Figure 13 This is a sequence diagram illustrating the execution of actions based on the permission information of multiple users according to a second embodiment of the present invention.

[0199] First, the home appliance 100 can sense preset events S1310. Furthermore, if the home appliance 100 senses a preset event, it can operate in voice recognition mode S1320. Specifically, the home appliance 100 can output audio signals corresponding to each preset event and receive user voice input via microphone 120.

[0200] Furthermore, the home appliance 100 can send voice data S1330 corresponding to the input voice to the server 200.

[0201] Furthermore, server 200 can use the received voice data to identify the user S1340 corresponding to the speaker. Specifically, server 200 can use a speaker recognition model to identify the user corresponding to the speaker.

[0202] Furthermore, the server 200 can determine the action S1350 of the home appliance 100 corresponding to the received voice data. Specifically, the server 200 can use a voice recognition module to perform voice recognition on the received voice data, determine the control command, and determine the action of the home appliance 100 corresponding to the control command.

[0203] Furthermore, the server 200 can send information about the identified user and the determined action information S1360 to the home appliance 100.

[0204] Furthermore, for the determined action, the home appliance 100 can determine whether the identified user has the necessary permissions (S1370). Specifically, the home appliance 100 can use the permission information of multiple users to determine whether the identified user has the necessary permissions for the determined action.

[0205] Furthermore, the home appliance 100 can provide feedback S1380 based on whether the user has the necessary permissions. Specifically, if the identified user has the necessary permissions, the home appliance 100 can perform the determined action.

[0206] Conversely, if the identified user lacks authorization or is not a pre-registered user, the determined action may not be performed. In this case, the home appliance 100 can provide guidance to the user that the action cannot be performed. For example, the home appliance 100 can output a preset guidance message sound or display a preset guidance message on the display 180.

[0207] Thus, even if home appliances do not have a speaker recognition model and voice recognition module, a server capable of performing speaker recognition and voice recognition functions can be used to identify the user corresponding to the input voice and determine the action corresponding to the input voice.

[0208] On the one hand, regarding the target Figure 13 The illustrations and explanations provided illustrate the determination of permission information for actions taken by the identified user for home appliances. However, in practice, this can also be achieved by having the server execute the aforementioned actions and send the result of the judgment on whether the user has permission to use the home appliances.

[0209] Figure 14 This is a flowchart illustrating a voice recognition method for a household appliance according to an embodiment of the present invention.

[0210] refer to Figure 14 If either a first event of sensing user behavior or a second event of sensing a trigger command language for activating speech recognition mode occurs, an audio signal S1410 corresponding to the event that occurred is output.

[0211] Specifically, the decision to send the first event can be determined by sensors that detect user behavior. Here, the user behavior can be configured differently depending on the type of sensor used to detect it.

[0212] For example, if the sensor is used to sense the opening and closing of a door installed on the main body of a household appliance, then the user's behavior can be set to opening or closing the door. Therefore, if the door is opened or closed, it can be determined that the first event has occurred.

[0213] Additionally, if the sensor is a motion sensor, the user's behavior could be the user's approach to a household appliance. Therefore, if the approach of a user to a household appliance is sensed, it can be determined that the first event has occurred.

[0214] Furthermore, whether a second event occurs can be determined based on whether the voice input through the microphone of the household appliance includes a trigger command language. Here, the trigger command language can be referred to as a wake-up command language or an activation command language, etc.

[0215] Furthermore, the output audio signal can be set differently for each first event or second event.

[0216] More specifically, the audio signal corresponding to the first event can be a voice message notifying the activation of voice recognition mode. For example, the audio signal corresponding to the first event could be a voice message specifically notifying the activation of voice recognition mode, such as "What help do you need?". On the other hand, the types of voice messages are not limited to the examples above.

[0217] Furthermore, the audio signal corresponding to the second event can be a simple notification sound. For example, the audio signal corresponding to the second event can be a simple notification sound like a "ding-dong". On the other hand, the types of notification sounds are not limited to the examples mentioned above.

[0218] Furthermore, operation S1420 is performed in voice recognition mode. In this voice recognition mode, user questions and responses input through the microphone are processed into natural language action patterns. This voice recognition mode can be implemented either by directly processing the user's questions and responses into natural language using a voice recognition module, or by sending the voice input through the microphone to the server and receiving the voice recognition result from the server.

[0219] Specifically, once operation begins in voice recognition mode, voice recognition can be performed on the voice input through the microphone.

[0220] Furthermore, actions corresponding to the speech recognition results can be executed. On the one hand, before executing an action corresponding to the speech recognition results, the user's permissions can be determined, and the action can be executed based on whether or not the user has the necessary permissions.

[0221] Specifically, if user voice is first input in speech recognition mode, the user's voice can be recognized and the user identified. More specifically, the user can be identified using the user's voice input in speech recognition mode and a speaker recognition model stored in memory.

[0222] Furthermore, the user's permission information can be read from the memory. If the action corresponding to the user's voice matches the read permission information, then the action corresponding to the user's voice (i.e., the action corresponding to the voice recognition result) can be executed.

[0223] Conversely, if the action corresponding to the user's voice does not conform to the reading permission information, or if the user corresponding to the input user voice has not been identified, the action corresponding to the user's voice may not be executed.

[0224] On the one hand, input indicating whether or not an event is activated can be accepted for each of the first and second events. Specifically, input indicating whether or not an event is activated for each of the first and second events can be received through a user interface device installed on the home appliance.

[0225] If the second event is disabled by inputting a deactivation command via the user interface device, the microphone of the home appliance can be turned off in standby mode. Here, standby mode means a state where a preset event is sensed in order to initiate operation in voice recognition mode.

[0226] Furthermore, if the first event occurs subsequently, the microphone can be turned on again. Thus, if the second event is deactivated, there is no need to sense the trigger command language, allowing the microphone to be turned off to prevent power consumption.

[0227] Conversely, if the user interface device inputs a deactivation command for the first event, the microphone of the home appliance can be kept on in standby mode while the sensors sensing user behavior are turned off. Thus, if the user behavior is deactivated, sensing is unnecessary, allowing the sensors to be shut down to prevent power consumption.

[0228] On the one hand, the entity that senses and triggers the command language and the entity that operates in voice recognition mode can be implemented as different entities. Specifically, a home appliance may include a first processor that senses and triggers the command language and a second processor that operates in voice recognition mode.

[0229] Furthermore, the first processor can remain on in standby mode. Also, the first processor can recognize trigger commands, including those uttered by the user's voice.

[0230] Furthermore, if the first processor recognizes a trigger command, it can activate the second processor. Specifically, if the standby state is maintained for a preset time or longer, the second processor can transition from the activated state to the deactivated state. And, if the first processor recognizes a trigger command, it can activate the second processor.

[0231] Therefore, the home appliance control method of the present invention operates in voice recognition mode even if the user does not verbally trigger a command, as long as user behavior predicted to be a voice command is sensed, thus solving the technical effect of solving the inconvenience of the user having to verbally activate a command. Furthermore, upon sensing user behavior, a voice message notifying the activation of voice recognition mode is output, thus specifically notifying the user to operate in voice recognition mode even when it is uncertain whether the user intends to use the home appliance's voice recognition mode, thereby guiding the user to issue control commands or ask questions. Moreover, if the event of sensing a command trigger is disabled, the microphone is turned off in standby mode; if the event of sensing user behavior is disabled, the sensor is turned off in standby mode, thereby saving power consumed by unnecessary components in standby mode.

[0232] and Figure 14 The same speech recognition method can be used in situations where... Figure 2 or Figure 3 It can be applied to household appliances with the same configuration, or it can be applied to household appliances with other different configurations.

[0233] Furthermore, the control method for household appliances described above can be implemented as at least one execution program for performing the control method for household appliances described above, and such execution program can be stored in a non-transitory readable medium.

[0234] Non-transitory readable media are not media that store data for a short period of time, such as registers, caches, and memory. Rather, they mean media that store data semi-permanently and can be read by the machine. Specifically, the various applications or programs mentioned above can be stored and provided on non-transitory readable media such as CDs, DVDs, hard drives, Blu-ray discs, USB drives, memory cards, and ROMs.

[0235] The preferred embodiments of the present invention have been illustrated and described above. However, the present invention is not limited to the specific embodiments described. Of course, those skilled in the art can make various modifications without departing from the spirit of the invention as claimed in the claims. Such modifications are within the scope of the claims.

[0236] Figure Labels

[0237] 100: Home appliances 110: Sensors

[0238] 120: Microphone; 130: Speaker

[0239] 140: Processor; 150: User interface device

[0240] 160: Memory device; 170: Communication device

[0241] 180: Monitor

Claims

1. A household appliance, comprising: Door; sensor; microphone; speaker; as well as processor, The processor is further configured as follows: Based on the identification of one of the first and second events, the system operates in voice recognition mode and controls the speaker to output an audio signal corresponding to the identified first event and one of the second events. The first event is a user action detected by the sensor, and the second event is a trigger command language for activating voice recognition mode input through the microphone. Based on the detection of door closure after operation in the voice recognition mode, the voice recognition mode is switched to standby mode, wherein the operation in the voice recognition mode is set to open the door based on the user behavior. The audio signal is configured differently for each of the first and second events. The standby mode is a state that detects whether a predetermined event has occurred to initiate operations in the speech recognition mode. The household appliances also include: The user interface device is configured to receive selections regarding whether to activate or deactivate the detection of a first event and whether to activate or deactivate the detection of a second event, respectively; and The processor is further configured as follows: Based on disabling the detection of the first event through the user interface device, the microphone remains on in the standby mode, and the sensors used to detect the user's behavior are turned off.

2. The household appliance according to claim 1, in, The sensor is used to detect the opening and closing of the door, and The processor is also configured to: Based on the detection of the door opening by the sensor, the occurrence of the first event is identified, and The occurrence of the first event is identified based on the detection of the door closing by the sensor.

3. The household appliance according to claim 1, in, The sensor is a motion detection sensor, and The processor is also configured to: The occurrence of the first event is determined based on the sensor detecting the user's approach to the home appliance.

4. The household appliance according to any one of claims 1 to 3, in, The audio signal corresponding to the first event is a voice message notifying that the voice recognition mode has been enabled, and The audio signal corresponding to the second event is a simple notification sound.

5. The household appliance according to claim 4, further comprising: in, The processor is also configured to: Based on the detection of the occurrence of the second event, the user interface device is deactivated, and based on the occurrence of the first event, the voice recognition mode is operated.

6. The household appliance according to claim 1, in, The processor includes: A first processor is configured to recognize the trigger command language; and The second processor is configured to operate in the speech recognition mode. The first processor is further configured to be in the standby mode and to output a notification signal to the second processor based on user voice input via the microphone, including the trigger command language. The second processor is further configured to be in a turned-off state in the standby mode and to be turned on upon receiving the notification signal, and to operate in the voice recognition mode.

7. The household appliance according to claim 1, further comprising: The memory is configured to store each user's permission information. The processor is further configured as follows: Based on the user's voice being input through the microphone in voice recognition mode, the user is identified by recognizing the user's voice, and the operation is executed based on the permission information of the identified user matched with the operation corresponding to the user's voice.

8. The household appliance according to claim 1, further comprising: The memory is configured to store the speech recognition module used to perform speech recognition operations.

9. A method for controlling a household appliance, the method comprising: Based on the identification of the occurrence of one of the first event and the second event, the system operates in voice recognition mode and outputs an audio signal corresponding to one of the identified first event and the second event. The first event is the occurrence and detection of user behavior, and the second event is the input of a trigger command language for activating voice recognition mode. Based on the detection of door closure after operation in the voice recognition mode, the voice recognition mode is switched to standby mode, wherein the operation in the voice recognition mode is set to open the door based on the user behavior; and Based on disabling the detection of the first event via the user interface device, the microphone remains on in the standby mode, and the sensors used to detect the user's behavior are turned off. The audio signal is configured differently for each of the first and second events. The standby mode is a state that detects whether a predetermined event occurs to initiate the operation in the voice recognition mode.

10. The control method according to claim 9, further comprising: Based on the opening of the door, it is determined that the first event has occurred.

11. The control method according to claim 9, further comprising: The occurrence of the first event is determined based on the detection of the user's approach to the home appliance.

12. The control method according to any one of claims 9 to 11, in, The audio signal corresponding to the first event is a voice message notifying that the voice recognition mode has been enabled, and The audio signal corresponding to the second event is a simple notification sound.

13. The control method according to claim 12, further comprising: Based on the detection of the occurrence of the second event, the user interface device provided on the home appliance is deactivated, and based on the occurrence of the first event, the voice recognition mode is operated.

14. The control method according to claim 9, further comprising: Based on the fact that the first processor, which is in the on state in standby mode, recognizes that the trigger command language has been input, it outputs a notification signal to the second processor, which is in the off state in standby mode. as well as Upon receiving the notification signal, the second processor is activated and operates in voice recognition mode.

15. The control method according to claim 9, further comprising: The standby mode was maintained for a predetermined period of time, and the state of the second processor was changed from the on state and kept in the off state.

Images