Household appliance interaction method, wearable device and home control system

Abstract

The invention discloses a household appliance interaction method, wearable equipment and a home control system. The method comprises the following steps: receiving interaction data corresponding to a user; performing intention recognition on the interaction data by using a large language model to obtain a target intention; generating dynamic display data according to the target intention; wherein the dynamic display data are voices, images and / or characters corresponding to the interaction data; and displaying the dynamic display data. By means of the mode, the household appliance interaction mode can be more natural and convenient, and the user interaction experience is improved.

Description

Technical Field

[0001] The present application relates to the field of home appliance technology, and in particular to an interaction method for home appliance equipment, a wearable device, and a home control system. Background Art

[0002] With the development of Internet of Things and artificial intelligence technologies, smart home control systems are becoming increasingly popular.

[0003] However, related smart home control methods still have some problems, such as a single input mode and a single interaction method. Summary of the invention

[0004] The home appliance interaction method, wearable device, and home control system provided in this application can make the home appliance interaction mode more natural and convenient, and enhance the user interaction experience.

[0005] In a first aspect, the present application provides an interaction method for household appliances, which is applied to wearable devices, and the method includes: receiving interaction data corresponding to a user; using a large language model to perform intent recognition on the interaction data to obtain a target intent; generating dynamic display data according to the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data; and displaying the dynamic display data.

[0006] Among them, the interaction data includes: voice data, using a large language model to perform intent recognition on the interaction data to obtain the target intent, including: using a large language model to perform intent recognition on the voice data to obtain home appliances in the voice data and the target intent for the home appliances.

[0007] Among them, the interaction data includes: voice data and image data, and the large language model is used to identify the intention of the interaction data to obtain the target intention, including: using the large language model to identify the intention of the voice data and image data to obtain the target intention.

[0008] Among them, the large language model is used to perform intent recognition on voice data and image data to obtain the target intent, including: using the large language model to perform content recognition on voice data to obtain key content; determining the target object corresponding to the key content from the image data; and obtaining the target intent based on the key content and the target object.

[0009] Among them, generating dynamic display data according to the target intention includes: obtaining voice, image and / or text corresponding to the target object; and generating dynamic display data according to the voice, image and / or text.

[0010] The display of dynamic display data includes: obtaining the user's age stage; determining the voice playback speed according to the age stage; and displaying the dynamic display data according to the voice playback speed.

[0011] The method further includes: receiving user instructions during the display of dynamic display data; and adjusting the presentation mode and content of the dynamic display data according to the user instructions.

[0012] The wearable device is a smart glasses, and the display of dynamic display data includes: superimposing images and / or texts in the dynamic display data within the visible range of the smart glasses for display.

[0013] In a second aspect, the present application provides a wearable device, which includes a processor and a memory and a communication interface coupled to the processor, the memory storing at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method provided in the first aspect.

[0014] In a third aspect, the present application provides a home control system, which includes: a cloud; a wearable device, which is communicatively connected to the cloud, receives interaction data corresponding to the user, and sends the interaction data to the cloud; the cloud uses a large language model to perform intent recognition on the interaction data, obtains the target intent, and generates dynamic display data based on the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data; the wearable device receives the dynamic display data and displays it.

[0015] The beneficial effects of the present application are as follows: different from the prior art, the interaction method of home appliances, wearable devices, and home control systems provided by the present application utilize wearable devices to receive interaction data corresponding to users, thereby increasing the interaction input methods of the home control system. Furthermore, a large language model is used to perform intent recognition on the interaction data to obtain the target intent, and dynamic display data is generated according to the target intent, thereby improving the in-depth understanding of the interaction data, improving the accuracy of the dynamic display data, and utilizing wearable devices as a display medium for the dynamic display data, thereby making the interaction method of home appliances more natural and convenient and improving the user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0017] Figure 1 It is a structural schematic diagram of an embodiment of a home control system provided by the present application;

[0018] Figure 2 is a structural schematic diagram of another embodiment of the home control system provided by the present application;

[0019] Figure 3It is a flowchart of an embodiment of a control method for household electrical appliances provided by the present application;

[0020] Figure 4 is a flow chart of another embodiment of the control method of household electrical appliances provided by the present application;

[0021] Figure 5 It is a flowchart of an embodiment of an interaction method for household appliances provided by the present application;

[0022] Figure 6 is a flowchart of another embodiment of the interactive method for household appliances provided by the present application;

[0023] Figure 7 is a flowchart of another embodiment of the interactive method for household appliances provided by the present application;

[0024] Figure 8 This application provides Figure 7 A flow chart of an embodiment of step 72 in FIG.

[0025] Fig. 9 is a flowchart of another embodiment of the interactive method for household appliances provided by the present application;

[0026] Fig.10 is a structural schematic diagram of an embodiment of a wearable device provided by the present application;

[0027] Fig.11 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] With the development of Internet of Things and artificial intelligence technologies, smart home control systems are becoming increasingly popular.

[0031] However, related smart home control methods still have some problems, such as a single input mode, lack of deep understanding of user intentions, and unintuitive presentation of home appliance status information.

[0032] Based on this, the present application proposes to use wearable devices to collect multimodal data, improve the diversity of data collection, increase the interactive input methods of the home control system, and further, use a large language model in the cloud to analyze the multimodal data, generate control instructions, improve the in-depth understanding of the multimodal data, improve the accuracy of the control instructions, and use wearable devices as interactive media to make home appliance control more natural and convenient. Please refer to any of the following embodiments for details.

[0033] See also Figure 1 , Figure 1 1 is a schematic diagram of a structure of an embodiment of a home control system provided by the present application. The home control system 100 includes: a cloud 10 and a wearable device 20.

[0034] The wearable device 20 is connected to the cloud 10 for communication, and sends the collected multimodal data to the cloud 10, where the multimodal data includes at least one of voice data, image data, and posture data.

[0035] The cloud 10 uses a large language model to analyze the multimodal data, generates control instructions, sends the control instructions to the home appliance corresponding to the multimodal data, and receives the device status feedback from the home appliance; the wearable device 20 receives the device status and displays it.

[0036] In some embodiments, the cloud 10 selects a target device state that meets the user's preference from the device states, and sends the target device state to the wearable device 20 .

[0037] In some embodiments, the wearable device 20 is a pair of smart glasses, and the multimodal data includes voice data, image data, and posture data; the cloud 10 uses a large language model to analyze the voice data, image data, and posture data, obtains the type of home appliance corresponding to the voice data, and the home appliance that meets the type of home appliance in the image data, and generates control instructions for the corresponding home appliance.

[0038] In some embodiments, after receiving and displaying the device status, the smart glasses receive gesture instructions and send the gesture instructions to the cloud 10, where the gesture instructions are related to the device status.

[0039] In some embodiments, after receiving and displaying the device status, the smart glasses receive voice instructions and send the voice instructions to the cloud 10, where the voice instructions are related to the device status.

[0040] In some embodiments, after receiving and displaying the device status, the smart glasses receive gesture instructions and voice instructions, and send the gesture instructions and voice instructions to the cloud 10, where the gesture instructions and voice instructions are related to the device status.

[0041] In some embodiments, the smart glasses display the device status superimposed within the visual range of the smart glasses.

[0042] In this embodiment, the wearable device 20 is used to collect multimodal data, improve the diversity of data collection, and increase the interactive input methods of the home control system. Furthermore, the cloud 10 uses a large language model to analyze the multimodal data, generate control instructions, enhance the in-depth understanding of the multimodal data, improve the accuracy of the control instructions, and use the wearable device 20 as an interactive medium to make home appliance control more natural and convenient.

[0043] See also Figure 2 , Figure 2 1 is a schematic diagram of another embodiment of a home control system provided by the present application. The home control system 100 comprises: a cloud 10, a home gateway 30 and a wearable device 20.

[0044] The home gateway 30 is respectively connected to the cloud 10, the wearable device 20 and the home appliance (not shown), receives the multimodal data sent by the wearable device 20 and forwards it to the cloud 10, and receives the control instructions sent by the cloud 10 and forwards them to the home appliance.

[0045] In some embodiments, the cloud 10 selects a target device state that meets the user's preference from the device state, and sends the target device state to the wearable device 20 via the home gateway 30 .

[0046] In some embodiments, the wearable device 20 is a pair of smart glasses, and the multimodal data includes voice data, image data, and posture data; the cloud 10 uses a large language model to analyze the voice data, image data, and posture data, obtains the type of home appliance corresponding to the voice data, and the home appliance that meets the type of home appliance in the image data, and generates control instructions for the corresponding home appliance.

[0047] In some embodiments, after receiving and displaying the device status, the smart glasses receive gesture instructions and send the gesture instructions to the cloud 10 through the home gateway 30, where the gesture instructions are related to the device status.

[0048] In some embodiments, after receiving and displaying the device status, the smart glasses receive voice commands and send the voice commands to the cloud 10 through the home gateway 30, where the voice commands are related to the device status.

[0049] In some embodiments, after receiving and displaying the device status, the smart glasses receive gesture commands and voice commands, and send the gesture commands and voice commands to the cloud 10 through the home gateway 30, where the gesture commands and voice commands are related to the device status.

[0050] In some embodiments, the smart glasses display the device status superimposed within the visual range of the smart glasses.

[0051] In this embodiment, wearable devices are used to collect multimodal data, improve the diversity of data collection, and increase the interactive input methods of the home control system. Furthermore, the cloud uses a large language model to analyze the multimodal data, generate control instructions, enhance the in-depth understanding of the multimodal data, improve the accuracy of the control instructions, and use wearable devices as an interactive medium to make home appliance control more natural and convenient.

[0052] See also Figure 3 , Figure 3 1 is a flow chart of an embodiment of a control method for a household appliance provided by the present application. Applied to a wearable device, the method includes:

[0053] Step 31: Collect multimodal data, where the multimodal data includes at least one of voice data, image data, and posture data.

[0054] In some embodiments, multimodal data can be collected using relevant sensors on the wearable device. For example, voice data can be collected using a microphone on the wearable device. Image data can be collected using an image sensor on the wearable device. Gyroscopes and inertial navigation components on the wearable device can be used to collect attitude data. In other embodiments, attitude data can be collected in combination with image data collected by an image sensor.

[0055] Step 32: Send the multimodal data to the cloud, so that the cloud uses a large language model to analyze the multimodal data and generate control instructions.

[0056] Step 33: Receive and display the device status of the home appliance sent from the cloud; wherein the device status is obtained after the home appliance executes the control instruction.

[0057] In this embodiment, wearable devices are used to collect multimodal data, improve the diversity of data collection, and increase the interactive input methods of the home control system. Furthermore, the cloud uses a large language model to analyze the multimodal data, generate control instructions, enhance the in-depth understanding of the multimodal data, improve the accuracy of the control instructions, and use wearable devices as an interactive medium to make home appliance control more natural and convenient.

[0058] See also Figure 4 , Figure 4This is a flow chart of another embodiment of the control method of the household appliance provided by the present application. Applied in the cloud, the method includes:

[0059] Step 41: receiving multimodal data sent by the wearable device, where the multimodal data includes at least one of voice data, image data, and posture data.

[0060] Step 42: Analyze the multimodal data using the large language model, generate control instructions, and send the control instructions to the home appliance corresponding to the multimodal data.

[0061] Step 43: Receive the device status fed back by the home appliance.

[0062] Step 44: Send the device status to the wearable device.

[0063] In an application scenario, taking the wearable device as smart glasses as an example, the control process of home appliances with multi-modal input is as follows:

[0064] 1) The user issues a control command through the smart glasses, such as saying "turn on the water heater" and looking at the target water heater.

[0065] 2) Smart glasses collect multimodal data such as voice, vision, and head posture.

[0066] 3) The multimodal data is uploaded to the cloud server after preprocessing.

[0067] 4) The cloud server uses a large multimodal model to analyze data and combines contextual information to understand user intent.

[0068] 5) The cloud server generates precise control instructions and sends them to the target water heater through the home gateway.

[0069] 6) The water heater starts and feeds back the result. The user can intuitively view the remaining hot water and water temperature of the water heater through the smart glasses.

[0070] In an application scenario, taking smart glasses as an example, the process of visualizing the status of home appliances based on AR is as follows:

[0071] 1) Smart home appliances (such as dishwashers) report their operating status to the home gateway in real time.

[0072] 2) The home gateway uploads the status information to the cloud server.

[0073] 3) The cloud server generates AR visualization content based on user preferences, such as displaying the current step and remaining time of the dishwasher.

[0074] 4) AR content is sent to smart glasses.

[0075] 5) Smart glasses overlay AR content on the user’s field of view.

[0076] 6) Users can interact with the AR interface through sight, gestures, or voice, such as adjusting the water temperature in a water dispenser.

[0077] In one application scenario, taking the wearable device as smart glasses as an example, the water heater status visualization and control are as follows:

[0078] 1) The user uses the voice command "check the status of the water heater" and looks at the water heater, and the smart glasses collect relevant multimodal input.

[0079] 2) The cloud server uses a multimodal large model to understand the user's intention and retrieve the current operating status data of the water heater, including the remaining hot water, water temperature, etc.

[0080] 3) The cloud server generates an AR interface based on the user's location and line of sight, which is superimposed on the user's field of view, intuitively displaying the various status indicators of the water heater.

[0081] 4) The user can further control the water heater through gestures or voice, such as adjusting the water temperature. The smart glasses send the user's control instructions to the home gateway, which is ultimately communicated to the water heater device.

[0082] 5) The water heater executes the user's control instructions and provides real-time feedback on the latest operating status data. The cloud server updates the display content of the AR interface accordingly.

[0083] In one application scenario, taking the wearable device as smart glasses as an example, the dishwasher status visualization and control are as follows:

[0084] 1) The user says the voice command "check the status of the dishwasher" while focusing on the dishwasher at home.

[0085] 2) Smart glasses collect multimodal inputs such as voice and sight and send them to cloud servers for processing.

[0086] 3) The cloud server multimodal large model understands the user's query intention and pulls the current operating status of the dishwasher from the home gateway, including the washing program stage, remaining time, etc.

[0087] 4) The cloud server generates an AR interface based on the user's location and line of sight, which is superimposed on the user's field of view, intuitively displaying the dishwasher's status information.

[0088] 5) The user can further control the dishwasher through gestures or voice, such as adjusting the water temperature or pausing / resuming the washing process. The smart glasses send the user's control instructions to the home gateway, which is ultimately conveyed to the dishwasher device.

[0089] 6) The dishwasher executes the user's control instructions and provides real-time feedback on the latest operating status data. The cloud server updates the display content of the AR interface accordingly.

[0090] In the above embodiments of the present application, multimodal input improves the accuracy of home appliance control and reduces operation ambiguity. Context awareness makes the system smarter and can predict user needs. AR status visualization provides intuitive and real-time home appliance information and improves user experience. Smart glasses as an interactive medium make home appliance control more natural and convenient. The home control system has good scalability and is easy to integrate new home appliances and functions, such as water dispensers, TVs, etc.

[0091] In some embodiments, with the development of the Internet of Things and artificial intelligence technology, smart home control systems are becoming increasingly popular. However, related smart home control methods still have some problems, such as a single input mode, a single interaction method, etc.

[0092] Based on this, the present application proposes to use wearable devices to receive user-corresponding interaction data, increase the interactive input mode of the home control system, further, use a large language model to perform intent recognition on the interaction data, obtain the target intent, and generate dynamic display data according to the target intent, thereby improving the in-depth understanding of the interaction data, improving the accuracy of the dynamic display data, and using wearable devices as a display medium for the dynamic display data, so as to make the home appliance interaction mode more natural and convenient and enhance the user interaction experience. Please refer to any of the following embodiments for details.

[0093] See also Figure 5 , Figure 5 1 is a flow chart of an embodiment of an interaction method for household appliances provided by the present application. Applied to a wearable device, the method includes:

[0094] Step 51: Receive interaction data corresponding to the user.

[0095] The interaction data includes at least one of voice data, image data and gesture data.

[0096] In some embodiments, the interaction data at least includes a target object, wherein the target object may be a function in a home appliance, a functional component in a home appliance, or a home appliance.

[0097] In some embodiments, the target object may be a product in a home environment, such as a mobile phone, a tablet, and / or a recipe.

[0098] Step 52: Use the large language model to perform intent recognition on the interaction data to obtain the target intent.

[0099] In some embodiments, the large language model is a model trained in advance. In some embodiments, the target intent may represent an understanding intent of a target object of the home appliance.

[0100] Step 53: Generate dynamic display data according to the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data.

[0101] In some embodiments, the target object that needs to be understood can be obtained from the target intent, and then the voice, image and / or text corresponding to the target object can be obtained; dynamic display data can be generated based on the voice, image and / or text.

[0102] For example, if the large language model is used to identify the manual of a home appliance that needs to be known, the voice, image and / or text related to the manual can be obtained based on the home appliance to form dynamic display data.

[0103] For example, if a large language model is used to identify that one needs to know how a dish is prepared, then voice, images and / or text related to the preparation of the dish can be obtained based on the dish to form dynamic display data.

[0104] In some embodiments, the image may be a video image and / or a single-frame image, that is, the dynamic display data may be composed of a video and multiple pictures.

[0105] Step 54: Display dynamic display data.

[0106] Wearable devices are used to display dynamic data so that users can quickly and intuitively understand relevant content through wearable devices.

[0107] In some embodiments, the dynamically displayed data includes images and / or voices, and the wearable device can be used to display images and / or voices to solve the problem of users being illiterate or insensitive to text, thereby improving the user experience.

[0108] In this embodiment, wearable devices are used to receive interaction data corresponding to users, and interactive input methods of the home control system are increased. Furthermore, a large language model is used to identify the intent of the interaction data to obtain the target intent, and dynamic display data is generated according to the target intent, thereby improving the in-depth understanding of the interaction data, improving the accuracy of the dynamic display data, and using wearable devices as a display medium for the dynamic display data, making the interaction method of home appliances more natural and convenient, and improving the user interaction experience.

[0109] See also Figure 6 , Figure 6 1 is a flow chart of another embodiment of the interaction method of the household appliance provided by the present application. Applied to a wearable device, the method includes:

[0110] Step 61: Receive the user's voice data.

[0111] In some embodiments, the voice data at least includes a target object. The target object may be a function in a home appliance and / or a functional component in the home appliance. For example, the corresponding text content in the voice data is: I want to know what functions the dishwasher has. Then the target object is the dishwasher.

[0112] In some embodiments, the target object may be a product in a home environment, such as a mobile phone, a tablet, and / or a recipe.

[0113] Step 62: Use the large language model to perform intent recognition on the voice data to obtain the home appliances in the voice data and the target intent for the home appliances.

[0114] In some embodiments, the speech data is converted into text using a large language model to obtain text data, and then the text data is subjected to intent recognition to obtain the home appliance in the speech data and the target intent for the home appliance. For example, the corresponding text content in the speech data is: I want to know what functions the dishwasher has. Then the home appliance in the speech data is a dishwasher, and the target intent is to know the functions of the dishwasher.

[0115] Step 63: Generate dynamic display data according to the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data.

[0116] In some embodiments, the target object that needs to be understood can be obtained from the target intent as a dishwasher, and then the voice, image and / or text corresponding to all functions of the dishwasher can be obtained; dynamic display data is generated based on the voice, image and / or text.

[0117] Step 64: Display dynamic display data.

[0118] In this embodiment, a wearable device is used to interact with a user and receive the user's voice data; a large language model is used to perform intent recognition on the interaction data to obtain the target intent; dynamic display data is generated according to the target intent, and the wearable device is used as a display medium for the dynamic display data, making the interaction mode of home appliances more natural and convenient, and improving the user interaction experience.

[0119] See also Figure 7 , Figure 7 1 is a flow chart of another embodiment of the interaction method of the household appliance provided by the present application. Applied to a wearable device, the method includes:

[0120] Step 71: Receive the user's voice data and image data.

[0121] In some embodiments, the image data can be collected within the collection field of view of the image sensor of the household appliance on the wearable device.

[0122] For example, when the user's voice data is collected, the image data is collected synchronously.

[0123] Step 72: Use the large language model to perform intent recognition on the voice data and image data to obtain the target intent.

[0124] In some embodiments, a large language model can be used to simultaneously perform intent recognition on voice data and image data, and then combine the content in the voice data and the target object in the image data to obtain the target intent.

[0125] In some embodiments, see Figure 8 , step 72 may be the following process:

[0126] Step 721: Use a large language model to perform content recognition on the speech data to obtain key content.

[0127] For example, the corresponding text content in the voice data is: What is this useful for? Using the large language model to perform content recognition on the voice data, the key content obtained is: I want to know the function of something. At this time, we do not know what this thing is specifically.

[0128] Step 722: Determine the target object corresponding to the key content from the image data.

[0129] Furthermore, after obtaining the key content from the voice data, the large language model determines the target object corresponding to the key content from the image data. For example, if the large language model recognizes the button of the range hood from the image data, the button can be used as the target object.

[0130] Step 723: Obtain target intent based on key content and target object.

[0131] In some embodiments, when the key content of the voice data is: wanting to know the function of something, and the target object in the image data is the buttons of a range hood, the target intention is to know the functions of the buttons of the range hood.

[0132] Step 73: Generate dynamic display data according to the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data.

[0133] For example, if the target intention is to know the functions of the buttons of the range hood, dynamic display data can be generated according to the intention, and the dynamic display data can be a dynamic instruction manual. Through the display of the wearable device, the user can know the functions of the buttons of the range hood.

[0134] Step 74: Display dynamic display data.

[0135] In some embodiments, when displaying dynamic display data using a wearable device, the user's age stage can be obtained; the voice playback speed can be determined according to the age stage; and the dynamic display data can be displayed according to the voice playback speed. For example, the higher the age stage, the older the user is, and the slower the voice playback speed. For example, if the user is in the elderly stage, when displaying the dynamic display data, the relevant content of the dynamic display data can be played according to the voice playback speed corresponding to the elderly stage.

[0136] In this embodiment, wearable devices are used to interact with users and receive voice data and image data between users and home appliances. A large language model is used to perform intent recognition on the voice data and image data to obtain the target intent. Dynamic display data is generated based on the target intent, and wearable devices are used as a display medium for the dynamic display data, making the interaction mode of home appliances more natural and convenient, thereby enhancing the user interaction experience.

[0137] In the present application, the wearable device may be one of a smart bracelet, a smart watch and / or smart glasses. When displaying dynamic display data, corresponding display is performed according to the data type of the dynamic display data.

[0138] In some embodiments, the wearable device is a smart glasses, and the images and / or text in the dynamic display data can be superimposed within the visual range of the smart glasses for display. For example, if the dynamic display data includes an image, the image is superimposed within the visual range of the smart glasses for display.

[0139] For example, if the dynamically displayed data includes text, the text will be superimposed within the visible range of the smart glasses during display.

[0140] For example, if the dynamic display data includes text and images, the text and images are superimposed in a corresponding arrangement within the visual range of the smart glasses during display.

[0141] During this process, if the dynamic display data also includes voice, the voice can be played during the display process.

[0142] In some embodiments, during the display of dynamic display data, a user instruction is received; and the presentation mode and content of the dynamic display data are adjusted according to the user instruction, such as adjusting the display speed and the playing accent, such as changing from Mandarin to a dialect or from Mandarin to other languages.

[0143] See also Fig. 9 , Fig. 9 This is a flow chart of another embodiment of the method for interacting with household appliances provided by the present application. The method includes:

[0144] Step 91: Receive the user's cooking requirements.

[0145] The cooking requirement can be sent to the wearable device in the form of voice, image, text, etc. That is, the wearable device receives voice data, image data and / or text data with the cooking requirement.

[0146] Step 92: Input the dish demand into the large language model to obtain the dish production data corresponding to the dish demand.

[0147] Step 93: Display the dish preparation data.

[0148] In the process of displaying the dish preparation data, user instructions are received; and the presentation method and content of the dish preparation data are adjusted according to the user instructions.

[0149] In this embodiment, wearable devices are used to interact with users and receive users' dish making requirements; and dish making data corresponding to the dish making requirements are displayed using a large language model. This solves the problems of lack of interactivity and intuitiveness of recipes in related technologies, allowing users to understand the operating steps more intuitively and improve the user experience.

[0150] In one application scenario, taking the wearable device as smart glasses as an example, a dynamic manual for a water heater based on smart glasses. The elderly user wears smart glasses, looks at a function button on the water heater, and issues a voice command in dialect, "I want to know what this button does." Smart glasses capture the user's voice and visual information and transmit it to a multimodal large model (large language model). The multimodal large model recognizes the dialect of the elderly user, understands that the user's intention is to understand the function of the function button, and generates a dynamic manual containing multiple modal information such as large font text, high-contrast pictures, slow speech, simplified AR animation, etc. The dynamic manual is presented in the user's field of view through the AR display function of the smart glasses, for example, the function of the function button is displayed on the screen in the form of large font text and slow speech, and the operation steps of the function are superimposed on the image of the water heater in the form of a simplified AR animation.

[0151] In one application scenario, taking wearable devices as smart glasses as an example, a multimodal recipe for braised pork based on a multimodal large model. An elderly user selects the braised pork dish he wants to learn through his mobile phone. The system transmits the dish information to the multimodal large model (large language model). The multimodal large model generates a braised pork recipe containing multiple modal information such as detailed text steps, clear pictures, slow voice explanations, and segmented video demonstrations. Users can watch the video demonstration of the recipe, listen to the slow voice explanation, or browse the detailed text steps and clear pictures through their mobile phones. The system can adjust the presentation and content of the recipe based on the user's learning progress and feedback, such as pausing the video, repeating the explanation, etc., such as providing a more detailed explanation or demonstration when the user encounters difficulties.

[0152] See also Fig.10 , Fig.10 1 is a schematic diagram of a structure of an embodiment of a wearable device provided by the present application. The wearable device 20 includes a processor 201, a memory 202 and a communication interface 203 coupled to the processor 201, and the memory 202 stores at least one computer program, which is used to implement the method of any of the above embodiments when loaded and executed by the processor.

[0153] See also Fig.11 , Fig.11 1 is a schematic diagram of the structure of an embodiment of an electronic device provided by the present application. The electronic device 300 includes a processor 301, a memory 302 and a communication interface 303 coupled to the processor 301, and the memory 302 stores at least one computer program, which is used to implement the method of any of the above embodiments when loaded and executed by the processor.

[0154] The electronic device 300 may be any of the above-mentioned cloud devices, wearable devices, and home gateways.

[0155] In some embodiments, the wearable device in the above-mentioned home control system 100 is connected to the cloud for communication, receives the interaction data between the user and the home appliance, and sends the interaction data to the cloud. The cloud uses a large language model to perform intent recognition on the interaction data, obtains the target intent, and generates dynamic display data according to the target intent; wherein the dynamic display data is the voice, image and / or text corresponding to the interaction data; and displays the dynamic display data. That is, the above-mentioned home control system 100 can implement the method of any embodiment of the present application.

[0156] To summarize, the interaction method for home appliances, wearable devices, and home control systems provided by the present application utilize wearable devices to receive interaction data corresponding to users, thereby increasing the interaction input methods of the home control system. Furthermore, a large language model is used to perform intent recognition on the interaction data to obtain the target intent, and dynamic display data is generated based on the target intent, thereby improving the in-depth understanding of the interaction data, improving the accuracy of the dynamic display data, and utilizing wearable devices as a display medium for the dynamic display data, thereby making the interaction method of home appliances more natural and convenient and improving the user interaction experience.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0158] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processing circuit component (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0159] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for interacting with household appliances, characterized in that: Applied to a wearable device, the method comprises: Receive interaction data corresponding to the user; Using a large language model to perform intent recognition on the interaction data to obtain a target intent; Generate dynamic display data according to the target intention; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data; The dynamic display data is displayed.

2. The method according to claim 1, characterized in that The interaction data includes: voice data, and the use of a large language model to perform intent recognition on the interaction data to obtain a target intent includes: The large language model is used to perform intent recognition on the voice data to obtain the home appliance in the voice data and the target intent for the home appliance.

3. The method according to claim 1, characterized in that The interaction data includes: voice data and image data, and the use of a large language model to perform intent recognition on the interaction data to obtain a target intent includes: The large language model is used to perform intent recognition on the voice data and the image data to obtain the target intent.

4. The method according to claim 3, characterized in that: The using the large language model to perform intent recognition on the voice data and the image data to obtain the target intent includes: Using the large language model to perform content recognition on the speech data to obtain key content; Determining a target object corresponding to the key content from the image data; The target intent is obtained based on the key content and the target object.

5. The method according to claim 4, characterized in that The generating of dynamic display data according to the target intent includes: Acquire the voice, image and / or text corresponding to the target object; The dynamic display data is generated according to the voice, the image and / or the text.

6. The method according to any one of claims 1 to 5, characterized in that: The displaying of the dynamic display data includes: Get the age stage of the user; Determining the speech playback speed according to the age stage; The dynamic display data is displayed according to the voice playback speed.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: In the process of displaying the dynamic display data, receiving a user instruction; The presentation mode and content of the dynamic display data are adjusted according to the user instruction.

8. The method according to any one of claims 1 to 5, characterized in that: The wearable device is a smart glasses, and the displaying of the dynamic display data includes: The images and / or texts in the dynamic display data are superimposed and displayed within the visual range of the smart glasses.

9. A wearable device, characterized in that: The wearable device includes a processor and a memory and a communication interface coupled to the processor, the memory storing at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method according to any one of claims 1 to 8.

10. A home control system, characterized in that: The home control system comprises: Cloud; The wearable device is connected to the cloud for communication, receives the interaction data corresponding to the user, and sends the interaction data to the cloud; The cloud uses a large language model to perform intent recognition on the interaction data, obtains a target intent, and generates dynamic display data according to the target intent; wherein the dynamic display data is voice, image and / or text corresponding to the interaction data; The wearable device receives and displays the dynamic display data.

Images