Device control method, mapping relationship establishment method, and related device

Abstract

The application discloses a device control method and related device. The method comprises: obtaining data for expressing a target control intention for a target device; based on the data, determining a device control mode matched with the data by querying a pre-defined mapping relationship; the mapping relationship comprises a mapping relationship between data for expressing different control intentions for the target device and corresponding device control modes; and controlling the target device according to the device control mode. The application also provides a device control device, a device, a computer readable storage medium storing a computer program and a computer program product. The scheme provided by the application can solve the problem of long time consumption in determining a device control intention and low efficiency of the device control mode in the prior art, and ensure a good user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a device control method, a mapping relationship establishment method and related devices. BACKGROUND

[0002] An intelligent home platform (also known as a home automation platform), such as Home Assistant, is a software system used to control and manage various home devices in a home, realizing home automation.

[0003] At present, the intelligent home platform mainly uses retrieval-augmented generation (RAG) technology to realize control of home devices. Based on this technology, the device control process can be as shown in Figure 1 The process includes the following steps:

[0004] When the intelligent home device platform detects a human voice signal exceeding a certain decibel (such as 40db), it is considered that the user has issued a home device control signal, thereby converting the detected human voice signal into text, and then converting the text into a target vector - for example, a text embedding model (M3E) can be used to convert the text into a dense vector representation;

[0005] Based on the target vector obtained by conversion, a pre-set vector library (such as a PostgreSQL database) is retrieved - the vector library stores vectors obtained by converting knowledge in various professional fields related to home devices;

[0006] If the target vector hits a certain vector in the vector library, the platform will use the relevant knowledge of the home device corresponding to the vector as a prompt word for the large model; the above text and prompt word are input into the large model, so that the large model can understand the device control intent expressed by the text based on the prompt word;

[0007] The platform controls the target home device based on the device control intent understood by the large model.

[0008] The above process has the following defects: in order to accurately determine the device control intent, the large model needs to be involved. However, the running of the large model often takes a long time, which will make the determination of the device control intent take too long and be inefficient, affecting the user experience. SUMMARY

[0009] The embodiments of the present application provide a device control method to solve the problem that in the prior art, the determination of the device control intent takes too long and is inefficient due to the need to determine the device control intent based on a large model.

[0010] The embodiment of the present application further provides a device control apparatus, an electronic device, a computer readable storage medium and a computer program product.

[0011] The embodiment of the present application adopts the following technical solutions:

[0012] In one aspect, a device control method is provided, comprising:

[0013] obtaining data used to express a target control intention for a target device;

[0014] based on the data, determining a device control mode matched with the data by querying a pre-defined mapping relationship; the mapping relationship comprises a mapping relationship between data respectively used to express different control intentions for the target device and corresponding device control modes;

[0015] controlling the target device according to the device control mode.

[0016] In another aspect, a mapping relationship establishment method is provided, comprising:

[0017] obtaining first description data of functions of the target device;

[0018] obtaining second description data of device control modes used to implement the functions;

[0019] using a preset mapping relationship establishment strategy to establish a mapping relationship between data respectively used to express different control intentions for the target device and corresponding device control modes;

[0020] the mapping relationship establishment strategy comprises: for each function, respectively performing: determining data of a control intention for the function based on the first description data of the function, and determining a device control mode used to implement the function based on the second description data of the device control mode used to implement the function.

[0021] In another aspect, a device control apparatus is provided, comprising:

[0022] a control intention data obtaining unit configured to obtain data used to express a target control intention for a target device;

[0023] a control mode determining unit configured to, based on the data, determine a device control mode matched with the data by querying a pre-defined mapping relationship; the mapping relationship comprises a mapping relationship between data respectively used to express different control intentions for the target device and corresponding device control modes;

[0024] a device control unit configured to control the target device according to the device control mode.

[0025] In another aspect, a mapping relationship establishing apparatus is provided, comprising:

[0026] a first description data obtaining unit configured to obtain first description data of a function of the target device;

[0027] a second description data obtaining unit configured to obtain second description data of a device control mode for implementing the function;

[0028] a mapping relationship establishing unit configured to establish, by using a preset mapping relationship establishing strategy, a mapping relationship between data respectively expressing different control intentions for the target device and corresponding device control modes;

[0029] the mapping relationship establishing strategy comprises, for each function, respectively performing: determining, based on the first description data of the function, data of a control intention for the function, and determining, based on the second description data of the device control mode for implementing the function, the device control mode for implementing the function.

[0030] In another aspect, an electronic device is provided, comprising a memory and a processor, wherein,

[0031] the memory is configured to store a computer program;

[0032] the processor is coupled to the memory and is configured to execute the computer program stored in the memory, so as to execute the above-mentioned device control method or mapping relationship establishing method.

[0033] In another aspect, a computer readable storage medium storing a computer program is provided, the computer program being executable by a computer to implement the above-mentioned device control method or mapping relationship establishing method.

[0034] In another aspect, a computer program product is provided, comprising a computer program executable by a processor to implement the above-mentioned device control method or mapping relationship establishing method.

[0035] The above-mentioned at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0036] According to the scheme provided in the embodiment of the present application, in the case that the data for expressing the target control intention to the target device is acquired, the control mode of the target device can be determined by querying the mapping relationship between the data for expressing the target control intention to the target device and the device control mode in advance according to the data. Compared with the prior art of "understanding the device control intention based on a large model, and then controlling the device", since the mapping relationship is relatively simple and easy to implement, the determination process of the device control mode is more efficient and has a shorter response time, which can ensure good user experience. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and its description, and do not constitute improper limitations to the present application. In the drawings:

[0038] Figure 1 A specific implementation flowchart of a device control method in the prior art;

[0039] Figure 2 A specific implementation flowchart of a device control method provided in the embodiment of the present application;

[0040] Figure 3 A specific implementation flowchart of establishing a mapping relationship between data of different control intentions and corresponding device control modes provided in the embodiment of the present application;

[0041] Figure 4 A specific implementation flowchart of establishing a mapping relationship between data of different control intentions and corresponding device control modes provided in the embodiment of the present application;

[0042] Figure 5 A specific implementation flowchart of a device control method provided in the embodiment of the present application;

[0043] Figure 6 A specific implementation flowchart of another device control method provided in the embodiment of the present application;

[0044] Figure 7 A specific structure schematic diagram of a device control apparatus provided in the embodiment of the present application;

[0045] Figure 8 A specific structure schematic diagram of a mapping relationship establishment apparatus provided in the embodiment of the present application;

[0046] Figure 9 A structure schematic diagram of an electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in connection with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0048] Those of ordinary skill in the art can know that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0049] The terms "first", "second", and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0050] In order to solve the problem of long time spent in determining device control intention and low efficiency of the device control method adopted by the prior art, the embodiments of the present application provide a device control method.

[0051] The execution subject of the method can be various types of computing devices, or can be an application or application (Application, APP) installed on a computing device. The computing device can be a user terminal such as a mobile phone, a tablet computer, a smart wearable device, etc., or can be a server, a control device of a smart home system, etc.

[0052] Among them, the control device of the smart home system can be a client of Home Assistant. Home Assistant is an open source smart home platform, which is a Python-based home automation system. Home Assistant can help users control and manage various devices in the home through various sensors, smart devices (such as smart light bulbs, thermostats, door locks, etc.), and mobile applications or voice assistants (such as Amazon Alexa, Google Assistant, etc.), to realize home automation.

[0053] For ease of description, the embodiments of the present application take the client of the Home Assistant as an example to introduce the method. Those skilled in the art can understand that the embodiments of the present application take the control device of the smart home system as an example to introduce the method, which is only an exemplary description and does not limit the scope of protection of the claims corresponding to the present scheme.

[0054] In addition, different steps of the method can be implemented by different computing devices or software, and the embodiments of the present application do not limit which step is implemented by which computing device or software.

[0055] Please refer to the accompanying drawings of the specification Figure 2 A device control method flow diagram provided by the embodiments of the present application includes the following steps:

[0056] Step 11: Obtain data for expressing the target control intention for the target device.

[0057] The target device refers to the device that the user wants to control. In the smart home scenario, for example, the target device can be a smart lamp, a smart TV, a security camera, etc.

[0058] The target control intention refers to the explicit expression of the user's desired state or action performed by the target device. For example, the target device is a smart lamp, and the user's target control intention for the smart lamp can be, but is not limited to, one or more of turning on / off the smart lamp, adjusting the brightness of the smart lamp, changing the color temperature setting of the smart lamp, and switching the mode of the smart lamp, etc.

[0059] The data for expressing the target control intention for the target device refers to the data that can explicitly indicate the target device to perform an operation or achieve a certain state.

[0060] In the embodiments of the present application, the data for expressing the target control intention for the target device can be any format of data, such as structured data, such as numbers, dates, texts, etc.; or unstructured data, such as pictures, audio, video, etc.

[0061] In a typical smart home scenario, voice data obtained by collecting and analog-to-digital converting a sound signal uttered by a user can be used as data for expressing a target control intent for a target device. For example, a user can use a client of Home Assistant to collect a sound signal of "please adjust the temperature of the air conditioner to 26°C" uttered by the user, and then the client of Home Assistant converts the sound signal into corresponding voice data as data for expressing a target control intent for a target device. Subsequently, the client of Home Assistant can implement corresponding control for the target device based on the voice data.

[0062] Considering that a user can express a target control intent for a target device through different expression contents and expression approaches, in embodiments of the present application, data expressing the following contents can be regarded as data for expressing all or part of a target control intent for a target device:

[0063] A specific operation type for a target device. For example, taking a smart electric lamp as an example, the specific operation type can include, but is not limited to, one or more of turning off, turning on, increasing brightness, and reducing brightness. In a specific example, a user can express a target control intent including a specific operation type of "turning off" for a target device electric lamp by uttering a sound signal "good night". The client of Home Assistant can obtain data for expressing a target control intent for a target device by collecting the sound signal.

[0064] A control parameter related to an operation. The control parameter is used to make the target device perform a control operation to reach a state desired by the user, such as the brightness level of a smart electric lamp, the temperature value, the color, etc. For example, taking an air conditioner as an example, when the operation is to adjust the temperature of the air conditioner, the control parameter related to the operation can be, for example, 26°C. In a specific example, a user can express a target control intent including a control parameter of "26°C" by uttering a sound signal "please adjust the temperature of the air conditioner to 26°C". The client of Home Assistant can obtain data for expressing a target control intent for a target device by collecting the sound signal.

[0065] Trigger condition of operation. The trigger condition can be a specific instruction, a specific event, a specific time point, etc. For example, taking the air conditioner as an example, when the operation is to start, the trigger condition can be when the specified time (such as 8 o'clock at night) arrives. In a specific example, the user can express the target control intention containing the trigger condition of “8 o'clock at night” by issuing a sound signal “please turn on the lights in the whole house at 8 o'clock at night”. The client of Home Assistant can obtain the data used to express the target control intention for the target device by collecting the sound signal.

[0066] In an optional implementation, a target control intention for a target device can be a target control intention for one target device or a target control intention for multiple target devices. For example: the control operation of implementing the control operation of starting the smart lamp, starting the air conditioner, and playing the specified music through the smart sound box implements the target control intention of “master home scene”, which contains multiple target devices: smart lamp, air conditioner, and smart sound box.

[0067] In an optional implementation, the same target control intention can include one control operation for a target device or multiple control operations. For example: the target control intention of implementing the control operation of starting the sleep mode of the air conditioner contains multiple control operations: starting the air conditioner, temperature setting, wind speed adjustment, louver direction adjustment, and multiple control operations.

[0068] Step 12: Based on the data used to express the target control intention for the target device, the device control mode matched with the data is determined by querying the pre-defined mapping relationship.

[0069] In the embodiment of the application, the mapping relationship includes the mapping relationship between the data used to express different control intentions for the target device and the corresponding device control mode.

[0070] In an optional implementation, the device control mode can be an information set used to describe the information composition of the device control instruction. Based on the information used to describe the device control instruction, the specific content of the corresponding device control instruction can be determined subsequently.

[0071] For example, for the air conditioner device, the main device control instructions can include: start instruction, stop instruction, temperature adjustment instruction, dehumidification instruction, heating instruction, cooling instruction, and wind sweeping instruction, etc. The information used to describe the instructions can be 0000, 0001, 0011, 0012, 0015, 0018, and 0020, respectively. For the smart television device, the main device control instructions can include: online playing instruction, local playing instruction, opening application game instruction, and opening audio-visual experience instruction, etc. The information used to describe the instructions can be 1010, 1011, 1021, and 1022, respectively.

[0072] It should be noted that if the device control mode is an information set constituted by information for describing device control instructions, the correspondence between information for describing different device control instructions and corresponding device control instructions can be set in advance in the client of the Home Assistant, so that the client of the Home Assistant can subsequently determine the corresponding device control instruction according to the information for describing the device control instruction.

[0073] In another optional implementation, the device control mode can be an instruction set directly constituted by device control instructions.

[0074] For example, for a smart light device, in the Home assistant system, the control operation on the smart light device can be implemented by calling a specified service (for example, the service for controlling the light can be light.turn_on, light.turn_off), and the control operation on the smart light device is, for example, turning on the living room light and setting the brightness to 65. The corresponding device control instruction can be, for example:

[0075] service: light.turn_on

[0076] entity_id: light.living_room_light

[0077] brightness: 65 ;

[0078] Turning off the living room light, the corresponding device control instruction can be, for example:

[0079] service: light.turn_off

[0080] entity_id: light.living_room_light.

[0081] As described above, by performing step 11, the data for expressing the control intention for the target device is obtained, which can be text data, voice data or image data, etc. On the basis of obtaining the data, in order to control the target device, the device control mode mapped with the data can be further queried. Since the device control mode is an information set constituted by information for describing device control instructions, or an instruction set directly constituted by device control instructions, based on the queried device control mode, it can be determined what kind of control instruction can be used to control the target device for the control intention expressed by the data.

[0082] In summary, the mapping relationship established in the embodiments of the present application can associate the control intention for the target device with the corresponding relationship between the actual control command or action, and the system understands the user's demand by querying the above mapping relationship to convert into the corresponding device control operation.

[0083] For example, for a smart light device, the data of the target device control intention can include: smart light on, smart light off, smart light adjust brightness to 65, smart light adjust color temperature to 4000, etc. The mapping relationship between the control mode can be established and recorded in the following table:

[0084]

[0085] Through the corresponding relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode in the above table, the mapping relationship between the smart light "turn on the living room light" intention and the "turn on the living room light" control mode is established.

[0086] In an optional embodiment, each mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode can be provided with a unique identifier, for example, a unique "intention ID" can be provided for each mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode. When the client of the Home Assistant receives the user intention, the "control intention description" field is queried, and the "intention" that meets the condition is queried, and the corresponding "control mode" is found according to the unique identifier "intention ID".

[0087] In an optional embodiment, as shown in Figure 3 The mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode in the above step 12 can be established by the following sub-steps:

[0088] Step 21: Obtain the first description data of the function of the target device.

[0089] The description data of the function of the target device refers to the data of the literal exposition or interpretation information of the specific operation, service provision, etc. that can be performed by the target device.

[0090] In an optional embodiment, the function list of the target device can be selected as the first description data of the target device. The function list is a list that lists all available functions of the target device in the smart home system. For a target device, the function list can include: function name, brief description of each function, etc.

[0091] In an alternative embodiment, the list of functions can be obtained by reading an installation document provided by the target device manufacturer. The list of functions provided by the device manufacturer can be stored in the installation package of the target device and copied to a specified location in the smart home system when the device is installed. The list of functions can also be obtained by reading a device installation file stored in a fixed location of the storage space provided by the target device, and obtained directly from the storage space of the target device when the target device is installed. The list of functions can also be obtained by reading a device resource package stored in an online resource, and obtained from a specified location according to the recorded address when the target device is installed.

[0092] In an alternative embodiment, considering that the smart home system can be provided with a device management service, the list of functions of the target device can also be obtained by the device management service. Specifically, when the device integration is discovered by the device management service, the smart home system can be notified to read the device management service through an agent, and the list of functions of the target device can be obtained after conversion.

[0093] The device management service can uniformly manage the devices in the smart home system, including centralized management, monitoring, configuration, and troubleshooting of the target device.

[0094] An agent is an automatic inference and decision engine that can accept user input or queries and make internal decisions to execute the query to return the correct result. Depending on its specific functions, the agent can have a device information obtaining agent, a reasoning agent, a retrieval agent, a device management agent, and a device control agent, which respectively implement the functions of information obtaining, reasoning, retrieval, obtaining the list of functions of the device from the device management service, and controlling the target device.

[0095] Step 22: obtaining second description data of the device control mode for implementing the function;

[0096] The description data of the device control mode for implementing the function refers to data that records textual descriptions or explanatory information on how to operate the target device to perform the function.

[0097] In an alternative embodiment, the description data of the device control mode for implementing the function of the target device can be obtained by obtaining an instruction document of the target device. The instruction document of the target device is a document that describes one or more of the functions, operation modes, technical specifications, or maintenance requirements of the target device. The instruction document can include the control mode of the functions that the target device can implement, such as the operation flow of each function (steps for starting, running, and shutting down the device), how to operate the device (information describing device control instructions, instructions that can implement the operation of the device, etc.), and usage precautions.

[0098] In an alternative embodiment, the description document of the target device can be obtained by a vendor-provided installation package, built-in storage space of the target device, online acquisition, etc. as previously described. The description document of the target device can also be obtained by obtaining third-party-provided materials, such as searching through a search engine, obtaining user-uploaded materials to a knowledge base, etc.

[0099] In an alternative embodiment, considering that the client of the Home Assistant can be installed with an agent-related service, the agent can be used to obtain the description document of the smart device provided by a third party, such as searching through a search engine, obtaining user-uploaded materials to a knowledge base, etc., and generate a txt file.

[0100] Step 23: Establishing a mapping relationship according to a preset mapping relationship establishment strategy. The mapping relationship establishment strategy includes: for each function, respectively performing: determining data of a control intention for the function based on first description data of the function, and determining a device control mode for implementing the function based on second description data of the device control mode for implementing the function.

[0101] The mapping relationship establishment strategy refers to a method of creating a corresponding relationship between different sets. Specifically, in this embodiment, it is the corresponding relationship between the data of the target control intention and the target device control mode.

[0102] In an alternative embodiment, the data of the target control intention can be determined from the description of each function in the first description data.

[0103] For example, the first description data of the smart electric lamp is a function description text of the smart electric lamp, which lists each function and a brief description:

[0104] Electric lamp on: the electric lamp is lit when the power is turned on.

[0105] Electric lamp off: the electric lamp is turned off when the power is turned off.

[0106] Brightness adjustment: the user can adjust the brightness of the light as needed.

[0107] Color temperature adjustment: allows the user to adjust the color temperature of the light from warm to cold.

[0108] Color change: supports RGB or full-color adjustment, which can change the color of the light to adapt to different scenes.

[0109] Timing task: a timer can be set to automatically turn on or off the light.

[0110] Scene mode: pre-set a variety of lighting scenes, such as reading, watching, dining, party, etc.

[0111] For the first description data described above, the content in the first description data can be identified by the basic function of the agent, and the function name text: light on, light off, brightness adjustment, color temperature adjustment, color change, timing task, scene mode, etc. The text of the function can be extracted as the data of the control intention of the intelligent lamp device for each function, and the input data that can reflect the meaning of the function text can be used as the data of the target control intention for the function.

[0112] In this embodiment, the voice input by the sound collection device, the text input collected by the client, and any other data used to express the target control intention of the target device can reflect the meaning of the function text, that is, can be used as the data of the target control intention for the function.

[0113] In an optional implementation, the description of the control mode of each function in the second description data can be obtained as the target device control mode for each function.

[0114] In an optional implementation, the specification document can include the control mode of the function that can be implemented by the target device, such as: the operation flow (steps of starting, running, and closing the device) of a certain function, how to operate the device, matters needing attention, etc. The control mode of a certain function can be obtained in the second description data by using the retrieval method.

[0115] In a specific example, the description data of the device control mode for implementing the function can record the control mode that can achieve the function such as the adjustment of a certain thermostat or the opening / closing of a control switch by changing some parameters / variables. The record is as follows:

[0116] Control power state: change the power on or off of the device by setting the value of the switch variable switch = ON / OFF;

[0117] Air conditioner temperature adjustment: change the air conditioner temperature by setting the value of the temperature variable temperature = [temperature value].

[0118] By searching the keywords "control power supply" and "air conditioner temperature control" in the function name "control power supply" and "air conditioner temperature control", the above text record in the second description data is obtained. By using a text processing tool (such as a large language model (LLM: Large Language Model is a natural language processing technology based on deep learning, which learns the rules and patterns of language by training a large amount of text data, and can generate coherent and meaningful text for language understanding and generation tasks)) to read and process, the corresponding instructions for the specific function are generated: power on - switch = ON, power off - switch = OFF, air conditioner temperature setting: temperature = [temperature parameter].

[0119] If the device control method recorded in the second description data is an information set composed of information describing device control instructions, the information set composed of information describing device control instructions for the target device can also be generated in the above manner.

[0120] In an optional embodiment, the "function" text in the first description data for a function determined based on the above method is stored in the "control intent data" part of the mapping relationship between the data expressing different control intents for the target device and the corresponding device control method; the control method determined according to this function is stored in the "control method" part of the mapping relationship between the data expressing different control intents for the target device and the corresponding device control method, thereby establishing the mapping relationship between the data expressing different control intents for the target device and the corresponding device control method for the function.

[0121] For each function of the target device, the above operation is performed in sequence to generate the mapping relationship between the data expressing different control intents for the target device and the corresponding device control method for all functions of the target device.

[0122] As described above, the form of the data of the target control intent can include various forms, such as voice input by a sound collecting device, text input collected by a client, etc. Any data expressing a target control intent for a target device, such as a target control intent, can be used as data of a target control intent for the function. The mapping relationship between the data expressing a target control intent for a target device and the corresponding device control method can be one or more forms of data expressing a target control intent for a target device and the corresponding device control method.

[0123] In an optional embodiment, the mapping relationship in step 12 can also be established by the following steps:

[0124] For each of the functions, the following operations are performed: determining the device control mode for implementing the function according to the second description data of the device control mode for implementing the function;

[0125] The mapping relationship between the control intention data for the function and the device control mode for implementing the function is established by taking the first description data of the function as the data of the control intention for the function.

[0126] In an optional embodiment, the control mode of the target device for a function can be obtained by using the retrieval function of the agent. Each function in the function list is retrieved as a keyword in the specification document by the retrieval agent, the corresponding control operation sentence of each function is found, and the corresponding control mode record of each function is generated by using a large language model to extract key information from the corresponding sentence.

[0127] In the present embodiment, the first description data of the function can be taken as the data of the control intention for the function, and the input data capable of embodying the meaning of the first description data can be taken as the data of the target control intention for the function.

[0128] For example, the function list of the target device is taken as the first description data, the names of each function such as lamp on, lamp off, brightness adjustment, color temperature adjustment, color change, timing task, scene mode are listed in the function list of the smart electric lamp, and the text recorded for each function in the function list is taken as the control intention of the smart electric lamp device for each function. The data capable of expressing the description intention of each function text can be taken as the data of the control intention of the smart electric lamp device for each function.

[0129] In the present embodiment, the record of the control mode corresponding to each function is stored in the "control mode" part used to express the mapping relationship between the data of different control intentions for the target device and the corresponding device control mode, and the text recorded in the first description data of the corresponding function is stored in the "control intention data" part used to express the mapping relationship between the data of different control intentions for the target device and the corresponding device control mode, thereby generating the mapping relationship between the control intention data for each function and the device control mode for implementing the function.

[0130] In an optional embodiment, as shown in Figure 4 The establishment of the above mapping relationship can also be implemented by the following sub-steps:

[0131] Step 31: generating the use suggestion data of the function based on the second description data of the device control mode for implementing the function.

[0132] The use suggestion data is use suggestion for each specific function of a target device for the purpose of facilitating user use, which can be recognized, retrieved, and queried by the smart home system, and can be information and data as background knowledge of the large language model. The use suggestion data can include use precautions of the target device, connection installation configuration guide, device basic operation method, troubleshooting operation method, maintenance method, multilingual support information, and other related data.

[0133] The device use suggestion data can be generated in the form of text, the form of question and answer pairs, multimedia, PDF documents, web pages, and various forms.

[0134] In an optional embodiment, the use suggestion data about a certain function of the target device is generated based on the specification document of the target device obtained in the previous step through sentence integration and information extraction of the large language model.

[0135] In a specific embodiment, each basic function of the target device is used as a retrieval keyword, and the use suggestion data is retrieved from the smart device specification document by the intelligent agent. The intelligent agent can find the required information and quickly locate the required content in a large amount of data. Specifically, different target devices can have different use methods, and common keywords and common formats can be set in advance. When generating the use suggestion data of the target device, the intelligent agent retrieves the specification document of the target device according to the common keywords of the use suggestion data, and obtains the use suggestion data about a certain function of the target device.

[0136] In an optional embodiment, the specification document of the target device can be used as the background knowledge of the large language model, and the basic use suggestion data generated by the intelligent agent is further integrated by the large language model to generate coherent readable text as the use suggestion data of the target device.

[0137] For example, the use suggestion data for the smart TV can include:

[0138] Several ways to establish a network TV network connection;

[0139] Which video input signals can be received by the network TV, and how to establish a connection for each video input;

[0140] Some basic operation introductions: how to turn on and off, adjust the volume, switch the input source, and use the remote control, etc.

[0141] The user interface layout of the smart TV, including the main menu, the setting menu, the application menu, etc.

[0142] The use suggestion data of the target device includes but is not limited to the above, and the use suggestion data can be extracted from the use document of the target device according to different characteristics of the target device, a document of the use suggestion data is generated, and the document is stored in a specified storage location of the smart home system.

[0143] Step 32: Based on the first description data of the function, the use suggestion data and the pre-set intention question collection prompt words, a control intention collection question for the function is generated.

[0144] The intention question collection prompt word refers to a prompt word used to construct a control intention collection question, which can prompt the user to express their needs.

[0145] The control intention collection question is a set of intention collection questions designed by the smart home system to enable the user to associate the possible use intention and control method of the target device.

[0146] In an optional embodiment, commonly used words that can express the user's intention are pre-set as intention question collection prompt words for generating control intention collection questions for the target device. For example, "hope", "want", "need", "need help", "hope to understand", "want to inquire" and the like. These question collection prompt words can be used in conjunction with the function list and the sentences in the use suggestion information to form control intention collection questions. The agent can combine the intention collection question prompt words with the "function" or control method in the function list or use suggestion information, and the large language model can be used to integrate and generate control intention collection questions.

[0147] In an optional embodiment, the inference function of the agent and the text processing function of the large language model can be used to generate a set of control intention collection questions for a self-defined target control intention such as "scene mode", "event trigger" and "schedule time trigger" in combination with the intention collection question prompt words, based on the first description data and the use suggestion data of the target device as background knowledge.

[0148] In this embodiment, the question types and question collection prompt words related to the trigger condition, device type, operation type, device name and location, time setting, device linkage, user preference, application scenario, etc. can be pre-designed, and a set of control intention collection questions related in semantics can be generated according to a self-defined control intention.

[0149] For example, a self-defined control intention related to the scene mode function can use a trigger condition as data expressing the target control intention for the target device, i.e. "the behavior of the smart door opening", and the related questions can be set as:

[0150] Do you want to set the smart door opening as a trigger condition?

[0151] What devices do you want to automatically turn on when the smart door opens?

[0152] What devices do you want to automatically turn off when the smart door opens?

[0153] In an optional implementation, the control intent collection question can also be generated using retrieval augmented generation (RAG) technology, which uses information from private or proprietary data sources to assist text generation, combines retrieval models (designed to search large data sets or knowledge bases) and generation models (such as large language models, which use retrieved information to generate readable text responses) to generate text for the control intent collection question, which can include a set of questions that ask about the user's subjective intent, usage preferences for devices, usage preferences for scenarios, preferences for control methods, personalized needs, and other characteristics.

[0154] Step 33: Obtain the corresponding answers given by the user as data for the control intent of the function by asking the user the control intent collection question.

[0155] In an optional implementation, one or more rounds of question and answer can be used to collect the answers given by the user and determine the data that can express the control intent of the function.

[0156] For example, the control intent collection question "What time do you want the air conditioner to automatically turn on?" is based on the timing of the opening function in the air conditioner function list, and the data for the user's control intent determined based on the user's answer is a time data "6 pm".

[0157] For example, according to the question "Do you want to set the smart door opening as a trigger condition?" in the aforementioned control intent collection question, the user is asked and responds affirmatively, obtaining the "control intent data" part of the mapping relationship between the "smart door opening trigger condition" as the control intent and the corresponding device control method for the "home scenario".

[0158] In this embodiment, the reasoning function of the agent can be invoked in combination with a large language model to analyze the user's answer and determine the data that can express the control intent of the function.

[0159] Step 34: Determine the device control method for implementing the function based on the second description data of the device control method for implementing the function.

[0160] In an optional implementation, for the custom control intent, the device control method for implementing the custom function is determined in the second description data.

[0161] In the embodiment, the reasoning function of the agent can be called by using a large language model, and the second description data of the target device related to the custom control intent is searched and identified to obtain the specific control mode related to the control operation to be implemented by the target device.

[0162] Step 35: Establish a mapping relationship between the data of the control intent of the function and the device control mode for implementing the function.

[0163] In an optional implementation, the target control intent data determined in steps 33 and 34 and the control mode for the corresponding target device are recorded in a mapping relationship between the data for expressing different control intents of the target device and the corresponding device control mode.

[0164] In an optional implementation, a set of control intent collection questions can be designed for scene-related intents, and the user is asked and multiple rounds of question and answer are performed. For example, for the sleep mode intent, the user can be asked: “Do you need to automatically start the sleep mode at a fixed time in the evening?” If the answer is yes, continue to ask: “When do you want to start the sleep mode? What temperature do you want the air conditioner to be set to during the sleep mode?” and other questions. The specific control operations identified in the user feedback information, such as “start the sleep mode at a fixed time”, “start the sleep mode at a certain time”, and “adjust the air conditioner temperature to the user's desired temperature”, are taken as the control mode corresponding to the “start the sleep mode intent”. The mapping relationship between the data for expressing different control intents of the target device and the corresponding device control mode is recorded in the knowledge base, as shown in the following table:

[0165]

[0166] In an optional implementation, based on the data, before determining the device control mode matching the data by querying the pre-defined mapping relationship, the object storage technology can be used to store the pre-defined mapping relationship in the knowledge base of the target device.

[0167] The object storage technology in the embodiment refers to encapsulating data as objects for storage, and each object includes data itself and related metadata information such as file name, size, and creation time. Each object usually includes data and related metadata.

[0168] In the embodiment, one target device can be taken as an object, and the mapping relationship between the data used to express different control intentions for the target device and the corresponding device control mode related to the target device can be stored in the database with object storage as the storage mode in the form of a table, a question pair, a webpage, a text, a database table, etc.

[0169] In an optional embodiment, the mapping relationship between the data used to express different control intentions for the target device and the corresponding device control mode related to the target device is stored in the MINIO database (MINIO is an open source, high performance, distributed object storage server) as the knowledge base of the target device.

[0170] In an optional embodiment, the mapping relationship between the data used to express different control intentions for the target device and the corresponding device control mode related to the target device can be stored in the knowledge base in a vectorized form. Specifically, the intention can be stored in a vectorized database as the knowledge base of the target device. The knowledge base contains pre-defined intentions and their corresponding vector representations, for example:

[0171] Intention: "watching a movie" -> vector: [v1, v2, v3,...], and the corresponding control mode is: dimming the light, turning on the projector, and entering the movie selection menu;

[0172] Intention: "reading" -> vector: [u1, u2, u3,...], and the corresponding control mode is: turning off the overhead light, turning on the desk lamp, and turning on the eye protection mode;

[0173] Intention: "sleeping" -> vector: [w1, w2, w3,...], and the corresponding control mode is: turning off the bedroom light and turning on the air conditioner sleep mode.

[0174] When the client of Home Assistant receives the user intention, it first converts the received user intention into a vectorized representation [x1, x2, x3,...], calculates the similarity between the user intention vector and each intention vector in the database using a similarity calculation method such as cosine similarity or Manhattan distance, finds the most similar intention vector, for example, in the above example, the vector similarity with the "sleeping" intention is the highest. Next, the client of Home Assistant implements the control operation "turning off the bedroom light and turning on the air conditioner sleep mode" on the bedroom light and air conditioner devices.

[0175] In an optional implementation, the device control method can further include: loading the pre-defined mapping relationship from the knowledge base of the target device in a hot loading manner; and determining the device control mode matched with the data by querying the loaded mapping relationship based on the data.

[0176] The hot loading manner refers to a manner of loading a new knowledge base of a target device to a system without restarting the system, or updating an existing knowledge base of a target device.

[0177] In an optional implementation, the hot loading can be implemented in an event listening manner. A listening service is set in a Home Assistant client to listen to knowledge base events of a target device. When a new knowledge base event is generated after an existing knowledge base of a target device has an update event or a new target device is online, the listening service listens to the event, and triggers a program or an event of loading a new knowledge base file to implement the hot loading of the knowledge base of the target device.

[0178] In an optional implementation, considering that the Home Assistant client is developed based on the Home Assistant platform, the function of reloading configuration in the Home Assistant platform developer tool can be used to implement the hot loading of the knowledge base file of the target device.

[0179] In this embodiment, the Home Assistant client pre-stores the pre-defined mapping relationship of the target device in the knowledge base. When the data of the target control intent of the target device is obtained, the control intent in the pre-stored mapping relationship is queried in the intent in the mapping relationship of the knowledge base. When the query hits, the control mode corresponding to the intent in the mapping relationship is obtained, which is the determined device control mode matched with the data of the target control intent of the target device.

[0180] In order to make the user intention and the control intent more accurate, in an optional implementation, the method can further include the following steps: performing vector fitting calculation based on the data and the obtained mapping relationship to obtain a vector fitting calculation result; and determining the device control mode matched with the data according to the vector fitting calculation result.

[0181] In the embodiment, the mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode can be vectorized, which can be realized by natural language processing technology, and the mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode can be converted into structured data including intention description, target device name, other information related to control (such as execution time, control parameter, trigger condition, etc.), control instruction and the like, and the structured data is further converted into fixed-dimension vectorized data. When the smart home system receives the use intention of a user for a certain target device or multiple target devices, the user intention is converted into vectorized data, and vector fitting calculation is performed on the vectorized mapping relationship data between the data respectively used to express different control intentions for the target device and the corresponding device control mode to determine the matching degree of the user intention vector and the mapping relationship vector between the data respectively used to express different control intentions for the target device and the corresponding device control mode in the knowledge base, find the most matched mapping relationship vector between the data respectively used to express different control intentions for the target device and the corresponding device control mode, and then determine the specific control mode of the target device.

[0182] In an optional embodiment, the description document of the target device, the control intention collection question and the custom intention data can be stored in a text file. Three folders can be stored in the root directory of the target device, which are respectively identified as the target device description document, the control intention collection question and the custom intention data. The target device description document includes the smart device description document provided by a third party or uploaded by a user; the control intention collection question includes the control intention collection question text data; and the custom intention data includes the answer text collected by asking and answering with the user, and the text document of the mapping relationship between the data respectively used to express different control intentions for the target device and the corresponding device control mode established according to the user answer. In each directory of the three directories, the subdirectories of each directory are named as the integrated device name, such as Xiaomi air purifier and Sony television. In the subdirectories, the txt files of each corresponding document are stored.

[0183] Specifically, the description document of a smart television can be stored in the following manner:

[0184] tv.txt

[0185] <<EOF

[0186] The smart television has the following characteristics:

[0187] (1) It has strong hardware devices, including a high-speed processor and a certain storage space, for running and storing application programs;

[0188] (2) Equipped with a smart operating system, users can install, run and uninstall software, games and other applications themselves;

[0189] (3) It can connect to the public internet;

[0190] (4) It has multiple interactive applications, such as new human-computer interaction methods, multi-screen interaction, content sharing, etc.

[0191] Key features of smart TVs:

[0192] 1) Possesses broadband internet access capability;

[0193] 2) It can receive and play back various data streams such as images and music obtained from the Internet;

[0194] 3) It can download and run various specially developed applications, including games;

[0195] 4) It has network communication capabilities, allowing users to power on / off and adjust volume via the network;

[0196] 5) It has a brand-new remote control device and can connect and interact with various mobile terminals.

[0197] Related functions:

[0198] Online streaming: Includes the latest and hottest movies, TV series, variety shows, anime, and more. It also features a unique follow-up function, allowing you to bookmark your favorite popular shows and never miss any updates.

[0199] Local playback: It has local video playback capabilities, which is unmatched by similar products. Simply connect a USB flash drive or external hard drive via OTG cable to play almost all high-definition video formats on the market, such as MKV, TS, FLV, AVI, VOB, MOV, WMV, MP4, RMVB, etc. It also perfectly supports Blu-ray high-definition and 3D videos.

[0200] Apps and Games: It comes with a wealth of built-in games and apps, such as Douban Music, Plants vs. Zombies, Bejeweled, and Duomi Music. The API interface will be opened to developers, and more games and apps will be added with system updates, providing a richer entertainment experience than just videos and audio.

[0201] Audiovisual experience: The first product in its class to have both Dolby and DTS licenses, completely eliminating the periodic noise caused by silence or software decoding when playing high-definition videos using audio codecs such as AC3 and DTS. All of this is to provide you with the best viewing and audiovisual experience.

[0202] < <EOF

[0203] Step 13: Control the target device according to the device control mode.

[0204] In an optional embodiment, as previously described regarding the control mode of the target device has been determined, in this step, the control operation is implemented on the target device according to the determined control mode.

[0205] In a specific example, as previously described, the control mode of turning on the light has been determined, and the recorded instructions for the mapping relationship between the data used to express different control intentions for the target device and the corresponding device control mode are as follows:

[0206] service: light.turn_on

[0207] entity_id: light.living_room_light.

[0208] A control mode execution program can be pre-designed in the client of Home Assistant, and when it receives a determination that the control mode for the target device needs to be executed, the automatic execution program automatically loads and executes the above determined instructions to achieve the control operation on the target device.

[0209] If the device control mode is a set of information composed of information describing device control instructions, the corresponding relationship between the information describing different device control instructions and the corresponding device control instructions can be looked up in the pre-set client of Home Assistant, and after determining the corresponding device control instructions, the above determined instructions are loaded and executed by the aforementioned control mode execution program to achieve the control operation on the target device.

[0210] In an optional embodiment, for the online target device, the process of controlling the target device can be implemented as shown in the flowchart: Figure 5

[0211] Step S0: The device integration of the subsumption service in the smart home system is notified to the large language model in the smart home system.

[0212] Step S1: The large language model of the smart home system reads the subsumption service through the device subsumption Agent, and after conversion, the function list of the target device is obtained.

[0213] ​Step S2: (1) The smart home system obtains the specification document of the smart device provided by a third party (search engine, user-uploaded document to the knowledge base, etc.) through a device information acquisition agent, generates a txt file, and stores it in the smart home knowledge base; (2) The specification document of the smart device is retrieved through a retrieval agent to obtain relevant key information of the use information, and then a large language model is used to integrate the sentences to obtain use suggestion data; wherein the device information acquisition agent is used to acquire the specification document and generate a text document of the target device specification document;

[0214] Step S3: The reasoning agent combines the large language model to take the function list and the smart device use suggestion data as background knowledge, combines the intent question collection prompt word, and generates a device operation intent question set (control intent collection question) by the reasoning agent, wherein the large language model integrates background knowledge from different sources (such as target device functions, use suggestion data, operation steps, etc.) to provide a basis for reasoning, and the reasoning agent determines which information needs to be provided by the user based on the information provided by the large language model, and further generates the control intent collection question;

[0215] Step S4: The smart home system asks the user one by one in the intent collection question set about the use function scene, related intent, etc. After one or more rounds of question and answer, the user feedback reply information is collected; the reply information text is converted according to the collected reply information, the reply information text is processed by the large language model, and then the reasoning agent generates a record of the mapping relationship between the data respectively used to express different control intents of the target device and the corresponding device control mode according to the content of the function list; which can be converted into a vector form and stored in the smart home knowledge base;

[0216] Step S5: The large language model of the smart home system takes the existing mapping relationship description between the data respectively used to express different control intents of the target device and the corresponding device control mode as background knowledge, calls the reasoning agent to use the pre-set intent prompt word for reasoning, further enriches the mapping relationship record between the data respectively used to express different control intents of the target device and the corresponding device control mode, generates custom intent data, and can convert the custom intent data into a vector form and store it in the smart home knowledge base.

[0217] Step S6: When the user subsequently issues an intent instruction, the smart home system queries the intent in the knowledge base, the smart home system performs fitting calculation on the user intent and the vectorized intent data stored in the knowledge base, finds the corresponding control mode according to the mapping relationship, gives the control mode of the target device, and then controls the related operation of the target device by the device control agent.

[0218] In an alternative embodiment, the control of the target device can also be implemented by the flow shown in the flowchart as follows: Figure 6

[0219] The user collects questions according to the control intention proposed by the reasoning agent, and provides the smart home system with information such as scene and intention.

[0220] The device management agent reads the management service and obtains the function list of the target device after conversion.

[0221] The data acquisition agent acquires the instruction document of the target device, generates a txt text file of the instruction document and stores it in the target device knowledge base, and generates use suggestion information of the target device through the arrangement of the large language model.

[0222] The reasoning agent obtains the use suggestion information of the target device, integrates the sentences provided by the large language model, combines the intention question collection prompt words set in advance or arranged by the large language model, and determines the information that may need to be provided by the user by using logical reasoning, to generate the control intention collection question of the target device. Through asking the user the above control intention collection question, the information related to the user's use scene and intention is obtained, the user intention data is collected, and the mapping relationship between the target device control mode corresponding to various different intentions proposed by the user is obtained after the large language model processing, that is, the mapping relationship between "user intention-target device control mode"; The reasoning agent uses the intention prompt word to perform reasoning with the existing mapping relationship description between the data respectively used to express different control intentions of the target device and the corresponding device control mode as background knowledge, and generates self-defined intention data by adjusting and enriching the above mapping relationship through the large language model, and stores it in the target device knowledge base.

[0223] The instruction document of the target device, the control intention collection question and the self-defined intention data are stored in the knowledge base of the target device.

[0224] Based on the same inventive concept as the above embodiments, in order to solve the problem that the device control mode adopted by the prior art consumes a long time to determine the device control intention and has low efficiency, and to ensure good user experience, the embodiments of the present application also provide a mapping relationship establishing method, comprising:

[0225] Step: obtaining first description data of the function of the target device.

[0226] The first description data of the function of the target device is obtained in the manner described in the above embodiments.

[0227] Step: obtaining second description data of the device control mode for realizing the function of the target device.

[0228] ​The second description data of the device control mode for realizing the function of the target device is acquired in the manner described in the foregoing embodiments.

[0229] The mapping relationship between the data for expressing different control intentions for the target device and the corresponding device control mode is established in the manner described in the foregoing embodiments by using the preset mapping relationship establishment strategy.

[0230] The mapping relationship establishment strategy comprises: for each target device function, respectively performing: determining the data for expressing the control intention for the function based on the first description data of the function, and determining the device control mode for realizing the function based on the second description data of the device control mode for realizing the function.

[0231] The mapping relationship between the data for expressing different control intentions for the target device and the corresponding device control mode is established in the manner described in the foregoing embodiments by using the preset mapping relationship establishment strategy.

[0232] For the same inventive concept as the foregoing embodiments, in order to solve the problem that the device control mode used in the prior art takes a long time to determine the device control intention and is low in efficiency, and to ensure good user experience, the embodiments of the present application further provide a device control apparatus.

[0233] The specific structural diagram of the apparatus is shown in Figure 7 The apparatus comprises the following functional units:

[0234] The control intention data acquisition unit 71 is configured to acquire data for expressing a target control intention for a target device.

[0235] The control mode determination unit 72 is configured to determine a device control mode matched with the data by querying a pre-defined mapping relationship based on the data; the mapping relationship comprises a mapping relationship between data for expressing different control intentions for the target device and corresponding device control modes.

[0236] The device control unit 73 is configured to control the target device according to the device control mode.

[0237] In an optional embodiment, the control mode determination unit 72 can further comprise a mapping relationship establishment subunit, comprising the following functional subunits:

[0238] The first description data acquisition subunit is configured to acquire first description data of a function of the target device.

[0239] The second description data acquisition subunit is configured to acquire second description data of a device control mode for realizing the function.

[0240] The mapping relationship establishing subunit is configured to establish the mapping relationship by using a preset mapping relationship establishing strategy.

[0241] The mapping relationship establishing strategy comprises: for each function, respectively performing: determining control intention data for the function based on first description data of the function, and determining a device control mode for implementing the function based on second description data of the device control mode for implementing the function.

[0242] In an optional implementation, the mapping relationship establishing subunit can further comprise the following functional subunits:

[0243] For each function, the following operations are performed by the following subunits respectively:

[0244] The device control mode determining subunit is configured to determine the device control mode for implementing the function according to the second description data of the device control mode for implementing the function.

[0245] The mapping relationship determining subunit is configured to establish a mapping relationship between the control intention data for the function and the device control mode for implementing the function by taking the first description data of the function as the control intention data for the function.

[0246] In an optional implementation, the mapping relationship establishing subunit can further comprise the following functional subunits:

[0247] For each function, the following operations are performed by the following subunits respectively:

[0248] The use suggestion data generating subunit is configured to generate use suggestion data of the function based on the second description data of the device control mode for implementing the function.

[0249] The control intention collection question generating subunit is configured to generate a control intention collection question for the function based on the first description data of the function, the use suggestion data, and a pre-set intention question collection prompt.

[0250] The control intention data obtaining subunit is configured to obtain a corresponding answer given by a user as the control intention data for the function by asking the user the control intention collection question.

[0251] The device control mode determining subunit is configured to determine the device control mode for implementing the function according to the second description data of the device control mode for implementing the function.

[0252] The mapping relationship determining subunit is configured to establish a mapping relationship between the control intention data for the function and the device control mode for implementing the function.

[0253] In an alternative embodiment, the device control apparatus provided by the embodiments of the present application can further comprise:

[0254] a knowledge base storage unit configured to store the pre-defined mapping relationship to the knowledge base of the target device by using object storage technology before determining the device control mode matching the data based on the data by querying the pre-defined mapping relationship.

[0255] In an alternative embodiment, the control mode determination unit can further comprise:

[0256] a hot loading sub-unit configured to load the pre-defined mapping relationship from the knowledge base of the target device by using a hot loading mode;

[0257] a control mode determination sub-unit configured to determine the device control mode matching the data by querying the loaded mapping relationship based on the data.

[0258] In an alternative embodiment, the control mode determination unit can further comprise:

[0259] a vector fitting calculation sub-unit configured to perform vector fitting calculation based on the data and the mapping relationship obtained by querying to obtain a vector fitting calculation result;

[0260] a control mode determination sub-unit configured to determine the device control mode matching the data according to the vector fitting calculation result.

[0261] For the same inventive concept as the foregoing embodiments, in order to solve the problem of the device control mode in the prior art that takes a long time to determine the device control intent and is low in efficiency, and to ensure good user experience, the embodiments of the present application also provide a mapping relationship establishment apparatus, as shown in Figure 8 The mapping relationship establishment apparatus comprises the following functional sub-units:

[0262] a first description data acquisition unit 81 configured to acquire first description data of a function of the target device;

[0263] a second description data acquisition unit 82 configured to acquire second description data of a device control mode for implementing the function;

[0264] a mapping relationship establishment unit 83 configured to establish a mapping relationship between data respectively expressing different control intents for the target device and corresponding device control modes by using a pre-set mapping relationship establishment strategy;

[0265] The mapping relationship establishing strategy comprises: for each function, respectively performing: determining, based on first description data of the function, data of a control intention for the function; and determining, based on second description data of a device control mode for implementing the function, the device control mode for implementing the function.

[0266] With the above device provided in the embodiments of the present application, in the case where the data for expressing the target control intention for the target device is acquired, the control mode of the target device can be determined by querying the mapping relationship between the "data respectively for expressing the target control intention for the target device" and the "device control mode" predefined in advance according to the data. Compared with the prior art of "understanding the device control intention based on a large model, and then controlling the device", since the querying of the mapping relationship is relatively simple and easy to implement, the determination process of the device control mode is more efficient and has a shorter response time, and the good experience of the user can be ensured.

[0267] Based on the same inventive concept as the above embodiments, the embodiments of the present application further provide an electronic device to solve the problem of the prior art that the large model retrieval is performed after receiving the instruction issued by the user, and the time consumption is long and the efficiency is low.

[0268] As shown in Figure 9 The computing device includes a memory 91 and a processor 92. The memory 91 can be configured to store various data to support operations on the electronic device. Examples of the data include instructions for any application or method operating on the electronic device. The memory 91 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0269] The processor 92 is coupled to the memory 91 and is configured to execute the computer program stored in the memory 91 for performing the device control method or the mapping relationship establishing method described in the embodiments of the present application.

[0270] When the processor 92 executes the computer program in the memory 91, in addition to the above functions, it can also implement other functions. For details, refer to the descriptions of the above embodiments.

[0271] Further, as shown in Figure 9 The computing device further includes a display 94, a communication component 93, a power supply component 95, an audio component 96 and other components. Figure 9 Only some components are shown in the figure schematically, and it does not mean that the computing device only includesFigure 9 the components shown.

[0272] Accordingly, the embodiments of the present application also provide a computer readable storage medium storing a computer program, and the computer program is capable of implementing the device control method or the mapping relationship establishing method provided by the above-mentioned embodiments when being executed by a computer.

[0273] Accordingly, the embodiments of the present application also provide a computer program product, comprising a computer program, and the computer program is capable of implementing the device control method or the mapping relationship establishing method provided by the above-mentioned embodiments when being executed by a processor.

[0274] The device embodiments described above are only schematic, and the units shown as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. Those skilled in the art can understand and implement without creative labor.

[0275] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0276] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A device control method characterized by, The method comprises: obtaining data for expressing a target control intention for a target device; based on the data, determining a device control mode matching the data by querying a pre-defined mapping relationship; the mapping relationship comprises a mapping relationship between data respectively expressing different control intentions for the target device and corresponding device control modes; controlling the target device according to the device control mode; the mapping relationship is established in the following manner: obtaining first description data of a function of the target device; obtaining second description data of a device control mode for implementing the function; using a pre-set mapping relationship establishment strategy to establish the mapping relationship; using a pre-set mapping relationship establishment strategy to establish the mapping relationship, comprising: for each function, the following operations are respectively performed: based on the second description data of the device control mode for implementing the function, generating use suggestion data of the function; based on the first description data of the function, the use suggestion data and a pre-set intention question collection prompt word, generating a control intention collection question for the function; by asking the user the control intention collection question, obtaining a corresponding answer given by the user as data of the control intention for the function; determining a device control mode for implementing the function according to the second description data of the device control mode for implementing the function; establishing a mapping relationship between the data of the control intention for the function and the device control mode for implementing the function.

2. The method of claim 1, wherein, The method further comprises: before determining the device control mode matching the data by querying the pre-defined mapping relationship based on the data, storing the pre-defined mapping relationship to a knowledge base of the target device using object storage technology.

3. The method of claim 1, wherein, The method further comprises: loading the pre-defined mapping relationship from the knowledge base of the target device in a hot loading manner; based on the data, determining the device control mode matching the data by querying the loaded mapping relationship.

4. The method of claim 1 or 2, wherein, The method further comprises: based on the data and the mapping relationship obtained by querying, performing vector fitting calculation to obtain a vector fitting calculation result; based on the vector fitting calculation result, determining the device control mode matching the data.

5. An apparatus control device characterized by comprising: The device comprises: a control intention data obtaining unit, configured to obtain data for expressing a target control intention for a target device; a control mode determining unit, configured to determine a device control mode matching the data by querying a pre-defined mapping relationship based on the data; the mapping relationship comprises a mapping relationship between data respectively expressing different control intentions for the target device and corresponding device control modes; a device control unit, configured to control the target device according to the device control mode; the mapping relationship is established in the following manner: obtain first description data of a function of the target device; obtain second description data of a device control mode for implementing the function; establish a mapping relationship using a preset mapping relationship establishment strategy; establish the mapping relationship using the preset mapping relationship establishment strategy, comprising: for each of the functions, the following operations are performed respectively: generate use suggestion data of the function based on the second description data of the device control mode for implementing the function; generate a control intent collection question for the function based on the first description data of the function, the use suggestion data, and a pre-set intent question collection prompt word; obtain a corresponding answer given by a user as data of a control intent for the function by asking the user the control intent collection question; determine a device control mode for implementing the function based on the second description data of the device control mode for implementing the function; establish a mapping relationship between the data of the control intent for the function and the device control mode for implementing the function.

6. An electronic device, comprising: comprise: a memory and a processor, wherein the memory is configured to store a computer program; the processor is coupled to the memory and configured to execute the computer program stored in the memory, so as to execute the device control method of any one of claims 1-4.

Images