Smart home control method and device based on large language model, and electronic equipment

Abstract

The invention provides a smart home control method and device based on a large language model and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, large language models, natural language processing, smart home control and the like. According to the specific implementation scheme, after a home operation instruction is obtained, a reference home operation instruction matched with the home operation instruction and a corresponding reference equipment control element are determined, and a target equipment control element of the home operation instruction is determined by using a first large language model according to the reference home operation instruction and the reference equipment control element; and performing smart home control according to the target equipment control element. Therefore, through the first large language model, the target equipment control element of the home operation instruction can be accurately determined, so that intelligent home control is accurately realized based on the determined target equipment control element.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as deep learning, large language models, natural language processing, and smart home control, and in particular to a smart home control method, device, and electronic device based on a large language model. Background Art

[0002] A smart home is a living environment that connects various devices and systems in the home through the Internet and Internet of Things technologies, enabling intelligent control and management. With the development of information technology and the growth of consumer demand, smart homes have become an important part of modern family life.

[0003] In a smart home environment, how to accurately obtain the device control elements corresponding to home operation instructions is very important for achieving precise smart home control. Summary of the Invention

[0004] The present disclosure provides a smart home control method, device and electronic device based on a large language model.

[0005] According to one aspect of the present disclosure, a smart home control method based on a large language model is provided, the method comprising: obtaining a home operation instruction; determining a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction; using a first large language model, determining a target device control element of the home operation instruction based on the reference home operation instruction and the reference device control element; and performing smart home control according to the target device control element.

[0006] According to another aspect of the present disclosure, a smart home control device based on a large language model is provided, the device comprising: an acquisition module for acquiring a home operation instruction; a first determination module for determining a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction; a second determination module for determining a target device control element of the home operation instruction based on the reference home operation instruction and the reference device control element using the first large language model; and a home control module for performing smart home control according to the target device control element.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the smart home control method based on the large language model proposed above in the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the smart home control method based on the large language model proposed above in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the smart home control method based on a large language model proposed above in the present disclosure.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 6 It is a schematic block diagram of an electronic device used to implement the smart home control method based on a large language model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] Figure 1This is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the smart home control method based on a large language model in the embodiment of the present disclosure can be applied to a smart home control device based on a large language model. The device can be an electronic device, or can be configured in an electronic device so that the electronic device can perform smart home functions. It should be noted that the electronic device can be a home control center device that can control various smart devices in a smart home. It should be noted that the following embodiments of the present disclosure are described by taking the execution subject as an electronic device as an example.

[0020] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a smart speaker, a server, a server cluster, and other hardware devices with various operating systems, touch screens and / or display screens.

[0021] like Figure 1 As shown, the smart home control method based on the large language model may include the following steps:

[0022] Step 101: Obtain home operation instructions.

[0023] It should be noted that the home operation instruction can be input in the form of voice and / or text.

[0024] Step 102: Determine a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction.

[0025] In one embodiment of the present application, the matching degree between the home operation instruction and each home operation instruction in a preset home operation instruction library is determined, and a home operation instruction with a matching degree greater than or equal to a preset matching degree threshold is selected from the home operation instruction library as a reference home operation instruction.

[0026] It should be noted that the above-mentioned matching degree threshold is a matching degree critical value preset according to actual needs.

[0027] In another embodiment of the present application, the degree of matching between the home operation instruction and each home operation instruction in a preset home operation instruction library is determined, and the home operation instruction with the highest matching degree is obtained from the home operation instruction library as a reference home operation.

[0028] In this embodiment, based on the correspondence between the preset home operation instruction and its corresponding device control element, the reference device control element corresponding to the reference home operation instruction can be obtained.

[0029] Step 103 , using the first language model, according to the reference home operation instruction and the reference device control element, determines the target device control element of the home operation instruction.

[0030] It should be noted that the target device control element in this embodiment refers to the element required to control the corresponding smart home device in the smart home.

[0031] The target device control element in this embodiment may include at least one of a target device identifier, a target device function, and an adjustment target of the target device function.

[0032] In one embodiment of the present application, a possible implementation method of using the first largest language model to determine the target device control element of the home operation instruction based on the reference home operation instruction and the reference device control element is: adding the reference home operation instruction, the reference device control element and the home operation instruction to the corresponding positions in the prompt word template to obtain the prompt word; inputting the prompt word into the first largest language model to obtain the target device control element.

[0033] The prompt word is used to indicate that a corresponding device control element is generated for the home operation instruction by taking the reference home operation instruction and the reference device control element as an example, wherein the generated device control element is included in the reference control element.

[0034] Among them, it should be noted that when the prompt word is input into the first language model, the first language model can use the reference home operation instructions and reference device control elements in the prompt word as examples, and generate corresponding device control elements for the home operation instructions through its own capabilities.

[0035] In this embodiment, prompt words are used to guide the first language model to more accurately understand the task requirements, thereby accurately generating corresponding device control elements for the home operation instructions based on reference home operation instructions and reference device control elements.

[0036] As a possible implementation, the prompt word template in this example may include the first position to be filled and the corresponding second position to be filled in the reference example pair, as well as the third position to be filled and the corresponding fourth position to be filled in the reference example pair to be generated, and a rule for generating the content of the third position to be filled. The rule may be: based on the reference example pair and the content filled in the third position to be filled, the content of the fourth position to be filled is generated, and the content of the generated fourth position to be filled is included in the target content, wherein the target content is the content filled in the second position to be filled. Correspondingly, after obtaining the reference home operation instruction and the corresponding reference device control element, the reference home operation instruction can be filled in the first position to be filled, and the reference device control element can be filled in the second position to be filled, and the home operation instruction can be filled in the third position to be filled to obtain the prompt word.

[0037] Step 104: Perform smart home control according to the target device control elements.

[0038] In one embodiment of the present application, when the target device control element only includes the target device identifier, in order to achieve control of the corresponding smart home device, the target device function required to satisfy the home operation instruction can also be determined based on the home operation instruction; based on the home operation instruction, the adjustment target of the target device function is determined, and the smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to adjust according to the adjustment target. In this way, the corresponding smart home device can be accurately controlled, and the accuracy of smart home control is improved.

[0039] In one embodiment of the present application, in order to accurately determine the required target device function based on the home operation instruction, a possible implementation method for determining the target device function required to meet the home operation instruction according to the home operation instruction is: extracting the home operation action from the home operation instruction, and determining the first representation information of the home operation action, and obtaining a first reference representation from a preset representation library that matches the first representation information and has a device identifier as a target device identifier, and using the home operation action corresponding to the first reference representation as a reference home operation action, and using the device function corresponding to the home operation action as a reference device function, and using the first large language model to determine the device function of the home operation action according to the reference home operation action and the corresponding reference device function, and using the device function of the home operation action as the target device function of the home operation instruction.

[0040] The reference home operation action refers to the home operation action extracted from the reference home operation instruction.

[0041] In an exemplary embodiment, a device control instruction for a target device identifier can be generated based on the target device function and the adjustment target of the target device function, and the device control instruction can be sent to a smart home device corresponding to the target device identifier, so that the smart home device can instruct the target device function of the smart home device to be adjusted according to the adjustment target based on the device control instruction.

[0042] For example, the home operation instruction is: set the temperature of the air conditioner in the living room to 26 degrees. Assume that the device control element is: the target device identifier corresponding to the air conditioner in the living room, the target device function is the temperature adjustment function, and the adjustment target corresponding to the target device function is 26 degrees. Correspondingly, a device control instruction can be generated based on the above-mentioned target device control element, and the device control instruction is sent to the air conditioner in the living room, so that the temperature of the temperature adjustment function of the air conditioner is adjusted to 26 degrees.

[0043] In one embodiment of the present application, in order to accurately control the corresponding smart home device, when the target device control element includes the target device function, the target device identifier of the smart home device required to meet the home operation instruction can also be determined based on the home operation instruction, and the adjustment target of the target device function can be determined based on the home operation instruction, as well as the smart home device corresponding to the target device identifier, and the target device function of the smart home device can be controlled to adjust according to the adjustment target. Thus, in combination with the home operation instruction, the target device identifier and the adjustment target of the target device function are determined, and based on the target device function and the corresponding adjustment target, the smart home device with the target device identifier is accurately controlled, thereby improving the accuracy of smart home control.

[0044] It is understandable that in different application scenarios, according to the home operation instruction, the implementation method of determining the target device identifier of the smart home device required to meet the home operation instruction is different, and an exemplary description is as follows:

[0045] As a possible implementation method, the home operation instruction can be directly input into the third language model for analysis by the third language model to obtain the target device identifier of the smart home device required to meet the home operation instruction.

[0046] As another possible implementation method, the representation information of the home operation instruction can be determined, and the target reference representation that matches the representation information can be obtained from a preset representation library. The target home operation instruction corresponding to the target reference representation is used as the reference home operation instruction, and the reference device identifier of the reference home operation instruction is obtained. The first language model is used to determine the target device identifier of the home operation instruction based on the reference home operation instruction and the reference device identifier.

[0047] It should be noted that the first language model, the second language model, and the third language model in this example may be different large language models, or the same large language model, and this embodiment does not specifically limit this.

[0048] In one embodiment of the present application, when the target device control elements include: a target device identifier and a target device function, an adjustment target for the target device function can also be determined based on the home operation instruction. Correspondingly, the smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target. Thus, based on the first language model, the target device identifier and target device function of the home operation instruction are accurately determined, and the adjustment target of the target device function is determined from the home operation instruction. Thus, based on the target device identifier, target device function, and adjustment target, home control is accurately achieved for the smart home device corresponding to the target device identifier.

[0049] In one embodiment of the present application, when the above-mentioned target home control element only includes the target device identifier, the possible implementation method of smart home control based on the target device control element may be: returning the current status of the device function of the smart home device corresponding to the target device identifier; or, obtaining the default device control instruction corresponding to the target device identifier, and instructing the target device function corresponding to the target device identifier to adjust according to the preset adjustment target according to the default device control instruction; or, returning a corresponding error prompt message, wherein the error prompt information is used to prompt that the home operation instruction is incomplete.

[0050] In one embodiment of the present application, when the above-mentioned target home control element only includes the target device function, the possible implementation method of smart home control based on the target device control element may be: a prompt message may be returned to prompt the limitation of the device corresponding to the target device function, or a preset error prompt message may be returned to prompt that the home operation instruction is incomplete; or, in combination with other environmental information, the target device identifier corresponding to the home operation instruction and the adjustment target of the target device function are determined, and the corresponding smart home device is controlled according to the target device identifier, adjustment target and target device function.

[0051] Among them, a possible implementation method of controlling the corresponding smart home device according to the target device identification, adjustment target and target device function is: generating a device control instruction for the target device identification according to the adjustment target and the target device function, and sending the device control instruction to the smart home device of the target device identification, so that the target device function of the smart home device is adjusted according to the adjustment target.

[0052] The large language model-based smart home control method of the disclosed embodiment, after obtaining a home operation instruction, determines a reference home operation instruction that matches the home operation instruction, as well as a reference device control element corresponding to the reference home operation instruction. Using a first large language model, the method determines a target device control element for the home operation instruction based on the reference home operation instruction and the reference device control element, and performs smart home control based on the target device control element. Thus, by using the first large language model, combined with the reference home operation instruction that matches the home operation instruction and the corresponding reference device control element, the device control element analysis for the home operation instruction is performed, accurately determining the target device control element for the home operation instruction, thereby accurately achieving smart home control based on the determined target device control element.

[0053] In order to clearly explain the process of determining the reference home operation instruction that matches the home operation instruction and the reference device control element corresponding to the reference home operation instruction, the following is combined with Figure 2 A possible implementation method for determining a reference home operation instruction that matches a home operation instruction and a reference device control element corresponding to the reference home operation instruction is exemplarily described.

[0054] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure.

[0055] like Figure 2 As shown, the method may include the following steps:

[0056] Step 201: Obtain home operation instructions.

[0057] It should be noted that, for the specific description of step 201, reference may be made to the relevant descriptions in other embodiments, which will not be repeated here.

[0058] Step 202: Determine the representation information of the home operation instruction.

[0059] In one embodiment of the present application, one possible implementation method for determining representation information for a home operation instruction is to extract key home control content from the home operation instruction and perform representation processing on the key home control content to obtain representation information. In other words, in this embodiment, content irrelevant to home control can be deleted from the home operation instruction. This reduces the amount of data required for representation processing, prevents the impact of irrelevant content on subsequent processing, and improves the accuracy of subsequent home control.

[0060] Among them, the key content of home control refers to the key content related to home control.

[0061] In one embodiment of the present application, one possible implementation method for extracting key home control content from home operation instructions is to input the home operation instructions into a second language model to obtain the key home control content. Thus, the second language model is used to quickly and accurately determine the key home control content in the home operation instructions.

[0062] Step 203: Acquire a target reference representation that matches the representation information from a preset representation library.

[0063] In one embodiment of the present application, one possible implementation method for obtaining a target reference representation that matches the representation information from a preset representation library is to determine the degree of match between the representation information and each reference representation in the preset representation library; and then select, from the preset representation library, a reference representation with a degree of match greater than or equal to a preset matching threshold as the target reference representation. Thus, based on the degree of match, a target reference representation that matches the representation information is accurately obtained from the preset representation library.

[0064] The preset matching degree threshold is a matching degree preset according to actual needs, for example, the preset matching threshold is 0.8 or 0.9.

[0065] In one embodiment of the present application, one possible implementation method for obtaining a target reference representation that matches the representation information from a preset representation library is to determine the degree of match between the representation information and each reference representation in the preset representation library; and then select the reference representation with the highest degree of match from the preset representation library as the target reference representation. Thus, based on the degree of match, a target reference representation that matches the representation information is accurately obtained from the preset representation library.

[0066] In step 204 , the target home operation instruction corresponding to the target reference representation is used as a reference home operation instruction, and the device control element corresponding to the target home operation instruction is used as a reference device control element.

[0067] Step 205 , using the first language model, according to the reference home operation instruction and the reference device control element, determines the target device control element of the home operation instruction.

[0068] Step 206: Perform smart home control according to the target device control elements.

[0069] It should be noted that, for the specific description of steps 205-206, reference can be made to the relevant descriptions in other embodiments, which will not be repeated here.

[0070] In this embodiment, after obtaining the home operation instruction, combined with the representation information of the home operation instruction, the target reference representation that matches the home operation instruction is obtained from the preset representation library, and the home operation instruction corresponding to the target reference representation is used as the reference home operation instruction, and the reference device control element corresponding to the reference home operation instruction is obtained. Thus, the reference home operation instruction that matches the home operation instruction is determined by the matching degree between the representations, which can capture the deep meaning and contextual information of the home operation instruction, rather than just the surface character matching, so that the obtained reference home operation instruction and the corresponding reference device control element are more accurate, thereby improving the accuracy of the target device control element determined subsequently, and more accurately realizing smart home control.

[0071] In order to clearly understand the present disclosure, Figure 3 The method of this embodiment is described exemplarily.

[0072] Figure 3 It is a schematic diagram according to the third embodiment of the present disclosure. It should be noted that, in this embodiment, the reference device control element only includes the reference device identifier, and the target device control element only includes the target device identifier.

[0073] like Figure 3 As shown, the method may include:

[0074] Step 301: Obtain home operation instructions.

[0075] It should be noted that, for the specific description of step 301, reference can be made to the relevant descriptions in other embodiments, which will not be repeated here.

[0076] Step 302: Determine a reference home operation instruction that matches the home operation instruction, and a reference device identifier corresponding to the reference home operation instruction.

[0077] In one embodiment of the present application, a possible implementation method for determining a reference home operation instruction that matches a home operation instruction and a reference device identifier corresponding to the reference home operation instruction is as follows: determining representation information of the home operation instruction; obtaining a target reference representation that matches the representation information from a preset representation library; using the target home operation instruction corresponding to the target reference representation as the reference home operation instruction, and using the device identifier corresponding to the target home operation instruction as the reference device identifier. Thus, by determining the reference home operation instruction that matches the home operation instruction based on the matching degree between the representations, the deeper meaning and contextual information of the home operation instruction can be captured, rather than just superficial character matching, making the resulting reference home operation instruction more accurate.

[0078] Step 303: Using the first language model, according to the reference home operation instruction and the reference device identification, determine the target device identification of the home operation instruction.

[0079] In one embodiment of the present application, a first prompt word can be obtained based on a reference home operation instruction, a reference device identification, and a corresponding position of the home operation instruction input into the first prompt word template, and the first prompt word can be input into the first large language model to obtain the target device identification.

[0080] The first prompt word is used to indicate that a corresponding device identification is generated for the home operation instruction by taking the home operation instruction and its corresponding reference device identification as an example, wherein the generated device identification is included in the reference device identification.

[0081] Step 304: extract the home operation action from the home operation instruction.

[0082] The home operation action refers to the specific operation behavior expressed in the home operation instruction, including the operation action on the corresponding smart home device. For example, if the home operation instruction is: "Turn on the living room light", the corresponding home operation action is: "Turn on the light".

[0083] Step 305: Obtain a reference home operation action that matches the home operation action, and a reference device function corresponding to the reference home operation action.

[0084] In one embodiment of the present application, a possible implementation method for obtaining a second reference home operation instruction that matches a home operation action, and a reference device control element corresponding to the second reference home operation instruction is as follows: extracting the home operation action from the home operation instruction, determining first representation information of the home operation action, obtaining a first reference representation that matches the first representation information and whose device identifier is the target device identifier from a preset representation library, and using the home operation action corresponding to the first reference representation as a reference home operation action, and using the device function corresponding to the home operation action as a reference device function. Thus, by determining the reference home operation action that matches the home operation action through the degree of matching between the representations, the deep meaning and contextual information of the home operation action can be captured, rather than just the surface character matching, so that the obtained reference home operation action is more accurate.

[0085] The reference home operation action is obtained by extracting the home operation action from the reference home operation instruction.

[0086] Step 306 , using the first language model, according to the reference home operation action and the reference device function, determines the device function of the home operation action, and uses the device function of the home operation action as the target device function of the home operation instruction.

[0087] In one embodiment of the present application, a second prompt word can be obtained based on the reference home operation action, the reference device function and the home operation action input into the corresponding position in the second prompt word template, and the prompt word is input into the first large language model to obtain the device function of the home operation action, and the device function of the home operation action is used as the target device function of the home operation instruction.

[0088] Among them, the second prompt word is used to indicate that a corresponding device function is generated for the home operation action based on a reference home operation action and a corresponding reference device function, wherein the generated device function is included in the reference device function.

[0089] Step 307: Determine the adjustment target of the target device function according to the home operation instruction.

[0090] As an example, the home operation instruction can be input into the fourth language model to analyze the home operation instruction through the fourth language model to obtain the adjustment target of the target device function.

[0091] Step 308: Generate a device control instruction for the target device identifier according to the target device function and the adjustment target.

[0092] Step 309: Send the device control instruction to the smart home device corresponding to the target device identifier to instruct the target device function in the smart home device to be adjusted according to the adjustment target.

[0093] In this embodiment, based on the first language model, the target device identifier in the home operation instruction is accurately determined, and the target device function corresponding to the target device identifier in the home operation instruction is determined using the first language model, and the adjustment target of the target device function is determined according to the home operation instruction, and based on the determined target device identifier, target device function and adjustment target, the corresponding smart home device is accurately controlled, thereby improving the smart home control experience.

[0094] In order to clearly understand this application, Figure 4 This embodiment is described as an example.

[0095] Figure 4 1 is a schematic diagram according to the fourth embodiment of the present disclosure. It should be noted that in this embodiment, the representation information is represented in the form of a vector, and the execution subject is a home control center device as an example for exemplary description.

[0096] like Figure 4 As shown, the method may include:

[0097] Step 401: Acquire a home operation request, wherein the home operation request includes: home operation voice data.

[0098] It should be noted that, in this embodiment, the example in which the home operation instruction is input in the form of voice is used for illustrative description.

[0099] For the convenience of description, in this example, the home operation instructions input by voice are referred to as home operation voice data.

[0100] It should be noted that the home control center device in this embodiment can access various smart home devices in the same home control system and can control various smart home devices.

[0101] Step 402: Recognize the home operation voice data through a voice recognition service to obtain a text recognition result of the home operation voice data.

[0102] Step 403: Perform vector representation on the text recognition result through the word representation service to obtain a first representation vector.

[0103] Step 404: Obtain N first reference representation vectors that match the first representation vector from a preset vector library, and obtain the reference home operation instruction corresponding to each first reference representation vector and the reference device identifier of the reference home operation instruction.

[0104] It should be noted that the reference device identifier is the device identifier of a smart home device included in the home control system. In other words, the smart home device corresponding to the reference device identifier belongs to the home control system.

[0105] Here, N is preset according to actual needs. For example, N can be set to 10, 15, or 20, etc. This embodiment does not specifically limit the value of N.

[0106] Step 405 : Using the first language model, according to each reference home operation instruction and the corresponding reference device identifier, determine the target device identifier of the text recognition result.

[0107] As a possible implementation method, each reference home operation instruction and the corresponding reference device identification and text recognition result can be added to the corresponding position in the first prompt word template to obtain the first prompt word, and the first prompt word is input into the first large language model to obtain the target device identification of the text recognition result.

[0108] In one example, the first prompt word template may include the first position to be filled and the corresponding second position to be filled in N first reference example pairs, and the third position to be filled and the corresponding fourth position to be filled in the first to-be-generated example pair, and a rule for generating content at the third position to be filled, the rule may be: based on the first reference example pair and the content filled in the third position to be filled, generate content at the fourth position to be filled, and the generated content at the fourth position to be filled is included in the target content, wherein the target content is the content filled in the second position to be filled. Correspondingly, after obtaining each reference home operation instruction and the corresponding reference device identifier, for a reference home operation instruction, the reference home operation instruction can be filled in the first position to be filled in a first reference example, and the reference device identifier can be filled in the second position to be filled in the first reference example, and the text recognition result can be filled in the third position to be filled, so as to obtain the first prompt word.

[0109] For example, if N is 10, after obtaining 10 reference home operation instructions and corresponding reference device identifiers, the reference home operation instructions, the corresponding reference device identifiers, and the text recognition result are added to the first prompt word template. The obtained first prompt word template is as follows:

[0110] First reference example pair 1: reference home operation instruction 1 and corresponding reference device identifier 1;

[0111] Second reference example pair 2: reference device identifier 2 corresponding to reference home operation instruction 2;

[0112] …

[0113] Second reference example pair 10: reference device identifier 10 corresponding to reference home operation instruction 10;

[0114] The first example pair to be generated: the text recognition result and the corresponding fourth position to be filled.

[0115] It should be noted that the target device identifier at the fourth to-be-filled position generated by the first large language model is at least one of the reference device identifiers 1 to N.

[0116] In one embodiment of the present application, to further improve the accuracy of the determined target device identification, after obtaining the reference device identifications corresponding to N reference home operation instructions, the target home type to which the smart home device corresponding to the reference device identification belongs can be determined. M reference home operation instructions and corresponding reference device identifications are randomly selected from home types other than the target home type, wherein the smart home devices corresponding to each of the M reference home operation instructions belong to different home types. Then, a first prompt word is generated based on all the obtained reference home operation instructions and the corresponding reference device identifications, and the first prompt word is input into the first large language model to obtain the target device identification of the text recognition result.

[0117] Here, M is preset according to actual needs. For example, M can be 10.

[0118] Step 406: Determine the home operation action from the text recognition result.

[0119] In one embodiment of the present application, the text recognition result can be input into the second largest language model to obtain the home operation action through the second largest language model.

[0120] Step 407 : Use the word representation service to perform vector representation on the home operation action to obtain a second representation vector.

[0121] Step 408: Obtain N second reference representation vectors from a preset vector library that match the first representation vector and whose device identifier is the target device identifier, and obtain the reference home operation action corresponding to each second reference representation vector and the reference device function of the reference home operation action.

[0122] It should be noted that the reference device function is a device function of a certain smart home device included in the home control system. In other words, in this embodiment, the smart home device to which the reference device function belongs is included in the home control system.

[0123] Here, N is preset according to actual needs. For example, N can be set to 10, 15, or 20, etc. This embodiment does not specifically limit the value of N.

[0124] Step 409 : Using the first language model, according to each reference home operation action and the corresponding reference device function, determine the target device function of the home operation action.

[0125] As a possible implementation method, each reference home operation action and the corresponding reference device function and home operation action can be added to the corresponding position in the second prompt word template to obtain a second prompt word, and the second prompt word is input into the first large language model to obtain the target device function of the home operation action.

[0126] In one example, the second prompt word template may include the first position to be filled and the corresponding second position to be filled in N second reference example pairs, and the third position to be filled and the corresponding fourth position to be filled in the second to-be-generated example pair, and a rule for generating content at the third position to be filled, the rule may be: based on the second reference example pair and the content filled in the third position to be filled, generate content at the fourth position to be filled, and the generated content at the fourth position to be filled is included in the target content, wherein the target content is the content filled in the second position to be filled. Correspondingly, after obtaining each reference home operation action and the corresponding reference device function, for a reference home operation instruction, the reference home operation action can be filled in the first position to be filled in a first reference example, and the reference device function can be filled in the second position to be filled in the first reference example, and the home operation instruction can be filled in the third position to be filled, so as to obtain the second prompt word.

[0127] For example, if N is 10, after obtaining 10 reference home operation actions and corresponding reference device functions, and adding the reference home operation actions and corresponding reference device functions to the second prompt word template, an example of the obtained second prompt word template is as follows:

[0128] Second reference example pair 1: reference home operation action 1 and corresponding reference device function 1;

[0129] Second reference example pair 2: reference device function 2 corresponding to reference home operation action 2;

[0130] …

[0131] The second example pair to be generated: the home operation action and the corresponding fourth position to be filled.

[0132] It should be noted that the target device function for the fourth to-be-filled position generated by the first large language model is included in the reference device function set, where the reference device function set is based on reference device function 1, ..., reference device function N. In other words, the target device function for the fourth to-be-filled position generated by the first large language model is one of the aforementioned reference device function 1, ..., reference device function N.

[0133] In one embodiment of the present application, to further improve the accuracy of the determined target device function, after obtaining the reference device functions corresponding to N reference home operation actions, the target function type to which the reference device function belongs can be determined, and M reference home operation actions and corresponding reference device functions can be randomly selected from function types other than the target function type, wherein the function types to which the reference device functions corresponding to each of the M reference home operation actions belong are different. Then, based on all the obtained reference home operation actions and their corresponding reference device functions, a second prompt word is generated and input into the first large language model to obtain the target device function of the home operation behavior.

[0134] Here, M is preset according to actual needs. For example, M can be 10.

[0135] Step 410, providing the text recognition result, target device identifier and target device function to the instruction execution service, so that the instruction execution service determines the adjustment target of the target device function based on the text recognition result, and generates a device control instruction for the target device identifier according to the target device function and the adjustment target, and sends the device control instruction to the smart home device corresponding to the target device identifier.

[0136] Correspondingly, after the smart home device receives the device control instruction, the smart home device adjusts its target device function according to the adjustment target based on the control instruction.

[0137] The control instruction is used to instruct the target device function in the smart home device corresponding to the target device identifier to be adjusted according to the adjustment target.

[0138] In this embodiment, through the first language model, combined with the reference home operation instruction matched by the home operation instruction and the corresponding reference device identification, the target device identification of the home operation instruction is accurately determined, and through the first language model, combined with the reference home operation action related to the home operation action of the home operation instruction and the reference device function of the reference home operation action, the target device function of the home operation action of the home operation instruction is accurately determined. Then, based on the home operation instruction, the determined target device identification and the target device function, the smart home device can be accurately controlled, thereby improving the effect of home control.

[0139] In addition, the target device identification and target device function of the home operation instruction can also be provided to the corresponding user, and the user can be prompted to determine whether the target device identification and target device function are accurate. If they are inaccurate, the first device identification and first device function filled in by the user for the home operation instruction can be obtained, and the first representation vector of the home operation instruction and the corresponding first device identification can be added to the vector library, and the second device function can be used as the device function of the home operation action of the home operation instruction, and the device identification of the home operation instruction, the second representation vector of the home operation action and the device function of the home operation action can be added to the vector library, thereby realizing the update of the vector library.

[0140] Among them, it should be noted that if the corresponding user determines that the target device identification and the target device function are accurate, correspondingly, the first representation vector of the home operation instruction and the corresponding target device identification are added to the vector library, and the target device function is used as the device function of the home operation action of the home operation instruction, and the device identification of the home operation instruction, the second representation vector of the home operation action and the device function of the home operation action are added to the vector library, thereby realizing the update of the vector library.

[0141] It should be noted that the Mongo database can store detailed data for each smart home device in the home control system. For example, the corresponding smart home device's device ID, home name, home description, home function, and home location information can be stored. Furthermore, detailed data for each device function of the smart home device can also be stored. For example, for any device function of the smart home device, the device ID, function ID, function name, function function, and function parameter description of the smart home device to which the device function belongs can be stored. This makes it convenient to query the Mongo database for relevant information about the corresponding smart home device in the home control system and / or to query for relevant information about a specific device function within the corresponding smart home device.

[0142] In addition, the Mongo database can also store historical conversation data records for a user. For example, the record identifier, session identifier, conversation turn number, conversation summary, identified device identifier, and identified device function for each historical conversation data record can be stored. This makes it easy to access conversation data records for historical conversations through the Mongo database.

[0143] It should be noted that when a first reference representation vector matching the first representation vector is not obtained from a preset vector library, all device identifiers in the home control system can be obtained, and the first largest language model can be used to determine the target device identifier of the home operation instruction based on all device identifiers.

[0144] It should be noted that when a second reference representation vector that matches the second representation vector is not obtained from the preset vector library, all device functions corresponding to the target device identifier can be obtained, and the first large language model can be used to determine the target device function of the home operation action of the home operation instruction based on all device functions of the target device identifier.

[0145] In order to implement the above embodiments, the present disclosure also provides a smart home control device based on a large language model.

[0146] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure.

[0147] like Figure 5 As shown, the smart home control device 50 based on the large language model may include: an acquisition module 501, a first determination module 502, a second determination module 503 and a home control module 504, wherein:

[0148] The acquisition module 501 is used to acquire home operation instructions.

[0149] The first determining module 502 is configured to determine a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction.

[0150] The second determining module 503 is configured to determine the target device control element of the home operation instruction according to the reference home operation instruction and the reference device control element by using the first large language model.

[0151] The home control module 504 is used to perform smart home control according to the target device control elements.

[0152] As a possible implementation of the embodiment of the present disclosure, the first determining module 502 may include:

[0153] a first determining unit, configured to determine representation information of a home operation instruction;

[0154] an acquisition unit, configured to acquire a target reference representation matching the representation information from a preset representation library;

[0155] The second determining unit is configured to use the target home operation instruction corresponding to the target reference representation as a reference home operation instruction, and use the device control element corresponding to the target home operation instruction as a reference device control element.

[0156] As a possible implementation of the embodiment of the present disclosure, the first determining unit is specifically configured to:

[0157] Extract key home control content from home operation instructions;

[0158] The key contents of home control are characterized to obtain representation information.

[0159] As a possible implementation of the embodiment of the present disclosure, the first determining unit is specifically configured to:

[0160] Input the home operation instructions into the second language model to obtain the key content of home control.

[0161] As a possible implementation method of the embodiment of the present disclosure, the second determination module 503 is specifically used to: add the reference home operation instructions, the reference device control elements and the home operation instructions to the corresponding positions in the prompt word template to obtain the prompt words; input the prompt words into the first language model to obtain the target device control elements.

[0162] As a possible implementation of the embodiment of the present disclosure, the target device control element includes: a target device identifier, and also includes:

[0163] a third determining module, configured to determine, based on the home operation instruction, a target device function required to satisfy the home operation instruction;

[0164] a fourth determination module, configured to determine an adjustment target for a target device function according to the home operation instruction;

[0165] The home control module 504 is specifically used to:

[0166] Determine the smart home device corresponding to the target device identifier, and control the target device function of the smart home device to be adjusted according to the adjustment target.

[0167] As a possible implementation of the embodiment of the present disclosure, the target device control elements include: target device functions, and also include:

[0168] A fifth determining module is used to determine, based on the home operation instruction, a target device identifier of a smart home device that satisfies the home operation instruction;

[0169] a sixth determining module, configured to determine an adjustment target for a target device function according to the home operation instruction;

[0170] The home control module 504 is specifically used to:

[0171] Determine the smart home device corresponding to the target device identifier, and control the target device function of the smart home device to be adjusted according to the adjustment target.

[0172] As a possible implementation of the embodiment of the present disclosure, the target device control elements include: target device identification, target device function, and also include:

[0173] a seventh determination module, configured to determine an adjustment target for a target device function according to the home operation instruction;

[0174] The home control module 504 is specifically used to:

[0175] Determine the smart home device corresponding to the target device identifier, and control the target device function of the smart home device to be adjusted according to the adjustment target.

[0176] It should be noted that the aforementioned explanation of the embodiment of the smart home control method based on the large language model is also applicable to the embodiment of the data device and will not be repeated here.

[0177] The large language model-based smart home control device of the disclosed embodiment, after obtaining a home operation instruction, determines a reference home operation instruction that matches the home operation instruction, as well as a reference device control element corresponding to the reference home operation instruction. The device then uses the first large language model to determine the target device control element of the home operation instruction based on the reference home operation instruction and the reference device control element, and performs smart home control based on the target device control element. Thus, by using the first large language model, combined with the reference home operation instruction that matches the home operation instruction and the corresponding reference device control element, the device control element analysis of the home operation instruction is performed, and the target device control element of the home operation instruction can be accurately determined, thereby accurately achieving smart home control based on the determined target device control element.

[0178] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, comply with relevant laws and regulations, and do not violate public order and good morals.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0180] Figure 6 Schematic block diagram of an electronic device for implementing the smart home control method based on a large language model according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0181] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0182] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 601 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the smart home control method based on a large language model. For example, in some embodiments, the smart home control method based on a large language model can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the smart home control method based on a large language model described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the smart home control method based on the large language model in any other appropriate manner (for example, by means of firmware).

[0184] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable large language model-based smart home control device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0189] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0190] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0191] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A smart home control method based on a large language model, comprising: Get home operation instructions; Determining a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction; Determine the target device control element of the home operation instruction according to the reference home operation instruction and the reference device control element by using the first language model; Smart home control is performed according to the target device control elements.

2. The method according to claim 1, wherein: The determining of a reference home operation instruction matching the home operation instruction and a reference device control element corresponding to the reference home operation instruction includes: Determining representation information of the home operation instruction; Acquire a target reference representation matching the representation information from a preset representation library; The target home operation instruction corresponding to the target reference representation is used as the reference home operation instruction, and the device control element corresponding to the target home operation instruction is used as the reference device control element.

3. The method according to claim 2, wherein: The determining of the characterization information of the home operation instruction includes: Extracting key home control content from the home operation instructions; The home control key content is characterized to obtain the characterization information.

4. The method according to claim 3, wherein: The step of extracting key home control content from the home operation instruction includes: The home operation instructions are input into the second largest language model to obtain the key content of the home control.

5. The method according to claim 1, wherein: Using the first language model, according to the reference home operation instruction and the reference device control element, determining the target device control element of the home operation instruction includes: Adding the reference home operation instruction, the reference device control element and the home operation instruction to corresponding positions in a prompt word template to obtain a prompt word; The prompt word is input into the first large language model to obtain the target device control element.

6. The method according to any one of claims 1 to 5, wherein: The target device control element includes: a target device identifier, and before the smart home control is performed according to the target device control element, it also includes: Determining, according to the home operation instruction, a target device function required to satisfy the home operation instruction; Determining an adjustment target of the target device function according to the home operation instruction; The performing smart home control according to the target device control element includes: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

7. The method according to any one of claims 1 to 5, wherein: The target device control element includes: target device function, and before performing smart home control according to the target device control element, it also includes: According to the home operation instruction, determining a target device identifier of a smart home device required to satisfy the home operation instruction; Determining an adjustment target of the target device function according to the home operation instruction; The performing smart home control according to the target device control element includes: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

8. The method according to any one of claims 1 to 5, wherein: The target device control element includes: a target device identifier and a target device function. Before performing smart home control according to the target device control element, the following is further included: Determining an adjustment target of the target device function according to the home operation instruction; The performing smart home control according to the target device control element includes: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

9. A smart home control device based on a large language model, comprising: An acquisition module, used to acquire home operation instructions; A first determination module, configured to determine a reference home operation instruction that matches the home operation instruction, and a reference device control element corresponding to the reference home operation instruction; A second determination module is used to determine the target device control element of the home operation instruction according to the reference home operation instruction and the reference device control element by using the first large language model; The home control module is used to perform smart home control according to the target device control elements.

10. The device according to claim 9, wherein: The first determining module includes: A first determining unit, configured to determine representation information of the home operation instruction; An acquisition unit, configured to acquire a target reference representation matching the representation information from a preset representation library; The second determining unit is used to use the target home operation instruction corresponding to the target reference representation as the reference home operation instruction, and use the device control element corresponding to the target home operation instruction as the reference device control element.

11. The device according to claim 10, wherein: The first determining unit is specifically configured to: Extracting key home control content from the home operation instructions; The home control key content is characterized to obtain the characterization information.

12. The device according to claim 11, wherein: The first determining unit is specifically configured to: The home operation instructions are input into the second largest language model to obtain the key content of the home control.

13. The device according to claim 9, wherein: The second determining module is specifically used to: Adding the reference home operation instruction, the reference device control element and the home operation instruction to corresponding positions in a prompt word template to obtain a prompt word; The prompt word is input into the first large language model to obtain the target device control element.

14. The device according to any one of claims 9 to 13, wherein: The target device control element includes: target device identification, and also includes: A third determination module is used to determine, according to the home operation instruction, a target device function required to satisfy the home operation instruction; A fourth determination module, configured to determine an adjustment target of the target device function according to the home operation instruction; The home control module is specifically used for: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

15. The device according to any one of claims 9 to 13, wherein: The target device control elements include: target device functions, and also include: A fifth determination module, configured to determine, according to the home operation instruction, a target device identifier of a smart home device that satisfies the home operation instruction; a sixth determining module, configured to determine an adjustment target of the target device function according to the home operation instruction; The home control module is specifically used for: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

16. The device according to any one of claims 9 to 13, wherein: The target device control elements include: target device identification, target device function, and also include: a seventh determination module, configured to determine an adjustment target of the target device function according to the home operation instruction; The home control module is specifically used for: The smart home device corresponding to the target device identifier is determined, and the target device function of the smart home device is controlled to be adjusted according to the adjustment target.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images