Intelligent equipment control method and device, electronic equipment and storage medium

Abstract

The invention relates to an intelligent equipment control method and device, electronic equipment and a storage medium. The method comprises the following steps: sensing an intelligent agent, obtaining multi-modal data, and processing the multi-modal data to obtain a sensing result; the decision-making intelligent agent is used for acquiring a sensing result and processing the sensing result to obtain a decision-making result; executing the intelligent agent, obtaining a decision result, and determining to-be-controlled intelligent equipment and to-be-executed control according to the decision result; the execution agent is used for controlling the intelligent equipment according to control and feeding back a control result and an equipment state of the intelligent equipment to the sensing agent; and compressing the pre-trained large model to generate an original agent, and dividing the original agent into a sensing agent, a decision-making agent and an execution agent according to the roles of the agents. In this way, the three types of lightweight intelligent agents are adopted for cooperation, the problem that the interaction real-time performance is poor due to the fact that a single large model network is high in complexity and large in parameter scale can be solved, and the interaction real-time performance is improved.

Description

Technical Field

[0001] The present application relates to the technical field of smart home appliances, and in particular to a smart device control method, device, electronic device, and storage medium. Background Art

[0002] At present, in order to realize the intelligent control function, the smart home systems of various smart home manufacturers usually rely on rules, templates, etc. For example, in order to realize the language function, it is necessary to rely on the user's accurate expression. For example, in order to realize APP control, the user needs to make accurate settings. This results in a poor user experience.

[0003] With the rapid development of big models in recent years, the understanding and generation capabilities of big models represented by chatGPT have been widely recognized by all walks of life. In order to enhance the intelligent experience of their own smart home systems, various smart home manufacturers have actively promoted the access of their own smart home systems to big models.

[0004] However, due to the high network complexity and large parameter scale of the large model itself, connecting the large model to the smart home system will result in poor real-time interaction of the smart home system, which makes it difficult to apply the large model in the smart home field. Summary of the invention

[0005] In order to solve the above technical problems that due to the high network complexity and large parameter scale of the large model itself, the large model is connected to the smart home system, resulting in poor interactive real-time performance of the smart home system, which makes it difficult to apply the large model in the field of smart home, this application provides a smart device control method, device, electronic device, and storage medium. The specific technical solution is as follows:

[0006] In a first aspect, the present application provides a method for controlling an intelligent device, the method comprising:

[0007] A perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result;

[0008] A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result;

[0009] Executing the intelligent agent, obtaining the decision result, and determining the intelligent device to be controlled and the manipulation to be performed according to the decision result;

[0010] The execution agent controls the smart device according to the manipulation, and feeds back the control result and device status of the smart device to the perception agent;

[0011] The pre-trained large model is compressed to generate an original intelligent agent, and the original intelligent agent is divided into a perception agent, a decision-making agent, and an execution agent according to the role of the intelligent agent.

[0012] In an optional implementation, compressing the pre-trained large model to generate an original agent, and dividing the original agent into a perception agent, a decision agent, and an execution agent according to the agent role, including:

[0013] Obtaining full basic data in the field of smart home, and fine-tuning the basic large model according to the full basic data to obtain a pre-trained large model;

[0014] Using a preset compression algorithm to compress the pre-trained large model to generate an original intelligent agent;

[0015] According to the role of the agent, the original agent is divided into perception agent, decision-making agent and execution agent.

[0016] In an optional implementation, fine-tuning the basic large model according to the full amount of basic data to obtain a pre-trained large model includes:

[0017] According to the restriction of the basic large model on input characters and the device category of the smart device, the full amount of basic data is segmented to obtain a plurality of sub-sequence basic data;

[0018] For any of the subsequence basic data, a prompt instruction for constructing a basic large model according to the subsequence basic data;

[0019] The prompt instruction is input into the basic large model so that the basic large model can learn the full amount of basic data to obtain a pre-trained large model.

[0020] In an optional embodiment, the method of using a preset compression algorithm to compress the pre-trained large model to generate an original agent includes:

[0021] Determine the current first requirement for response speed, the second requirement for storage capacity, and the third requirement for energy consumption;

[0022] The first requirement, the second requirement, and the third requirement are integrated to obtain a target requirement;

[0023] According to the target requirements, a preset compression algorithm is determined, and the pre-trained large model is compressed using the preset compression algorithm to generate an original intelligent agent.

[0024] In an optional embodiment, the number of the perceptual agents is multiple;

[0025] The perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result, including:

[0026] A perceiving agent acquires multimodal data, processes the multimodal data, and obtains a perceiving result; wherein the perceiving agent is normal;

[0027] or,

[0028] The target perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result;

[0029] Among them, the perception agent is abnormal, and the target perception agent is selected from the remaining perception agents.

[0030] In an optional embodiment, the number of the decision-making agents is multiple;

[0031] The decision agent obtains the perception result, processes the perception result, and obtains a decision result, including:

[0032] A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result; wherein the decision-making agent is normal;

[0033] or,

[0034] The target decision agent obtains the perception result, processes the perception result, and obtains a decision result;

[0035] Among them, the decision-making agent is normal, and the target decision-making agent is selected from the remaining decision-making agents.

[0036] In an optional implementation, the perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result, including:

[0037] The perceptual agent acquires object voice data, environmental data, and device status data of the smart device;

[0038] The perception agent fuses the object voice data, the environment data, and the device status data to obtain a perception result.

[0039] In a second aspect, the present application provides a smart device control device, the device comprising:

[0040] A data processing module is used to perceive the intelligent agent, obtain multimodal data, process the multimodal data, and obtain a perception result;

[0041] A result processing module is used for the decision-making agent to obtain the perception result, process the perception result, and obtain a decision result;

[0042] A device and manipulation determination module, used to execute the intelligent agent, obtain the decision result, and determine the intelligent device to be controlled and the manipulation to be performed according to the decision result;

[0043] A device control module, used for the execution agent to control the smart device according to the manipulation, and to feed back the control result and device status of the smart device to the perception agent;

[0044] The pre-trained large model is compressed to generate an original intelligent agent, and the original intelligent agent is divided into a perception agent, a decision-making agent, and an execution agent according to the role of the intelligent agent.

[0045] In an optional implementation, compressing the pre-trained large model to generate an original agent, and dividing the original agent into a perception agent, a decision agent, and an execution agent according to the agent role, including:

[0046] Obtaining full basic data in the field of smart home, and fine-tuning the basic large model according to the full basic data to obtain a pre-trained large model;

[0047] Using a preset compression algorithm to compress the pre-trained large model to generate an original intelligent agent;

[0048] According to the role of the agent, the original agent is divided into perception agent, decision-making agent and execution agent.

[0049] In an optional implementation, fine-tuning the basic large model according to the full amount of basic data to obtain a pre-trained large model includes:

[0050] According to the restriction of the basic large model on input characters and the device category of the smart device, the full amount of basic data is segmented to obtain a plurality of sub-sequence basic data;

[0051] For any of the subsequence basic data, a prompt instruction for constructing a basic large model according to the subsequence basic data;

[0052] The prompt instruction is input into the basic large model so that the basic large model can learn the full amount of basic data to obtain a pre-trained large model.

[0053] In an optional embodiment, the method of using a preset compression algorithm to compress the pre-trained large model to generate an original agent includes:

[0054] Determine the current first requirement for response speed, the second requirement for storage capacity, and the third requirement for energy consumption;

[0055] The first requirement, the second requirement, and the third requirement are integrated to obtain a target requirement;

[0056] According to the target requirements, a preset compression algorithm is determined, and the pre-trained large model is compressed using the preset compression algorithm to generate an original intelligent agent.

[0057] In an optional implementation, the number of the perceptual agents is multiple; and the data processing module is specifically used for:

[0058] A perceiving agent acquires multimodal data, processes the multimodal data, and obtains a perceiving result; wherein the perceiving agent is normal;

[0059] or,

[0060] The target perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result;

[0061] Among them, the perception agent is abnormal, and the target perception agent is selected from the remaining perception agents.

[0062] In an optional implementation, the number of the decision-making agents is multiple; the result processing module is specifically used to:

[0063] A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result; wherein the decision-making agent is normal;

[0064] or,

[0065] The target decision agent obtains the perception result, processes the perception result, and obtains a decision result;

[0066] Among them, the decision-making agent is normal, and the target decision-making agent is selected from the remaining decision-making agents.

[0067] In an optional implementation, the data processing module is specifically used to:

[0068] The perceptual agent acquires object voice data, environmental data, and device status data of the smart device;

[0069] The perception agent fuses the object voice data, the environment data, and the device status data to obtain a perception result.

[0070] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0071] Memory, used to store computer programs;

[0072] The processor is used to implement any of the smart device control methods described in the first aspect above when executing the program stored in the memory.

[0073] In a fourth aspect, a storage medium is also provided, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer executes any of the intelligent device control methods described in the first aspect.

[0074] In a fifth aspect, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned intelligent device control methods.

[0075] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the intelligent device control method provided by the embodiment of the present application comprises a perception agent that obtains multimodal data, processes the multimodal data, and obtains perception results; a decision-making agent that obtains perception results, processes the perception results, and obtains decision results; an execution agent that obtains decision results, determines the intelligent device to be controlled and the manipulation to be performed according to the decision results, and an execution agent that controls the intelligent device according to the manipulation, and feeds back the control results and device status of the intelligent device to the perception agent, wherein the pre-trained large model is compressed to generate an original agent, and the original agent is divided into a perception agent, a decision-making agent, and an execution agent according to the agent role.

[0076] The original intelligent agent is generated by compressing the pre-trained large model, and the original intelligent agent is divided into perception agent, decision-making agent, and execution agent according to the role of the intelligent agent. The perception agent then obtains and processes the multimodal data to obtain the perception result. The decision-making agent processes the perception result to obtain the decision result. The execution agent determines the intelligent device to be controlled and the operation to be executed based on the decision result, controls the intelligent device according to the operation, and feeds back the control result and device status of the intelligent device to the perception agent. The three types of lightweight intelligent agents are used to collaborate, which can solve the problem of poor interactive real-time performance caused by the high network complexity and large parameter scale of a single large model, and can improve the interactive real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0078] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0079] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0080] Figure 1 An architecture diagram of an intelligent device control system provided in an embodiment of the present application;

[0081] Figure 2 A schematic diagram of an implementation flow of a smart device control method provided in an embodiment of the present application;

[0082] Figure 3 A schematic diagram of an agent iteration provided in an embodiment of the present application;

[0083] Figure 4 A schematic diagram of the structure of an intelligent device control device provided in an embodiment of the present application;

[0084] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0086] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0087] In the embodiments of the present application, Figure 1As shown in the figure, it is an architecture diagram of a smart device control system provided by an embodiment of the present application, which mainly includes three types of agents, namely, perception agents, decision agents, and execution agents. Among them, the agents in the field of smart home are trained by using general knowledge based on a large model, and are divided into three types of agents, namely, perception agents, decision agents, and execution agents, according to the roles of the agents.

[0088] The perception agent and decision-making agent can be understood as the brain of the smart home system, and the execution agent can be understood as the limbs of the smart home system. The three types of agents adopt a collaborative approach to complete the tasks of the smart home system together. This can solve the problem of poor interactive real-time performance caused by the high complexity of a single large model network and a large parameter scale, and can improve the interactive real-time performance.

[0089] Among them, the basic large model can be fine-tuned with the full amount of basic data in the smart home field (non-real time), so that the large model can learn from its own general data to obtain the full amount of basic data in the smart home field, and then compress the large model (common compression methods such as: Lora, knowledge distillation, etc.), so that the original intelligent agent can be generated.

[0090] Constructing primitive agents into intelligent device control systems, such as Figure 1 As shown in the figure, according to the role of the intelligent agent, it is divided into three types of intelligent agents, including perception intelligent agents, decision-making intelligent agents, and execution intelligent agents, which are responsible for the multimodal data perception of the smart home system, the decision-making of the smart home system, and the execution of the smart home system. In addition, the intelligent agents need to cooperate with each other to complete the entire smart home system, and can iterate themselves according to the user's behavioral habits, preferences, and real-time professional data in the smart home field during the execution process, so as to better perceive the changes in the home environment status, understand the user's intentions, and actively adjust the home environment.

[0091] In order to avoid traditional cloud deployment and user data privacy security issues, the agent adopts a local deployment method, that is, it is deployed according to the device category of the smart device in the smart home system. For example, an agent can be deployed on a smart device: the temperature and humidity sensor device deploys the perception agent, the smart home central control and smart home host device deploys the decision agent, and the fan, dehumidifier, etc. deploy the execution agent. You can also deploy multiple agents on a smart device: a smart air conditioner deploys a decision agent and an execution agent.

[0092] The key perception agents and decision agents adopt a distributed deployment scheme, deploying the perception agents and decision agents on different smart devices in the smart home system. When an agent is abnormally unavailable, an available agent can be selected from the remaining agents to replace the abnormally unavailable agent to ensure the availability of the smart home system.

[0093] In this way, the perception agent completes the perception of multimodal data (including user voice data, environmental data, device status data, etc.), and fuses the perception data through a neural network, and then provides it to the decision-making agent. The decision-making agent, based on its own general data and the basic data in the smart home field obtained through early fine-tuning, fuses and processes the results provided by the perception agent and the real-time professional data in the smart home field to obtain a decision result, and then sends the decision result to the execution agent. The execution agent is responsible for the control of specific smart devices, and feeds back the results of the smart device operation and the device status to the perception agent, thereby forming a closed-loop operation of the smart device control system.

[0094] Based on the above, if Figure 2 FIG. 1 is a schematic diagram of an implementation flow of a smart device control method provided in an embodiment of the present application. The method may specifically include the following steps:

[0095] S201, a perception agent obtains multimodal data, processes the multimodal data, and obtains a perception result.

[0096] In the embodiment of the present application, the perceptual agent can be one of many perceptual agents, which can obtain multimodal data and then process the multimodal data to obtain a perceptual result. As a lightweight large model, the perceptual agent can obtain a perceptual result after fusing and processing the multimodal data through its own neural network.

[0097] Multimodal data generally refers to object (e.g. user) voice data, environmental data, device status data of smart devices, etc. Among them, environmental data may include, for example, ambient temperature, ambient humidity, etc. In addition, it may also include ambient images, etc., which is not limited in the embodiments of the present application.

[0098] Based on this, for a perceptual intelligent agent, it can obtain the object (such as user) voice data, environmental data, and device status data of the smart device, and then fuse and process the object (such as user) voice data, environmental data, and device status data of the smart device through its own neural network to obtain the perception result.

[0099] In addition, the perception agent can be one of many perception agents. If it becomes abnormal and unavailable, a target perception agent can be selected from the remaining perception agents to perceive multimodal data to ensure the availability of the smart home system.

[0100] It should be noted that, for selecting the target perception agent from the remaining perception agents, a voting election method or other automatic election methods can be used to select the target perception agent from the remaining perception agents, and the embodiments of the present application do not limit this.

[0101] To this end, when the perceptual agent is normal, it can obtain the object's (such as user's) voice data, environmental data, and device status data of the smart device, and then fuse and process the object's (such as user's) voice data, environmental data, and device status data of the smart device through its own neural network to obtain the perception result.

[0102] In the case that the perception agent is abnormally unavailable, the target perception agent can obtain the object's (such as user's) voice data, environmental data, and device status data of the smart device, and then fuse and process the object's (such as user's) voice data, environmental data, and device status data of the smart device through its own neural network to obtain the perception result.

[0103] S202, the decision-making agent obtains the perception results, processes the perception results, and obtains the decision results.

[0104] In the embodiment of the present application, the decision-making agent can be one of many decision-making agents, which can obtain the perception results and then process the perception results to obtain the decision results. Among them, the decision-making agent, as a lightweight large model, can fuse and process the perception results provided by the perception agent and the real-time professional data in the smart home field based on its own general data and the basic data in the smart home field obtained through early fine-tuning to obtain the decision results.

[0105] Among them, there can be multiple decision-making agents, and it can be one decision-making agent among many decision-making agents. If it becomes abnormal and unavailable, a target perception agent can be selected from the remaining decision-making agents to process the perception results to ensure the availability of the smart home system.

[0106] It should be noted that, for selecting the target perception agent from the remaining decision-making agents, a voting election method or other automatic election methods can be used to select the target perception agent from the remaining decision-making agents. The embodiments of the present application do not limit this.

[0107] To this end, when the decision-making agent is normal, it can obtain perception results, and then based on its own general data and the basic data in the smart home field obtained through previous fine-tuning, the perception results provided by the perception agent and the real-time professional data in the smart home field are integrated and processed to obtain the decision results.

[0108] In the case that the decision-making agent is abnormally unavailable, the target decision-making agent can obtain the perception results, and then, based on its own general data and the basic data in the smart home field obtained through previous fine-tuning, the perception results provided by the perception agent and the real-time professional data in the smart home field are integrated and processed to obtain the decision results.

[0109] S203, executing the intelligent agent, obtaining the decision result, and determining the intelligent device to be controlled and the operation to be performed according to the decision result.

[0110] S204, the execution agent controls the intelligent device according to the manipulation, and feeds back the control result and device status of the intelligent device to the perception agent.

[0111] In an embodiment of the present application, the executing intelligent agent can obtain the above-mentioned decision results, determine the intelligent device to be controlled and the operation to be executed based on the decision results, control the intelligent device to be controlled according to the determined operation, and then feed back the control results and device status of the intelligent device to be controlled to the perceiving intelligent agent, thereby forming a closed-loop operation of the intelligent device control system.

[0112] It should be noted that the control results and device status of the intelligent device to be controlled are fed back to the perception agent to form a closed-loop operation of the intelligent device control system. On the one hand, this is to verify whether the decision of the decision-making agent is normally executed by the execution agent, and to adjust subsequent decisions according to the execution situation; on the other hand, it is necessary to give user feedback based on the execution situation of the execution agent, such as interacting with the user through voice.

[0113] In addition, for the basic big model, the full amount of basic data in the smart home field can be obtained, and the basic big model can be fine-tuned according to the full amount of basic data in the smart home field to obtain a pre-trained big model. The pre-trained big model is compressed using a preset compression algorithm to generate the original intelligent agent, and the original intelligent agent is divided into perception intelligent agent, decision-making intelligent agent, and execution intelligent agent according to the role of the intelligent agent.

[0114] Among them, for the basic large model, there is generally a limit on the length of the input characters, and for the full amount of basic data in the field of smart home, if it is classified according to the device category of smart devices, it can be divided into basic data of different device categories. In this way, the full amount of basic data in the field of smart home can be segmented and the basic large model can be learned in batches.

[0115] To this end, the full amount of basic data in the smart home field is segmented according to the restrictions of the basic big model on input characters and the device categories of smart devices to obtain multiple sub-sequence basic data. For any sub-sequence basic data, a prompt instruction of the basic big model is constructed according to the sub-sequence basic data, and the prompt instruction is input into the basic big model, so that the basic big model can learn the full amount of basic data in the smart home field in batches to obtain a pre-trained big model.

[0116] Different smart home systems have different requirements for response speed, storage capacity, and energy consumption. For example, some smart home systems require the response speed to be as fast as possible, the storage capacity to be as small as possible, and the energy consumption to be as small as possible. Therefore, due to the different requirements for response speed, storage capacity, and energy consumption, different compression algorithms need to be used to compress the pre-trained large model and generate the original intelligent agent to meet the requirements for response speed, storage capacity, and energy consumption.

[0117] To this end, the first requirement for response speed, the second requirement for storage capacity, and the third requirement for energy consumption are determined, and the first, second, and third requirements are integrated to obtain the target requirement. According to the target requirement, a preset compression algorithm is determined, and the pre-trained large model is compressed using the preset compression algorithm to generate the original intelligent agent. Among them, the first requirement, the second requirement, and the third requirement can be integrated by weighted summation, and the preset compression algorithm corresponding to the target requirement can be found.

[0118] In this way, for the basic large model, after the above-mentioned fine-tuning and compression, the original intelligent agent can be obtained, and then the original intelligent agent is divided into perception intelligent agent, decision intelligent agent, and execution intelligent agent according to the role of the intelligent agent. And for the three types of intelligent agents, the perception intelligent agent, decision intelligent agent, and execution intelligent agent will adjust the perception results, decision results, and execution strategies according to the user's feedback (plus professional data in the field of smart home appliances), and continuously correct the output of the intelligent agent, so as to optimize the behavior of the smart home system and make the smart home system more in line with the user's expectations, such as Figure 3 shown.

[0119] Corresponding to the above method embodiment, the present application embodiment also provides a smart device control device, such as Figure 4 As shown, the apparatus may include: a data processing module 410 , a result processing module 420 , equipment, a manipulation determination module 430 , and an equipment control module 440 .

[0120] The data processing module 410 is used to perceive the intelligent agent, obtain multimodal data, process the multimodal data, and obtain a perception result;

[0121] The result processing module 420 is used for the decision agent to obtain the perception result, process the perception result, and obtain the decision result;

[0122] The device and manipulation determination module 430 is used to execute the intelligent agent, obtain the decision result, and determine the intelligent device to be controlled and the manipulation to be performed according to the decision result;

[0123] The device control module 440 is used for the execution agent to control the smart device according to the manipulation, and feed back the control result and device status of the smart device to the perception agent;

[0124] The pre-trained large model is compressed to generate an original intelligent agent, and the original intelligent agent is divided into a perception agent, a decision-making agent, and an execution agent according to the role of the intelligent agent.

[0125] In an optional implementation, compressing the pre-trained large model to generate an original agent, and dividing the original agent into a perception agent, a decision agent, and an execution agent according to the agent role, including:

[0126] Obtaining full basic data in the field of smart home, and fine-tuning the basic large model according to the full basic data to obtain a pre-trained large model;

[0127] Using a preset compression algorithm to compress the pre-trained large model to generate an original intelligent agent;

[0128] According to the role of the agent, the original agent is divided into perception agent, decision-making agent and execution agent.

[0129] In an optional implementation, fine-tuning the basic large model according to the full amount of basic data to obtain a pre-trained large model includes:

[0130] According to the restriction of the basic large model on input characters and the device category of the smart device, the full amount of basic data is segmented to obtain a plurality of sub-sequence basic data;

[0131] For any of the subsequence basic data, a prompt instruction for constructing a basic large model according to the subsequence basic data;

[0132] The prompt instruction is input into the basic large model so that the basic large model can learn the full amount of basic data to obtain a pre-trained large model.

[0133] In an optional embodiment, the method of using a preset compression algorithm to compress the pre-trained large model to generate an original agent includes:

[0134] Determine the current first requirement for response speed, the second requirement for storage capacity, and the third requirement for energy consumption;

[0135] The first requirement, the second requirement, and the third requirement are integrated to obtain a target requirement;

[0136] According to the target requirements, a preset compression algorithm is determined, and the pre-trained large model is compressed using the preset compression algorithm to generate an original intelligent agent.

[0137] In an optional implementation, the number of the perceptual agents is multiple; and the data processing module is specifically used for:

[0138] A perceiving agent acquires multimodal data, processes the multimodal data, and obtains a perceiving result; wherein the perceiving agent is normal;

[0139] or,

[0140] The target perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result;

[0141] Among them, the perception agent is abnormal, and the target perception agent is selected from the remaining perception agents.

[0142] In an optional implementation, the number of the decision-making agents is multiple; the result processing module is specifically used to:

[0143] A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result; wherein the decision-making agent is normal;

[0144] or,

[0145] The target decision agent obtains the perception result, processes the perception result, and obtains a decision result;

[0146] Among them, the decision-making agent is normal, and the target decision-making agent is selected from the remaining decision-making agents.

[0147] In an optional implementation, the data processing module is specifically used to:

[0148] The perceptual agent acquires object voice data, environmental data, and device status data of the smart device;

[0149] The perception agent fuses the object voice data, the environment data, and the device status data to obtain a perception result.

[0150] The present application also provides an electronic device, such as Figure 5As shown, it includes a processor 51, a communication interface 52, a memory 53 and a communication bus 54, wherein the processor 51, the communication interface 52, and the memory 53 communicate with each other through the communication bus 54.

[0151] A memory 53, for storing computer programs;

[0152] The processor 51 is used to execute the program stored in the memory 53 to implement the following steps:

[0153] A perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result; a decision-making agent acquires the perception result, processes the perception result, and obtains a decision result; an execution agent acquires the decision result, and determines the smart device to be controlled and the manipulation to be performed according to the decision result; the execution agent controls the smart device according to the manipulation, and feeds back the control result and device status of the smart device to the perception agent; wherein the pre-trained large model is compressed to generate an original agent, and the original agent is divided into a perception agent, a decision-making agent, and an execution agent according to the agent role.

[0154] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above air conditioner and other devices.

[0156] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] In another embodiment provided in the present application, a storage medium is further provided, in which instructions are stored. When the storage medium is run on a computer, the computer executes the intelligent device control method described in any one of the above embodiments.

[0159] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the intelligent device control method described in any one of the above embodiments.

[0160] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium, or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0161] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0162] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0163] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for controlling an intelligent device, characterized in that: The method comprises: A perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result; A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result; Executing the intelligent agent, obtaining the decision result, and determining the intelligent device to be controlled and the manipulation to be performed according to the decision result; The execution agent controls the smart device according to the manipulation, and feeds back the control result and device status of the smart device to the perception agent; The pre-trained large model is compressed to generate an original intelligent agent, and the original intelligent agent is divided into a perception agent, a decision-making agent, and an execution agent according to the role of the intelligent agent.

2. The method according to claim 1, characterized in that The method of compressing the pre-trained large model to generate an original agent, and dividing the original agent into a perception agent, a decision agent, and an execution agent according to the agent role, includes: Obtaining full basic data in the field of smart home, and fine-tuning the basic large model according to the full basic data to obtain a pre-trained large model; Using a preset compression algorithm to compress the pre-trained large model to generate an original intelligent agent; According to the role of the agent, the original agent is divided into perception agent, decision-making agent and execution agent.

3. The method according to claim 2, characterized in that The step of fine-tuning the basic large model according to the full amount of basic data to obtain a pre-trained large model includes: According to the restriction of the basic large model on input characters and the device category of the smart device, the full amount of basic data is segmented to obtain a plurality of sub-sequence basic data; For any of the subsequence basic data, a prompt instruction for constructing a basic large model according to the subsequence basic data; The prompt instruction is input into the basic large model so that the basic large model can learn the full amount of basic data to obtain a pre-trained large model.

4. The method according to claim 2, characterized in that: The method of using a preset compression algorithm to compress the pre-trained large model to generate an original intelligent agent includes: Determine the current first requirement for response speed, the second requirement for storage capacity, and the third requirement for energy consumption; The first requirement, the second requirement, and the third requirement are integrated to obtain a target requirement; According to the target requirements, a preset compression algorithm is determined, and the pre-trained large model is compressed using the preset compression algorithm to generate an original intelligent agent.

5. The method according to claim 1, characterized in that The number of the perceptual agents is multiple; The perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result, including: A perceiving agent acquires multimodal data, processes the multimodal data, and obtains a perceiving result; wherein the perceiving agent is normal; or, The target perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result; Among them, the perception agent is abnormal, and the target perception agent is selected from the remaining perception agents.

6. The method according to claim 1, characterized in that The number of the decision-making agents is multiple; The decision agent obtains the perception result, processes the perception result, and obtains a decision result, including: A decision-making agent obtains the perception result, processes the perception result, and obtains a decision result; wherein the decision-making agent is normal; or, The target decision agent obtains the perception result, processes the perception result, and obtains a decision result; Among them, the decision-making agent is normal, and the target decision-making agent is selected from the remaining decision-making agents.

7. The method according to claim 1, characterized in that The perception agent acquires multimodal data, processes the multimodal data, and obtains a perception result, including: The perceptual agent acquires object voice data, environmental data, and device status data of the smart device; The perception agent fuses the object voice data, the environment data, and the device status data to obtain a perception result.

8. A smart device control device, characterized in that: The device comprises: A data processing module is used to perceive the intelligent agent, obtain multimodal data, process the multimodal data, and obtain a perception result; A result processing module is used for the decision-making agent to obtain the perception result, process the perception result, and obtain a decision result; A device and manipulation determination module, used to execute the intelligent agent, obtain the decision result, and determine the intelligent device to be controlled and the manipulation to be performed according to the decision result; A device control module, used for the execution agent to control the smart device according to the manipulation, and to feed back the control result and device status of the smart device to the perception agent; The pre-trained large model is compressed to generate an original intelligent agent, and the original intelligent agent is divided into a perception agent, a decision-making agent, and an execution agent according to the role of the intelligent agent.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images