Scene arrangement method and device based on generative large model

Abstract

The invention discloses a scene arrangement method and device based on a generative large model, and relates to the technical field of smart home, and the scene arrangement method based on the generative large model comprises the steps: recognizing obtained control data of a target object, and obtaining a recognition result; generating a target arrangement instruction based on arrangement parameters required by scene arrangement and a historical scene action sequence of a historical arrangement scene, and determining an arrangement prompt template according to the target arrangement instruction and equipment parameters of the intelligent equipment; the arrangement prompt template and the recognition result are input into a large generative model, a target scene is generated according to an arrangement scene action sequence output by the large generative model, and the sequence format of the arrangement scene action sequence is consistent with the sequence format of the historical scene action sequence. The technical problem of how to improve the arrangement efficiency of scene arrangement for high-order scene arrangement is solved.

Description

Technical Field

[0001] This application relates to the technical field of smart homes. Specifically, it relates to a scenario orchestration method and device based on a generative large model. Background Art

[0002] A "scenario" generally refers to the comprehensive functions provided by a smart home system. According to the user's behaviors and needs, multiple devices can be coordinated to achieve specific functions of the devices to meet the user's specific needs. For example, when the user arrives home, a "coming home" scenario can be set. Under the "coming home" scenario, devices such as lights, air conditioners, and TVs at home can be controlled to provide a more comfortable home environment for the user.

[0003] Currently, the ways to do scenario orchestration include:

[0004] Manual orchestration: The operation staff of smart home manufacturers design general scenario templates and then instantiate them under each user's home. This type of method highly depends on manual experience, has low efficiency, and it is not easy to take into account the "personality" of each user.

[0005] Scenario orchestration based on mining user behavior habits: Use statistical data mining algorithms such as frequency counting and association rule mining to mine user habits, and then orchestrate scenarios according to the habits. Substantially, it is still a rule-oriented method. This type of method is relatively simple and effective for orchestrating some simple scenarios, but it is not easy to expand to high-order scenarios.

[0006] Scenario orchestration based on deep learning models: Establish a model based on features such as the user's behavior sequence and spatio-temporal context at home to learn the user's intention and then generate scenarios. This type of method can generate high-order scenarios, but the modeling complexity is very high, some features are limited in collection, and no very successful cases have been found yet.

[0007] However, "scenario orchestration" is a difficult point in the field of smart homes, with a low lower limit and a high upper limit. It can both orchestrate simple timed tasks such as "turn on the rice cooker at 7 o'clock every morning" and design high-order scenarios such as "laundry and drying linkage" and "one-key leaving home". From the current technology perspective, for simple scenario orchestration, mining the user's behavior habits is sufficient. But for high-order scenario orchestration, multiple data features need to be considered. This type of data has poor structure, is difficult to collect and clean, and different data greatly increases the complexity of data processing, resulting in low efficiency of scenario orchestration.

[0008] Therefore, in the related technology, for high-order scenario orchestration, there is a technical problem of how to improve the orchestration efficiency of scenario orchestration.

[0009] For the technical problem in the related technology of how to improve the orchestration efficiency of scenario orchestration for high-order scenario orchestration, no effective solution has been proposed yet. Summary of the Invention

[0010] An embodiment of the present application provides a scenario orchestration method and device based on a generative large model, so as to at least solve the technical problem in the related art of how to improve the orchestration efficiency of scenario orchestration for high-order scenarios.

[0011] According to an embodiment of the present application, a scenario orchestration method based on a generative large model is provided, including: identifying the control data of the target object obtained to obtain an identification result; generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario, and determining an orchestration prompt template according to the target orchestration instruction and the device parameters of the intelligent device; inputting the orchestration prompt template and the identification result into the generative large model, and generating a target scenario according to the orchestration scenario action sequence output by the generative large model, where the sequence format of the orchestration scenario action sequence is the same as the sequence format of the historical scenario action sequence.

[0012] In an exemplary embodiment, generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario includes: determining the historical orchestration parameters used by the historical devices in the historical orchestration scenario from the orchestration parameters; determining a first action sequence corresponding to the historical device from the historical scenario action sequence, and generating a first orchestration instruction according to the historical orchestration parameters used by the historical device and the first action sequence; performing an instruction test on the first orchestration instruction to obtain a test score of the first orchestration instruction; for multiple first orchestration instructions, determining the first orchestration instruction with the maximum test score among the multiple first orchestration instructions as the second orchestration instruction, and determining the second orchestration instruction as the target orchestration instruction.

[0013] In an exemplary embodiment, determining the second orchestration instruction as the target orchestration instruction includes: rewriting the second orchestration instruction using a preset rule to obtain a third orchestration instruction, where the preset rule includes at least one of the following: rewriting the action sequence in the target orchestration instruction, rewriting the orchestration parameters in the target orchestration instruction; performing an instruction test on the third orchestration instruction to obtain a test score of the third orchestration instruction; for multiple third orchestration instructions, determining the third orchestration instruction with the maximum test score among the multiple third orchestration instructions as the fourth orchestration instruction, and determining the fourth orchestration instruction as the target orchestration instruction.

[0014] In an exemplary embodiment, generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario includes: determining a sample scenario for training the generative large model according to the historical scenarios that the intelligent device has already executed; determining the orchestration parameters required for scenario orchestration according to the scenario parameters of the sample scenario, where the orchestration parameters required for scenario orchestration at least include one of the following: the first spatio-temporal environment information required for the scenario orchestration, the first intention description information required for the scenario orchestration, the first family portrait information required for the scenario orchestration, the first object preference information required for the scenario orchestration; generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario.

[0015] In an exemplary embodiment, determining an orchestration prompt template according to the target orchestration instruction and the device parameters of the intelligent device includes: obtaining a blank scenario orchestration template that has been established, where the blank scenario orchestration template at least includes a lead area and an orchestration prompt area; generating the orchestration prompt template by inputting the orchestration lead required for the scenario orchestration into the lead area of the scenario orchestration template, and inputting the device parameters of the intelligent device, the target orchestration instruction, the input sample corresponding to the sample scenario, and the sample scenario into the orchestration prompt area of the scenario orchestration template.

[0016] In an exemplary embodiment, inputting the orchestration prompt template and the recognition result into the generative large model, and generating a target scenario according to the orchestration scenario action sequence output by the generative large model includes: parsing the recognition result to obtain target parameter information, where the target parameter information includes at least one of the following: the second spatio-temporal environment information input by the target object, the second intention description information input by the target object, the second family portrait information input by the target object, the second object preference information input by the target object; obtaining the parameter format of the first scenario parameters of the sample scenario, and converting the target parameter information into the second scenario parameters of the target scenario according to the parameter format; inputting the orchestration prompt template and the second scenario parameters into the generative large model, and generating a target scenario according to the orchestration scenario action sequence output by the generative large model.

[0017] In an exemplary embodiment, generating a target scenario according to the choreographed scenario action sequence output by the generative large model includes: determining the choreographed scenario actions included in the choreographed scenario action sequence and the device numbers corresponding to the choreographed scenario actions, where the choreographed scenario actions are used to represent the device actions of the intelligent device in the target scenario; generating a key-value correspondence based on the device numbers and the choreographed scenario actions; in the case where it is determined that there are multiple choreographed scenario actions, generating a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence, and generating the target scenario according to the standard action sequence set.

[0018] In an exemplary embodiment, before generating a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence and generating the target scenario according to the standard action sequence set, the method further includes: in the case where it is determined that the choreographed scenario action exists in the mutually exclusive action sequence library, determining the mutually exclusive scenario action of the choreographed scenario action from the mutually exclusive action sequence library, where the mutually exclusive action sequence library includes at least one mutually exclusive scenario action corresponding to the choreographed scenario action; if it is determined that the execution order of the mutually exclusive scenario action precedes that of the choreographed scenario action, setting the mutually exclusive scenario action and the choreographed scenario action as the value part of the key-value pair to update the key-value pair.

[0019] In an exemplary embodiment, generating a key-value pair based on the device number and the choreographed scenario action includes: obtaining the previous choreographed scenario action corresponding to the choreographed scenario action from a preset action sequence library, where the preset action sequence library includes the device actions of the intelligent device indicated by the device number and the execution order of the device actions; in the case where it is determined that the previous choreographed scenario action has been executed, generating the key-value pair based on the device number and the choreographed scenario action; in the case where it is determined that the previous choreographed scenario action has not been executed, generating the key-value pair based on the device number and the previous choreographed scenario action.

[0020] In an exemplary embodiment, generating a key-value pair based on the device number and the choreographed scenario action includes: parsing the choreographed scenario action to obtain the action type of the choreographed scenario action and the action parameters of the choreographed scenario action; if it is determined that the action type of the choreographed scenario action is consistent with the action type of the customized scenario action of the target object, obtaining the predicted scenario action of the customized scenario action, where the predicted scenario action is obtained by performing regression calculation on the customized scenario action using a linear regression model; in the case where it is determined that the action parameters of the choreographed scenario action are inconsistent with the action parameters of the customized scenario action, setting the action parameters of the predicted scenario action as the action parameters of the customized scenario action.

[0021] According to another aspect of the embodiments of the present application, there is also provided a scenario orchestration device based on a generative large model, including: a result obtaining module, configured to identify the control data of the target object obtained to obtain an identification result; a template determination module, configured to generate a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario, and determine an orchestration prompt template according to the target orchestration instruction and the device parameters of the intelligent device; a scenario generation module, configured to input the orchestration prompt template and the identification result into the generative large model, and generate a target scenario according to the orchestration scenario action sequence output by the generative large model, wherein the sequence format of the orchestration scenario action sequence is the same as the sequence format of the historical scenario action sequence.

[0022] According to yet another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned scenario orchestration method based on a generative large model when running.

[0023] According to yet another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the above-mentioned processor executes the above-mentioned scenario orchestration method based on a generative large model through the computer program.

[0024] In the embodiments of the present application, the control data of the target object obtained is identified to obtain an identification result; a target orchestration instruction is generated based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario, and an orchestration prompt template is determined according to the target orchestration instruction and the device parameters of the intelligent device; the orchestration prompt template and the identification result are input into the generative large model, and a target scenario is generated according to the orchestration scenario action sequence output by the generative large model, wherein the sequence format of the orchestration scenario action sequence is the same as the sequence format of the historical scenario action sequence; by adopting the above technical solution, the technical problem of how to improve the orchestration efficiency of scenario orchestration for high-order scenarios is solved, and thus the orchestration efficiency of scenario orchestration is improved. Description of the Drawings

[0025] The drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application, and are used together with the description to explain the principles of the present application.

[0026] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0027] Figure 1 It is a schematic diagram of the hardware environment of a scenario orchestration method based on a generative large model according to an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a scenario orchestration method based on a generative large model according to an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of the process of a scenario orchestration method based on a generative large model according to an embodiment of the present application (one);

[0030] Figure 4 It is a schematic diagram of the process of a scenario orchestration method based on a generative large model according to an embodiment of the present application (two);

[0031] Figure 5 It is a structural block diagram of a scenario orchestration device based on a generative large model according to an embodiment of the present application. Detailed implementation manners

[0032] In order to enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0034] According to one aspect of the embodiments of the present application, a scenario orchestration method based on a generative large model is provided. The scenario orchestration method based on the generative large model is widely applied to whole-house intelligent digital control application scenarios such as Smart Home, smart home, smart home appliance ecosystem, and Intelligence House ecosystem. Optionally, in this embodiment, the above-mentioned scenario orchestration method based on the generative large model can be applied to, for example, Figure 1 the hardware environment composed of the terminal device 102 and the server 104 as shown. As Figure 1 shown, the server 104 is connected to the terminal device 102 through a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data operation services for the server 104.

[0035] The above network can include, but is not limited to, at least one of the following: wired network, wireless network. The above wired network can include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The above wireless network can include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 is not limited to being a PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projection device, smart TV, smart drying rack, smart curtain, smart audio and video, smart socket, smart speaker, smart sound box, smart fresh air device, smart kitchen and bathroom equipment, smart bathroom equipment, smart floor sweeping robot, smart window cleaning robot, smart mopping robot, smart air purification device, smart steam box, smart microwave oven, smart kitchen water heater, smart purifier, smart water dispenser, smart door lock, etc.

[0036] In this embodiment, a scenario orchestration method based on a generative large model is provided, which is applied to the above terminal device. Figure 2 It is a flowchart of the scenario orchestration method based on the generative large model according to the embodiments of the present application. The process includes the following steps:

[0037] Step S202, identify the control data of the obtained target object to obtain an identification result;

[0038] Among them, the control data of the above target object may include, for example, voice data, action data, or text data. For voice data, it can be obtained through a sound pickup device. For action data, it can be obtained through an image acquisition device. For text data, it can be obtained by converting text in an image. This application does not limit the data type and acquisition method of the control data.

[0039] Step S204: Generate a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario, and determine a choreography prompt template according to the target choreography instruction and the device parameters of the intelligent device.

[0040] Step S206: Input the choreography prompt template and the recognition result into a generative large model, and generate a target scenario according to the choreography scenario action sequence output by the generative large model, where the sequence format of the choreography scenario action sequence is the same as the sequence format of the historical scenario action sequence.

[0041] Through the above steps, the control data of the obtained target object is recognized to obtain a recognition result; a target choreography instruction is generated based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario, and a choreography prompt template is determined according to the target choreography instruction and the device parameters of the intelligent device; the choreography prompt template and the recognition result are input into a generative large model, and a target scenario is generated according to the choreography scenario action sequence output by the generative large model, where the sequence format of the choreography scenario action sequence is the same as the sequence format of the historical scenario action sequence, solving the technical problem in the related art of how to improve the choreography efficiency for high-order scenario choreography, and thus improving the choreography efficiency of scenario choreography.

[0042] In an exemplary embodiment, for the technical solution of generating a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario in step S204, the specific steps are as follows: Step S11: Determine the historical choreography parameters used by the historical devices in the historical choreography scenario from the choreography parameters; Step S12: Determine the first action sequence corresponding to the historical device from the historical scenario action sequence, and generate a first choreography instruction according to the historical choreography parameters used by the historical device and the first action sequence; Step S13: Perform an instruction test on the first choreography instruction to obtain a test score of the first choreography instruction; Step S14: For multiple first choreography instructions, determine the first choreography instruction with the maximum test score among the multiple first choreography instructions as the second choreography instruction, and determine the second choreography instruction as the target choreography instruction.

[0043] It can be understood that in this embodiment, for example, the historical choreography scenario is indoor cooling, the historical device is an air conditioner, the historical choreography parameter is the temperature value of the air conditioner, and the first action sequence corresponding to the air conditioner is to turn on the air conditioner and use the air conditioner's temperature adjustment function to adjust the temperature of the air conditioner. Then the generated first choreography instruction can be to turn on the air conditioner and adjust the temperature value of the air conditioner. If there are multiple first choreography instructions, then based on test cases, score the multiple first choreography instructions and the user intention reflected by the historical choreography parameters and the first action sequence. The score increases as the proximity to the user intention increases.

[0044] After generating the first choreography instruction according to the historical choreography parameters used by the historical device and the first action sequence in this embodiment, by performing instruction testing on the multiple first choreography instructions and determining the first choreography instruction with the maximum test score among the multiple first choreography instructions as the target choreography instruction, a more accurate choreography instruction can be determined, thereby improving the accuracy of the output result of the generative large model.

[0045] In an exemplary embodiment, an implementation scheme for determining the second choreography instruction as the target choreography instruction is further proposed, which specifically includes: rewriting the second choreography instruction using a preset rule to obtain a third choreography instruction, where the preset rule includes at least one of the following: rewriting the action sequence in the target choreography instruction, rewriting the choreography parameters in the target choreography instruction; performing instruction testing on the third choreography instruction to obtain the test score of the third choreography instruction; for multiple third choreography instructions, determining the third choreography instruction with the maximum test score among the multiple third choreography instructions as the fourth choreography instruction, and determining the fourth choreography instruction as the target choreography instruction.

[0046] It should be noted that the above rewriting can be understood as a process of simulating and generating similar choreography instructions based on the second choreography instruction. For example, if the second choreography instruction is to open the window for ventilation in a ventilation scenario, it can be rewritten as turning on the ventilation function of the air conditioner, or rewritten as turning on the fan for ventilation, etc., as the third choreography instruction. For the multiple third choreography instructions obtained by rewriting, testing is still required, and the third choreography instruction with the maximum test score is selected as the fourth choreography instruction.

[0047] In an exemplary embodiment, for generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario in step S204, there are also other implementation methods: determining a sample scenario for training the generative large model according to the historical scenarios that the intelligent device has already executed; determining the orchestration parameters required for scenario orchestration according to the scenario parameters of the sample scenario, where the orchestration parameters required for scenario orchestration at least include one of the following: the first spatio-temporal environment information required for scenario orchestration, the first intention description information required for scenario orchestration, the first family portrait information required for scenario orchestration, the first object preference information required for scenario orchestration; generating a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario.

[0048] When generating the target orchestration instruction, it can be combined with Figure 3 This process is described as follows. As Figure 3 shown, the specific steps are as follows:

[0049] Step S301, generating instructions. Several instructions can be generated using a forward generation template.

[0050] Among them, when using GPT to generate instructions, the forward generation template used includes:

[0051] Instruction: The task assigned to GPT. The text description of the instruction has great room for optimization, and the optimization method of the instruction will be further described below.

[0052] Background knowledge: When asking GPT to complete a task, the background knowledge that needs to be provided to it, including a list of home appliance functions and some precautions. This part is objective and has little room for optimization.

[0053] Input: For example, the data input by the user, spatio-temporal context and weather, family portrait and user preference data, etc.

[0054] Specific examples are as follows:

[0055] I gave a friend an instruction and five inputs. The friend read the instruction and wrote an output for every one of the inputs.

[0056] Here are the input-output pairs:

[0057] input: It is now June 19th in summer, 22:00 at night, and the temperature is hot...

[0058] Output: Trigger condition: Manual; Electrical appliance action: Air conditioner startup…

[0059] Input: (Omitted)

[0060] Output: (Omitted)

[0061] the instruction was <complete>。

[0062] Step S302: Use these instructions to organize prompts respectively and run scores on the pre-prepared test cases to obtain the scores corresponding to the instructions.

[0063] Step S303: Obtain the instruction with the highest score.

[0064] Step S304: Enter the iterative process of instruction optimization.

[0065] Step S305: The instruction with the highest score can be rewritten using GPT, and the rewritten instructions are scored. After several iterations, an instruction with the highest score will be obtained.

[0066] In an exemplary embodiment, the implementation process of determining the orchestration prompt template according to the target orchestration instruction and the device parameters of the intelligent device in step S204 is further described: Obtain the established blank scenario orchestration template, where the blank scenario orchestration template at least includes a lead area and an orchestration prompt area; by inputting the orchestration lead required for the scenario orchestration into the lead area of the scenario orchestration template, and inputting the device parameters of the intelligent device, the target orchestration instruction, the input sample corresponding to the sample scenario, and the sample scenario into the orchestration prompt area of the scenario orchestration template, the orchestration prompt template is generated.

[0067] Optionally, the scenario parameters of the sample scenario in this embodiment may include historical spatio-temporal environment information during sample scenario orchestration, first intention description information during sample scenario orchestration, first family portrait information during sample scenario orchestration, and first object preference information during sample scenario orchestration.

[0068] For the input sample, it may include, for example, examples of written input and output.

[0069] In an exemplary embodiment, the implementation method of generating a target scenario according to the choreography scenario action sequence output by the generative large model by inputting the choreography prompt template and the recognition result is described through the following steps, which specifically include: Step S21, parsing the recognition result to obtain target parameter information, where the target parameter information includes at least one of the following: the second spatio-temporal environment information input by the target object, the second intention description information input by the target object, the second family portrait information input by the target object, the second object preference information input by the target object; Step S22, obtaining the parameter format of the first scenario parameters of the sample scenario, and converting the target parameter information into the second scenario parameters of the target scenario according to the parameter format; Step S23, inputting the choreography prompt template and the second scenario parameters into the generative large model, and generating a target scenario according to the choreography scenario action sequence output by the generative large model.

[0070] For the process of converting target parameter information into the second scenario parameters of the target scenario according to the parameter format, for example, if the parameter format of the first scenario parameters is "parameter type: parameter value", then the parameter format of the converted second scenario parameters is also "parameter type: parameter value". For example, if the first scenario parameters are represented as {"id": 01, "name": "light on"}, and the target scenario parameters are "electrical appliance action: air conditioner on", then the second scenario parameters of the target scenario after converting the target parameter information can be {"id": 11, "name": "air conditioner on"}.

[0071] In an exemplary embodiment, a technical solution for implementing step S206 to generate a target scenario according to the choreography scenario action sequence output by the generative large model is also proposed, including: determining the choreography scenario actions included in the choreography scenario action sequence, and the device numbers corresponding to the choreography scenario actions, where the choreography scenario actions are used to represent the device actions of the intelligent device in the target scenario; generating a key-value correspondence based on the device numbers and the choreography scenario actions; in the case of determining that there are multiple choreography scenario actions, generating a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence, and generating the target scenario according to the standard action sequence set.

[0072] In the above embodiments, the multiple key-value pairs corresponding to the key-value correspondence can be understood as multiple key-value pairs with the device number as the key and the key-value pairs for arranging the scenario actions. For example, for device number 1, the arranged scenario action can be to turn on the device or turn off the device, so the key-value pair can be "1: Turn on the device" or "2: Turn off the device". Then, for different arranged scenario actions of device number 2, different key-value pairs can also be generated, and the standard action sequence set can include the arranged scenario actions of different devices. Through the above embodiments, the arranged scenario action sequence can be converted into a standard action sequence set representing the device actions, and then the purpose of generating the target scenario can be achieved according to the standard action sequence set, improving the coordination between different devices in the target scenario and the richness of the target scenario.

[0073] In an exemplary embodiment, before generating the standard action sequence set based on the multiple key-value pairs corresponding to the key-value correspondence and generating the target scenario according to the standard action sequence set, further, in the case where it is determined that the arranged scenario action exists in the mutually exclusive action sequence library, the mutually exclusive scenario action of the arranged scenario action is determined from the mutually exclusive action sequence library, where the mutually exclusive action sequence library contains at least one mutually exclusive scenario action corresponding to the arranged scenario action; if it is determined that the execution order of the mutually exclusive scenario action precedes that of the arranged scenario action, the mutually exclusive scenario action and the arranged scenario action are set as the value part of the key-value pair to update the key-value pair.

[0074] In the above embodiments, by using the mutually exclusive action sequence library to optimize the arranged scenario actions, the accuracy of the arranged scenario actions can be ensured. For example, for the arranged scenario action of adjusting the temperature of the refrigerator, if the refrigerator is not yet opened, the mutually exclusive scenario action of opening the refrigerator is supplemented to update the key-value pair.

[0075] In an exemplary embodiment, for the implementation process of generating the key-value pair based on the device number and the arranged scenario action, it specifically includes: Step S31, obtaining the previous arranged scenario action corresponding to the arranged scenario action from the preset action sequence library, where the preset action sequence library contains the device actions of the intelligent device indicated by the device number and the execution order of the device actions; Step S32, generating the key-value pair based on the device number and the arranged scenario action in the case where it is determined that the previous arranged scenario action has been executed; Step S33, generating the key-value pair based on the device number and the previous arranged scenario action in the case where it is determined that the previous arranged scenario action has not been executed.

[0076] Based on the above embodiments, the mutually exclusive scenario actions and the choreographed scenario actions can be set as the value part of the key-value pair to update the key-value pair. Also, in the case where the previous choreographed scenario action has not been completed, the key-value pair can be generated based on the device number and the previous choreographed scenario action, enriching the generation method of the key-value pair, improving the accuracy of the key-value pair, and further improving the choreography accuracy of the target scenario.

[0077] In an exemplary embodiment, other technical solutions for implementing the generation of key-value pairs based on the device number and the choreographed scenario actions are also proposed, including: parsing the choreographed scenario action to obtain the action type of the choreographed scenario action and the action parameters of the choreographed scenario action; if it is determined that the action type of the choreographed scenario action is consistent with the action type of the customized scenario action of the target object, then obtaining the predicted scenario action of the customized scenario action, where the predicted scenario action is obtained by performing regression calculation on the customized scenario action using a linear regression model; in the case where it is determined that the action parameters of the choreographed scenario action are inconsistent with the action parameters of the customized scenario action, setting the action parameters of the predicted scenario action as the action parameters of the customized scenario action.

[0078] It should be noted that the above scenario choreography method based on ChatGPT (i.e., the above-mentioned generative large model) can organize the prompt (i.e., the choreography prompt template) according to the voice input by the user, the user's home portrait, the spatio-temporal context, and the additional features of the executable actions of the electrical appliances, and feed it to the model. The model will output electrical appliance control instructions according to the prompt and the input data, and send the instructions to the user through the Internet of Things.

[0079] The network structure of the GPT series models used in this embodiment can refer to the decoder part of transformers such as GPT-3.5 and ChatGPT. The GPT series models can perform various natural language processing (NLP) tasks, such as article generation, question answering systems, open-domain chatting, etc. The GPT series models have many advanced skills, such as world knowledge, in-context learning, zero-shot learning, response to instructions, understanding and generation of code, chain of thought (CoT), etc. The capabilities of different versions of the models have different emphases.

[0080] Among them, the models of the GPT series include, for example: the Codex series of models, which focus on code understanding and generation, logical reasoning, and long-distance dependencies. The text-davinci-003 model, which has been fine-tuned by instructions and reinforced by human feedback (RLHF), has the ability to respond to instructions, CoT, and logical reasoning. The gpt-35-turbo model, which is the underlying model of ChatGPT and has also been through RLHF, is better at open-domain chatting.

[0081] The prompts when using the models of the GPT series for scenario orchestration are, for example:

[0082] Question: Generate a smart home scenario based on the following prompt. Prompt: "I'm so tired." Require to output an action sequence of household appliances and return it in an ordered list.

[0083] Answer: Output the action sequence:

[0084] 1. Start the smart air conditioner and adjust the temperature to a comfortable level;

[0085] 2. Start the smart lighting system and set the scene lights to a light color;

[0086] 3. Start the home music system and play relaxing music;

[0087] 4. Start the smart curtain system to darken the interior;

[0088] 5. Start the smart humidifier to increase the humidity and reduce the dryness of the air;

[0089] 6. Start the smart air purifier to purify the indoor air.

[0090] To better understand the process of the above-mentioned scenario orchestration method based on the generative large model, the following will further illustrate the implementation method flow of the above-mentioned scenario orchestration based on the generative large model in combination with optional embodiments, but it is not used to limit the technical solutions of the embodiments of the present application.

[0091] As Figure 4 shown, the implementation process of the scenario orchestration method based on the generative large model is illustrated using the gpt-35-turbo model (hereinafter referred to as turbo) in ChatGPT.

[0092] First, the following preparatory work needs to be implemented:

[0093] (1) Obtain input features: data input by the user (equivalent to the above-mentioned intention description information), spatio-temporal context and weather (equivalent to the above-mentioned spatio-temporal environment information), home portrait (equivalent to the above-mentioned home portrait information), and user preference data (equivalent to the above-mentioned object preference information), etc.

[0094] For the data input by the user, it can be voice data (which can indirectly provide the user's intention), or an electrical appliance control instruction.

[0095] Among them, electrical appliance control instructions such as "turn on the air conditioner, turn on the TV, turn off the water heater", and ChatGPT has good processing capabilities for such text inputs.

[0096] The user intentions obtained from voice data, such as "I'm hungry", "I'm sleepy and want to sleep", do not directly reflect electrical appliance control instructions and require some reasonable inferences. ChatGPT can also process such input corpora under appropriate guidance.

[0097] The processing of electrical appliance control instruction corpora is relatively simple and can be effectively processed even without ChatGPT. The processing of intention-based input corpora is more difficult, and the core value of using ChatGPT for scenario arrangement lies in processing such corpora.

[0098] Regarding the spatio-temporal context and weather, for example, it includes: season, week, time (month-day-hour), time period (morning, morning, noon, afternoon, evening, early morning), region (province-city-district-county), weather (cloudy-sunny-rainy-snowy / cold-hot). The spatio-temporal context can affect the activation and function modes of certain electrical appliances. For example, the air conditioner is turned on in summer, the heater is turned on in winter, and the curtains are closed at night.

[0099] The family portrait can also affect the activation and function modes of certain electrical appliances. Specifically, it can include, for example, the number of family members, whether there are elderly people, whether there are children, the composition of electrical appliances in the family, and family habit preferences.

[0100] The user preference data can be preset or obtained from the data input by the user.

[0101] (2) Obtain the prompt. The prompt has a great impact on the output result. How to organize the various input features in (1) into an efficient prompt is a core issue. Generally, a prompt can be composed of four parts: instructions, context, input data, and output indicator. The context can be the problem background or can include several examples for the model to use as examples.

[0102] After exploration, obtaining high-quality output requires leveraging two techniques in prompt engineering: few-shot and Chain-of-Thought (CoT). Few-shot means providing a small number of examples when organizing the prompt to assist the model in completing the task. It can also use the given examples to constrain the output format. For scenario arrangement tasks, some scenarios arranged according to input features are generally given as examples. CoT is the abbreviation of Chain-of-thought and can be used in combination with few-shot.

[0103] The following is an organized prompt example:

[0104] "

[0105] instructions (i.e., the lead area):

[0106] Refer to the following examples to perform smart home scenario arrangement.

[0107] context:

[0108] Some common electrical appliances and their functions and actions are as follows:

[0109] 1. Lights (provide illumination): Turn on the lights, turn off the lights, adjust the light brightness, turn on the meeting mode of the lights, turn on the dining mode of the lights;

[0110] 2. Curtains (control the indoor light and protect privacy): Increase the opening / closing degree of the curtains, decrease the opening / closing degree of the curtains;

[0111] 3. Security equipment (ensure the safety of the home): Deploy the security equipment, disarm the security equipment;

[0112] 4. Air conditioner (adjust the indoor temperature and humidity, and ventilate): Turn on the air conditioner, turn off the air conditioner, adjust the air conditioner temperature, adjust the air conditioner wind speed, turn on the intelligent humidity control mode of the air conditioner, turn on the fresh air mode of the air conditioner, turn on the even wind mode of the air conditioner;

[0113] 5. (Omitted here)

[0114] Some common sense of scenario arrangement is as follows:

[0115] 1. Turn on the lights when the light is insufficient at night, and do not turn on the lights when the light is sufficient during the day;

[0116] 2. Turn on the air conditioner when it is hot, and turn on the electric heater when it is cold;

[0117] 3. (Omitted here)

[0118] Input: It is May 2nd, 13:00 pm, and the temperature is suitable. There are two people in my family. The electrical appliances in my family are: TV, curtains, lights, air conditioner, speaker, washing machine, refrigerator. Hint: "I'm sleepy and want to sleep".

[0119] Output: TV is turned off (because it's time to sleep and the TV sound affects sleep), speaker sleep mode is turned on (because listening to light music helps with sleep), curtain opening / closing degree is lowered (because it's noon and there is sufficient sunlight, closing the curtain to block the sunlight helps with sleep).

[0120] Input: It is June 23rd, 18:00 in the afternoon, and the weather is a bit hot. There are four people in my family, including the elderly and a baby. The electrical appliances in my family are: TV, air conditioner, water heater, gas stove, refrigerator, washing machine. Hint: "The baby has wet the pants".

[0121] Output: Washing machine is turned on (because the baby has wet the pants and needs to wash them).

[0122] input data:

[0123] It is November 23rd, 18:00 in the afternoon, and the weather is a bit cold. There are three people in my family, including the elderly. The electrical appliances in my family are: TV, air conditioner, water heater, gas stove, washing machine, lamp. Hint: "I'm hungry".

[0124] output indicator:

[0125] Output: Lamp is turned on (because it's evening and light is needed), air conditioner is turned on (because the weather is a bit cold), air conditioner temperature is adjusted (to adjust to a suitable temperature), gas stove is turned on (because it's time to cook).

[0126] ”

[0127] Among them, the input data part is the input part organized according to the user's intention, its spatio-temporal characteristics at that time, and the family portrait. The output indicator part is the original output part of the chatgpt model, and the other parts are preset in advance.

[0128] The above prompt provides 2 examples of input and output (i.e., the above input samples), and adds the reason for each action to be output according to the CoT idea to guide the model to output appropriate electrical appliance actions.

[0129] (3) Result caching mechanism

[0130] The organizational format of the caching mechanism is, for example, key-value pairs:

[0131] Key: Intention description; Context characteristics (month, time period, region, weather, external temperature, air pollution index); Family portrait (number of family members, presence of special groups, types of electrical appliances in the family).

[0132] Value: The original return result of chatgpt (sequence of electrical appliance actions).

[0133] After each call to the ChatGPT interface, if the key of the result is in the cache, overwrite the original value; if it is not in the cache, add a new key-value pair.

[0134] After the above preparations are completed, execute the implementation process as Figure 4 shown below:

[0135] Step S401, speech recognition. Collect the user's original voice input (e.g., "A few friends are coming to my home for dinner tonight. Help me arrange a scenario"), perform speech recognition, and convert it into an intention description text.

[0136] Step S402, retrieve the cache. Based on the intention description text, spatio-temporal context features (through the context service), and the portrait features of this user, etc., retrieve the closest cache result in the cache library.

[0137] The conditions for determining whether a hit occurs are as follows: 1. The semantic similarity of the intention description text is greater than the threshold; 2. The context features are the same; 3. The family portrait is the same. If all the above conditions are met, return the result with the largest semantic similarity of the intention description. If a hit occurs, execute step S404; if not, execute step S406.

[0138] For example: The user's utterance is "A few friends are coming to my home for dinner tonight. Help me arrange a scenario". If a record with the closest semantics is found in the cache library, and its intention description is "Arrange a scenario. I'm going to invite a few friends to my home for dinner tonight", then the semantic similarity of the two intentions is greater than the threshold, and the spatio-temporal context and portrait features of this cache result are consistent with the corresponding features of this utterance, so it is determined to be a hit.

[0139] Step S404, after a hit, perform post-processing. The specific processing steps may include:

[0140] Step 1, parse the action sequence returned by ChatGPT. Map the electrical appliance actions in the result to the standard electrical appliance action system. In one embodiment, for example, the prompt input is as follows:

[0141] (Some tokens are omitted here)

[0142] It is now July 2nd, summer, 16:00 in the afternoon, and the weather is a bit hot. There is one person at home in my family now. My family is in Sichuan. My family likes to listen to light music. The electrical appliances in my family are: speaker, air conditioner, water heater, gas stove, refrigerator, washing machine, lamp, oven. Hint: "A few friends are coming to my home for dinner tonight. Help me arrange a scenario".

[0143] The original output of Chatgpt is as follows:

[0144] Electrical appliance actions: turning on the light dining mode (providing appropriate lighting when eating to create a dining atmosphere), turning on the air conditioner (turning on the air conditioner to make the room temperature comfortable), setting the air conditioner temperature (setting a comfortable temperature of 22 degrees), setting the speaker playback (playing rock music to make friends enjoy the dining time more), opening the refrigerator door (taking out ingredients and drinks from the refrigerator in advance), turning on the gas stove (turning on the gas stove to cook for friends), turning on the oven (turning on the oven to prepare baked food).

[0145] The converted standardized action sequence is as follows:

[0146]

[0147] Step 2: Solve possible logical errors in the action sequence based on rules.

[0148] Occasionally, logical errors may occur in the output of the GPT model: for example, electrical appliances are turned on and off at the same time, and functional modes of appliances that are not turned on are turned on. These logical errors can be corrected using established rules. For example, in the above example, "Light dining mode turned on" appears, but "Light turned on" does not appear. Here, the action of "Light turned on" will be filled in.

[0149] Step 3, use user behavior habits to correct some electrical appliance action parameter settings. Judging from the original arrangement results of GPT output, some electrical appliance parameter settings are uncontrollable, such as air conditioning temperature settings, water heater temperature settings, curtain opening and closing settings, etc. Based on user habits and contextual features, targeted parameter prediction models (such as air conditioning temperature prediction models, water heater water temperature prediction models, etc.) can be used to re-predict and correct the original output value of GPT output. For example, in the above example, the air conditioning temperature is set to 22 degrees, but the air conditioning temperature prediction model predicts that the user prefers 25 degrees, so 25 degrees will cover the original 22 degrees.

[0150] Step S405: After multiple rounds of voice inquiries, the scene is sent to the user through the Internet of Things. For example, after arranging the scene, the user is asked multiple rounds to confirm the scene triggering conditions (immediate execution or scheduled execution, etc.), and the arranged scene is sent to the user through the IOT.

[0151] Step S406: If no hit is found, prompt is organized and the chatgpt interface is called. In prompt, the appliance functions, executable actions and common sense parts of scene arrangement, and examples used for few-shot are all preset. For a user's utterance, only the intention description, spatiotemporal context, family portrait and other information will be changed. Prompt is fed to the model through the openai interface, and the results returned by the model are cached.

[0152] Although the current deep learning model can also achieve scenario arrangement according to the user's intention-based corpus, it is difficult for the deep learning model to represent the intention corpus well, and a large number of training samples and computing power are required. ChatGPT is a state-of-the-art language model. If scenario arrangement is based on ChatGPT, its powerful model capabilities and in-context learning characteristics can be utilized to avoid complex feature processing and modeling processes. In an end-to-end manner, the task of scenario arrangement can be handed over to ChatGPT to better achieve scenario arrangement. Through the above embodiments, the CoT technology in prompt engineering is introduced into scenario arrangement to optimize the reasoning ability of the model, making the arranged scenarios interpretable, and the automatic instruction optimization can further improve the result accuracy.

[0153] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0154] Figure 5 is a structural block diagram of a scenario arrangement device based on a generative large model according to an embodiment of the present application, as Figure 5 shown, including:

[0155] A result obtaining module 52, configured to identify the control data of the obtained target object to obtain an identification result;

[0156] Among them, the control data of the above target object may include, for example, voice data, action data, or text data. For voice data, it can be obtained through a sound pickup device. For action data, it can be obtained through an image acquisition device. For text data, it can be obtained by converting text from an image. The present application does not limit the data type and acquisition method of the control data.

[0157] A template determination module 54, configured to generate a target arrangement instruction based on the arrangement parameters required for scenario arrangement and the historical scenario action sequence of the historical arrangement scenario, and determine an arrangement prompt template according to the target arrangement instruction and the device parameters of the intelligent device;

[0158] A scenario generation module 56, configured to input the orchestration prompt template and the recognition result into a generative large model, and generate a target scenario according to an orchestration scenario action sequence output by the generative large model, where a sequence format of the orchestration scenario action sequence is the same as a sequence format of the historical scenario action sequence.

[0159] Through the above device, control data of a target object obtained is recognized to obtain a recognition result; a target orchestration instruction is generated based on orchestration parameters required for scenario orchestration and a historical scenario action sequence of a historical orchestration scenario, and an orchestration prompt template is determined according to the target orchestration instruction and device parameters of an intelligent device; the orchestration prompt template and the recognition result are input into a generative large model, and a target scenario is generated according to an orchestration scenario action sequence output by the generative large model, where a sequence format of the orchestration scenario action sequence is the same as a sequence format of the historical scenario action sequence, thereby solving the technical problem in the related art of how to improve the orchestration efficiency of scenario orchestration for high-order scenario orchestration, and further improving the orchestration efficiency of scenario orchestration.

[0160] Optionally, in an exemplary embodiment, the above template determination module 54 is further configured to perform the following steps: Step S11, determine historical orchestration parameters used by a historical device in the historical orchestration scenario from the orchestration parameters; Step S12, determine a first action sequence corresponding to the historical device from the historical scenario action sequence, and generate a first orchestration instruction according to the historical orchestration parameters used by the historical device and the first action sequence; Step S13, perform an instruction test on the first orchestration instruction to obtain a test score of the first orchestration instruction; Step S14, for multiple first orchestration instructions, determine a first orchestration instruction with the maximum test score among the multiple first orchestration instructions as a second orchestration instruction, and determine the second orchestration instruction as the target orchestration instruction.

[0161] It can be understood that in this embodiment, for example, the historical orchestration scenario is indoor cooling, the historical device is an air conditioner, the historical orchestration parameter is the temperature value of the air conditioner, and the first action sequence corresponding to the air conditioner is to turn on the air conditioner and use the air conditioner to adjust the temperature, then the generated first orchestration instruction can be to turn on the air conditioner and adjust the temperature value of the air conditioner. If there are multiple first orchestration instructions, then based on test cases, multiple first orchestration instructions and the user intention reflected by the historical orchestration parameters and the first action sequence are scored, and the score increases as the degree of proximity to the user intention increases.

[0162] In an exemplary embodiment, the above-mentioned template determination module 54 is further configured to: rewrite the second choreography instruction using a preset rule to obtain a third choreography instruction, where the preset rule includes at least one of the following: rewrite the action sequence in the target choreography instruction, rewrite the choreography parameters in the target choreography instruction; perform an instruction test on the third choreography instruction to obtain a test score of the third choreography instruction; for multiple third choreography instructions, determine the third choreography instruction with the maximum test score among the multiple third choreography instructions as the fourth choreography instruction, and determine the fourth choreography instruction as the target choreography instruction.

[0163] It should be noted that the above rewriting can be understood as a process of simulating and generating similar choreography instructions based on the second choreography instruction. For example, if the second choreography instruction is to open the window for ventilation in a ventilation scenario, it can be rewritten as turning on the ventilation function of the air conditioner, or rewritten as turning on the fan for ventilation, etc. as the third choreography instruction. For the multiple third choreography instructions obtained by rewriting, testing is still required, and the third choreography instruction with the maximum test score is selected as the fourth choreography instruction.

[0164] In an exemplary embodiment, the above-mentioned template determination module 54 is further configured to: determine a sample scenario for training the generative large model according to the historical scenarios that the intelligent device has executed; determine the choreography parameters required for scenario choreography according to the scenario parameters of the sample scenario, where the choreography parameters required for scenario choreography include at least one of the following: the first spatio-temporal environment information required for scenario choreography, the first intention description information required for scenario choreography, the first family portrait information required for scenario choreography, the first object preference information required for scenario choreography; generate a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario.

[0165] In an exemplary embodiment, the above-mentioned template determination module 54 is further configured to: obtain a blank scenario choreography template that has been established, where the blank scenario choreography template includes at least a lead area and a choreography prompt area; by inputting the scenario lead required for scenario choreography into the lead area of the scenario choreography template, and inputting the device parameters of the intelligent device, the target choreography instruction, the input sample corresponding to the sample scenario, and the sample scenario into the choreography prompt area of the scenario choreography template, generate the choreography prompt template.

[0166] Optionally, the scenario parameters of the sample scenario in this embodiment may include the historical spatio-temporal environment information during sample scenario choreography, the first intention description information during sample scenario choreography, the first family portrait information during sample scenario choreography, and the first object preference information during sample scenario choreography.

[0167] For the input samples, it may include, for example, prepared input and output examples.

[0168] In an exemplary embodiment, the above-mentioned scenario generation module 56 is further configured to implement the following steps: Step S21, parse the recognition result to obtain target parameter information, where the target parameter information includes at least one of the following: the second spatio-temporal environment information input by the target object, the second intention description information input by the target object, the second family portrait information input by the target object, the second object preference information input by the target object; Step S22, obtain the parameter format of the first scenario parameters of the sample scenario, and convert the target parameter information into the second scenario parameters of the target scenario according to the parameter format; Step S23, input the choreography prompt template and the second scenario parameters into the generative large model, and generate a target scenario according to the choreography scenario action sequence output by the generative large model.

[0169] For the process of converting the target parameter information into the second scenario parameters of the target scenario according to the parameter format, for example, if the parameter format of the first scenario parameters is "parameter type: parameter value", then the parameter format of the converted second scenario parameters is also "parameter type: parameter value". For example, if the first scenario parameters are expressed as {"id": 01, "name": "light on"}, and the target scenario parameters are "electrical appliance action: air conditioner on", then the second scenario parameters of the target scenario after converting the target parameter information can be {"id": 11, "name": "air conditioner on"}.

[0170] In an exemplary embodiment, the above-mentioned scenario generation module 56 is further configured to: determine the choreography scenario actions included in the choreography scenario action sequence, and the device numbers corresponding to the choreography scenario actions, where the choreography scenario actions are used to represent the device actions of the intelligent device in the target scenario; generate a key-value correspondence based on the device numbers and the choreography scenario actions; in the case of determining that there are multiple choreography scenario actions, generate a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence, and generate the target scenario according to the standard action sequence set.

[0171] In the above embodiments, the multiple key-value pairs corresponding to the key-value correspondence can be understood as multiple key-value pairs with the device number as the key and the key-value pairs for arranging the scenario actions. For example, for device number 1, the arranged scenario action can be to turn on the device or turn off the device, and the key-value pairs can be "1: Turn on the device" or "2: Turn off the device". Then, for different arranged scenario actions of device number 2, different key-value pairs can also be generated, and the standard action sequence set can include the arranged scenario actions of different devices. Through the above embodiments, the arranged scenario action sequence can be converted into a standard action sequence set representing the device actions, and then the purpose of generating the target scenario can be achieved according to the standard action sequence set, improving the coordination between different devices in the target scenario and the richness of the target scenario.

[0172] In an exemplary embodiment, the above scenario generation module 56 is further configured to, before generating a standard action sequence set based on the multiple key-value pairs corresponding to the key-value correspondence and generating the target scenario according to the standard action sequence set, further determine, when it is determined that the arranged scenario action exists in the mutually exclusive action sequence library, the mutually exclusive scenario action of the arranged scenario action from the mutually exclusive action sequence library, where the mutually exclusive action sequence library includes at least one mutually exclusive scenario action corresponding to the arranged scenario action; if it is determined that the execution order of the mutually exclusive scenario action precedes that of the arranged scenario action, set the mutually exclusive scenario action and the arranged scenario action as the value part of the key-value pair to update the key-value pair.

[0173] In the above embodiments, by using the mutually exclusive action sequence library to optimize the arranged scenario actions, the accuracy of the arranged scenario actions can be ensured. For example, for the arranged scenario action of adjusting the temperature of the refrigerator, if the refrigerator is not yet turned on, the mutually exclusive scenario action of turning on the refrigerator is added to update the key-value pair.

[0174] In an exemplary embodiment, the above scenario generation module 56 is further configured to: Step S31, obtain the previous arranged scenario action corresponding to the arranged scenario action from the preset action sequence library, where the preset action sequence library includes the device actions of the intelligent device indicated by the device number and the execution order of the device actions; Step S32, generate the key-value pair based on the device number and the arranged scenario action when it is determined that the previous arranged scenario action has been executed; Step S33, generate the key-value pair based on the device number and the previous arranged scenario action when it is determined that the previous arranged scenario action has not been executed.

[0175] In an exemplary embodiment, the above-mentioned scenario generation module 56 is further configured to: parse the choreographed scenario action to obtain the action type of the choreographed scenario action and the action parameters of the choreographed scenario action; if it is determined that the action type of the choreographed scenario action is the same as the action type of the customized scenario action of the target object, obtain the predicted scenario action of the customized scenario action, where the predicted scenario action is obtained by performing regression calculation on the customized scenario action using a linear regression model; in the case where it is determined that the action parameters of the choreographed scenario action are inconsistent with the action parameters of the customized scenario action, set the action parameters of the predicted scenario action to the action parameters of the customized scenario action.

[0176] An embodiment of the present application also provides a storage medium, which includes a stored program, wherein the above program runs to execute the method of any one of the above.

[0177] Optionally, in this embodiment, the above storage medium may be set to store program code for performing the following steps:

[0178] S1, identify the control data of the obtained target object to obtain an identification result;

[0179] S2, generate a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreographed scenario, and determine a choreography prompt template according to the target choreography instruction and the device parameters of the intelligent device;

[0180] S3, input the choreography prompt template and the identification result into a generative large model, and generate a target scenario according to the choreographed scenario action sequence output by the generative large model, where the sequence format of the choreographed scenario action sequence is the same as the sequence format of the historical scenario action sequence.

[0181] An embodiment of the present application also provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0182] Optionally, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0183] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0184] S1, identify the control data of the obtained target object to obtain an identification result;

[0185] S2, generate a target orchestration instruction based on the orchestration parameters required for scenario orchestration and the historical scenario action sequence of the historical orchestration scenario, and determine an orchestration prompt template according to the target orchestration instruction and the device parameters of the intelligent device;

[0186] S3, input the orchestration prompt template and the recognition result into a generative large model, and generate a target scenario according to the orchestration scenario action sequence output by the generative large model, where the sequence format of the orchestration scenario action sequence is the same as that of the historical scenario action sequence.

[0187] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk, or optical disc, etc., various media that can store program codes.

[0188] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0189] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0190] The above is only the preferred implementation manner of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present application.< / complete>

Claims

1. A scenario orchestration method based on a generative large model, characterized in that, Including: Identifying the control data of the obtained target object to obtain an identification result; Generating a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario, and determining a choreography prompt template according to the target choreography instruction and the device parameters of the intelligent device; Inputting the choreography prompt template and the identification result into a generative large model, and generating a target scenario according to the choreography scenario action sequence output by the generative large model, wherein the sequence format of the choreography scenario action sequence is the same as the sequence format of the historical scenario action sequence.

2. The scenario orchestration method based on the generative large model according to claim 1, wherein, Generating a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario, including: Determining the historical choreography parameters used by the historical devices in the historical choreography scenario from the choreography parameters; Determining a first action sequence corresponding to the historical device from the historical scenario action sequence, and generating a first choreography instruction according to the historical choreography parameters used by the historical device and the first action sequence; Conducting an instruction test on the first choreography instruction to obtain a test score of the first choreography instruction; for multiple first choreography instructions, determining the first choreography instruction with the maximum test score among the multiple first choreography instructions as the second choreography instruction, and determining the second choreography instruction as the target choreography instruction.

3. The method for scenario orchestration based on a generative large model according to claim 2, wherein, Determining the second choreography instruction as the target choreography instruction, including: Rewriting the second choreography instruction using a preset rule to obtain a third choreography instruction, wherein the preset rule includes at least one of the following: rewriting the action sequence in the target choreography instruction, rewriting the choreography parameters in the target choreography instruction; Conducting an instruction test on the third choreography instruction to obtain a test score of the third choreography instruction; for multiple third choreography instructions, determining the third choreography instruction with the maximum test score among the multiple third choreography instructions as the fourth choreography instruction, and determining the fourth choreography instruction as the target choreography instruction.

4. The scenario orchestration method based on the generative large model according to claim 1, wherein Generating a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario, including: Determining a sample scenario for training the generative large model according to the historical scenarios that the intelligent device has executed; Determining the choreography parameters required for scenario choreography according to the scenario parameters of the sample scenario, wherein the choreography parameters required for scenario choreography include at least one of the following: the first spatio-temporal environment information required for scenario choreography, the first intention description information required for scenario choreography, the first family portrait information required for scenario choreography, the first object preference information required for scenario choreography; Generating a target choreography instruction based on the choreography parameters required for scenario choreography and the historical scenario action sequence of the historical choreography scenario.

5. The method for scenario orchestration based on a generative large model according to claim 4, wherein, Determining a choreography prompt template according to the target choreography instruction and the device parameters of the intelligent device, including: Obtaining a blank scenario choreography template that has been established, wherein the blank scenario choreography template includes at least a lead area and a choreography prompt area; By inputting the orchestration lead required for the scenario orchestration into the lead area of the scenario orchestration template, and inputting the device parameters of the intelligent device, the target orchestration instruction, the input sample corresponding to the sample scenario, and the sample scenario into the orchestration prompt area of the scenario orchestration template, the orchestration prompt template is generated.

6. The scenario orchestration method based on a generative large model according to claim 4, wherein Inputting the orchestration prompt template and the recognition result into the generative large model, and generating a target scenario according to the orchestration scenario action sequence output by the generative large model, including: Parsing the recognition result to obtain target parameter information, where the target parameter information includes at least one of the following: the second spatio-temporal environment information input by the target object, the second intention description information input by the target object, the second family portrait information input by the target object, and the second object preference information input by the target object; Obtaining the parameter format of the first scenario parameters of the sample scenario, and converting the target parameter information into the second scenario parameters of the target scenario according to the parameter format; Inputting the orchestration prompt template and the second scenario parameters into the generative large model, and generating a target scenario according to the orchestration scenario action sequence output by the generative large model.

7. The method for scenario orchestration based on a generative large model according to claim 1 or 6, characterized in that, Generating a target scenario according to the orchestration scenario action sequence output by the generative large model, including: Determining the orchestration scenario actions included in the orchestration scenario action sequence, and the device numbers corresponding to the orchestration scenario actions, where the orchestration scenario actions are used to represent the device actions of the intelligent device in the target scenario; Generating a key-value correspondence based on the device number and the orchestration scenario action; In the case of determining that there are multiple orchestration scenario actions, generating a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence, and generating the target scenario according to the standard action sequence set.

8. The scenario orchestration method based on a generative large model according to claim 7, wherein Before generating a standard action sequence set based on multiple groups of key-value pairs corresponding to the key-value correspondence and generating the target scenario according to the standard action sequence set, the method further includes: In the case of determining that the orchestration scenario action exists in the mutually exclusive action sequence library, determining the mutually exclusive scenario action of the orchestration scenario action from the mutually exclusive action sequence library, where the mutually exclusive action sequence library includes at least one mutually exclusive scenario action corresponding to the orchestration scenario action; If it is determined that the execution order of the mutually exclusive scenario action precedes the orchestration scenario action, setting the mutually exclusive scenario action and the orchestration scenario action as the value part of the key-value pair to update the key-value pair.

9. The method for scenario orchestration based on a generative large model according to claim 7, wherein Generating a key-value pair based on the device number and the orchestration scenario action, including: Obtaining the previous orchestration scenario action corresponding to the orchestration scenario action from the preset action sequence library, where the preset action sequence library includes the device actions of the intelligent device indicated by the device number and the execution order of the device actions; In the case of determining that the previous orchestration scenario action has been executed, generating the key-value pair based on the device number and the orchestration scenario action; In the case where it is determined that the previous choreographed scene action has not been completed, generate the key-value pair based on the device number and the previous choreographed scene action.

10. The method for scenario orchestration based on a generative large model according to claim 7, wherein Generating a key-value pair based on the device number and the choreographed scene action includes: Parse the choreographed scene action to obtain the action type of the choreographed scene action and the action parameters of the choreographed scene action; If it is determined that the action type of the choreographed scene action is the same as the action type of the customized scene action of the target object, obtain the predicted scene action of the customized scene action, where the predicted scene action is obtained by performing regression calculation on the customized scene action using a linear regression model; In the case where it is determined that the action parameters of the choreographed scene action are inconsistent with the action parameters of the customized scene action, set the action parameters of the predicted scene action to the action parameters of the customized scene action.

11. A scenario orchestration device based on a generative large model, characterized in that, Including: A result obtaining module for identifying the control data of the obtained target object to obtain an identification result; A template determination module for generating a target choreography instruction based on the choreography parameters required for scene choreography and the historical scene action sequence of the historical choreographed scene, and determining a choreography prompt template according to the target choreography instruction and the device parameters of the intelligent device; A scene generation module for inputting the choreography prompt template and the identification result into a generative large model, and generating a target scene according to the choreographed scene action sequence output by the generative large model, where the sequence format of the choreographed scene action sequence is the same as the sequence format of the historical scene action sequence.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where the program, when running, executes the method described in any one of claims 1 to 10 above.

13. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method described in any one of claims 1 to 10 through the computer program.

Images