Methods, devices, and storage media for constructing large-scale environmental interaction models based on protocol learning
By importing protocol learning templates and databases into the environment interaction model, a target model conforming to protocol rules is generated, solving the problem of low interaction compatibility in environment interaction scenarios and achieving highly scalable and low-cost model construction.
Patent Information
- Application Number
- CN202510651304.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2045-05-20
AI Technical Summary
Existing technologies lack the ability to expand into various environmental interaction scenarios, resulting in poor interaction compatibility.
By importing protocol learning templates into the initial model for protocol learning, an interactive model is generated. The model is then modified according to protocol-based decision rules and content output rules. Combined with the protocol database for data query and association, a target model is generated, enabling efficient migration of solutions for vertical domain scenarios.
The system enables the model training visibility setting to converge as the training process progresses, promotes the generalization of the model training invisible protocol, improves interaction compatibility and scalability, and reduces construction costs.
Smart Images

Figure CN120706539B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device and storage medium for constructing a large-scale environmental interaction model based on protocol learning. Background Technology
[0002] With the development of large-scale model technology, chatbots based on large-scale models have been widely used in important fields such as sales, customer service, management, and smart cities, greatly promoting the intelligentization of modern society and providing convenience for residents' lives. However, the related technologies lack the ability to expand into various environmental interaction scenarios, resulting in poor interaction compatibility.
[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention
[0004] The main purpose of this application is to provide a method, device and storage medium for constructing a large-scale environmental interaction model based on protocol learning, which aims to solve the technical problem that related technologies lack scene expansion capabilities in environmental interaction scenarios, resulting in low interaction compatibility.
[0005] To achieve the above objectives, this application proposes a method for a large-scale environmental interaction model based on protocol learning, the method comprising:
[0006] Import the protocol learning template into the initial model, perform protocol learning on the initial model, and generate an interactive model;
[0007] Based on the protocol-based decision rules and content output rules, the interaction model is modified to obtain a rule-based interaction model.
[0008] The rule-based interaction model is linked to the protocol database through data query to generate the target model, wherein the protocol database includes environmental interaction protocols.
[0009] In one embodiment, based on the initial model, a protocol learning template is imported to perform protocol learning training for the dialogue instruction task, and protocol learning training weights are generated.
[0010] The protocol learning training weights are imported into the initial model, the protocol learning is tested, and test results are generated.
[0011] Based on the test results, the protocol learning training weights are verified to obtain the verified protocol learning training weights.
[0012] The verified protocol learning training weights are imported into the initial model, the pre-training weights of the initial model are updated, and the interactive model is generated.
[0013] In one embodiment, based on the interaction model and reinforcement learning framework, logical rules of the business scenario are simulated to generate state observations;
[0014] The interaction model generates feedback decisions based on the state observations, and associates the state observations with the feedback decisions to generate the protocolized decision rules;
[0015] Based on the interaction model, combined with the protocol-based decision rules and preset conditions, content samples that meet the preset conditions are generated, and an alignment verification set is constructed through a hierarchical random sampling strategy.
[0016] Based on the alignment validation set, decision tree rules are extracted to generate the content output rules.
[0017] In one embodiment, the protocol-based decision rules are imported into the interaction model, the decision parameters of the protocol-based decision are adjusted, and a protocol-based interaction model is generated.
[0018] Import the content output rules into the protocol-based interaction model, adjust the content output filtering parameters, and generate the rule interaction model.
[0019] In one embodiment, the data of the environmental interaction protocol is stored in a database to construct an extensible protocol database;
[0020] Based on the rule-based interaction model, an association query table is established with the protocol database to generate the target model.
[0021] In one embodiment, a structured protocol interface is established between the target model and the protocol database based on an association query table;
[0022] Based on the target model, the protocol database, the related query table, and the structured protocol interface, the scenario deployment of the target model is executed.
[0023] In one embodiment, when environmental interaction is involved, control signals are constructed through protocol-based decision rules;
[0024] The control signals are parsed through hard-coded program code to make interactive requests to the associated resources and obtain response feedback;
[0025] The response feedback is fed into the target model to generate an interactive response that yields the interaction result;
[0026] Update the content of the immediate response based on the interaction response.
[0027] In one embodiment, if there is an operation type in the environmental interaction that is not in the protocol decision rules, then the protocol database is queried according to the association query table to obtain the interaction protocol corresponding to the operation type;
[0028] Based on the aforementioned interaction protocol, the control signals constructed through protocol-based decision rules are updated.
[0029] In addition, to achieve the above objectives, this application also proposes an environment interaction large model device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the protocol learning-based environment interaction large model construction method described above.
[0030] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the protocol-based learning-based environment interaction large model construction method described above.
[0031] This application provides a method for constructing a large-scale environmental interaction model based on protocol learning. The method includes importing a protocol learning template into an initial model, performing protocol learning on the initial model to generate an interaction model, modifying the interaction model according to protocol-based decision rules and content output rules to obtain a rule-based interaction model, and querying and associating the rule-based interaction model with a protocol database to generate a target model. The protocol database includes environmental interaction protocols. Based on a complete protocol learning setup, this application enables efficient migration of solutions for vertical scenarios through simple protocol configuration and the introduction of an additional low-cost database, without requiring expensive manual secondary development and costly further training or fine-tuning.
[0032] In summary, this application, through the adaptive design of protocol learning templates and by combining them with a protocol database, further expands the relevant protocol settings, enabling the visible settings for model training to converge as the training process proceeds. At the same time, it promotes the generalization of protocols that are not visible during model training, overcoming the technical problem of low interaction compatibility in related technologies and achieving a high degree of scalability. Attached Figure Description
[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1This is a flowchart illustrating the first embodiment of the protocol-based learning-based environment interaction large model construction method of this application;
[0036] Figure 2 This is a flowchart illustrating the second embodiment of the protocol-based learning-based environment interaction large model construction method of this application;
[0037] Figure 3 This is a flowchart of the learning process for this application agreement;
[0038] Figure 4 This is a flowchart illustrating the third embodiment of the protocol-based learning-based environment interaction large model construction method of this application;
[0039] Figure 5 This is a diagram of the interactive system of the learning model in this application.
[0040] Figure 6 This is a sample diagram of the protocol database for this application;
[0041] Figure 7 A flowchart of the complete training scheme;
[0042] Figure 8 This is a diagram of the industrial control software sub-item in the database of this application agreement;
[0043] Figure 9 This is a schematic diagram of the structure of the large-scale interactive environmental model device of this application.
[0044] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0045] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0046] The related technologies lack the ability to expand into different scenarios in environmental interaction, resulting in poor interaction compatibility.
[0047] This application provides a solution: First, import the protocol learning template into the initial model, perform protocol learning on the initial model to generate an interaction model, then modify the interaction model according to the protocol-based decision rules and content output rules to obtain a rule-based interaction model, and finally, perform data query and association between the rule-based interaction model and the protocol database to generate a target model, wherein the protocol database includes environmental interaction protocols.
[0048] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or environmental interaction large-scale model device capable of performing the above functions. The following description uses an environmental interaction large-scale model device as an example to illustrate this embodiment and the subsequent embodiments.
[0049] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0050] This application provides a method for constructing a large-scale environmental interaction model based on protocol learning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the protocol-based learning-based environment interaction large model construction method of this application.
[0051] In this embodiment, the method for constructing a large-scale environment interaction model based on protocol learning includes steps S10 to S30:
[0052] Step S10: Import the protocol learning template into the initial model, perform protocol learning on the initial model, and generate an interactive model.
[0053] In this embodiment, the initial model refers to a model that has undergone basic training on large-scale general data, and its output includes a multimodal base containing pre-trained weight parameters. The protocol learning template refers to a standardized set of constraints extracted from structured protocol data, used to define permitted / prohibited operations and association rules. The interactive model refers to an intelligent interactive model that integrates protocol reasoning capabilities, supporting real-time interaction to generate protocol-compliant output.
[0054] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the protocol learning template is imported into the initial model, and the initial model is trained on the protocol learning of the dialogue instruction task using the test dataset. The protocol learning training weights are obtained through this protocol learning training, and then the protocol learning training weights are imported into the initial model for testing. The test results are verified, and the initial model is updated through the verified protocol learning weights to obtain the interaction model.
[0055] As one method for obtaining test datasets, in the scenario of obtaining test datasets, they can be obtained by modifying open-source datasets according to preset training requirements, or by simulating and synthesizing data through existing large model systems, or by directly inputting and training manually.
[0056] Step S20: Modify the interaction model according to the protocol-based decision rules and content output rules to obtain the rule-based interaction model.
[0057] In this embodiment, the protocol-based decision rule refers to the judgment criteria that the machine can execute through protocol training. Simulated output refers to generating virtual operation results using the trained model to test the effect. Content output rules refer to the specific rule requirements that the model's answers or operations must follow.
[0058] As an optional implementation of protocol-based rule generation, in the scenario of constructing a large-scale environmental interaction model, protocol terms from a protocol learning template are loaded into a pre-trained initial model. A virtual decision-making scenario is built in a reinforcement learning environment, allowing the model to attempt to generate operation plans based on current data and protocol terms. The system automatically checks whether these plans comply with protocol requirements; compliant plans are recorded, while non-compliant plans are deleted. Protocol-based decision rules are generated by associating the operation results generated by each simulation with the corresponding records. Then, by using these simulated operation results and corresponding records, and mixing correct operation cases from real business scenarios, the interaction model is trained to understand the specific boundaries of content output. Finally, by combining adjustable preset rules, content output rules that both comply with protocol terms and solve practical problems are generated.
[0059] As a method for judging the quality of generated results, in the context of constructing a large-scale interactive environmental model, a virtual decision-making scenario is built in a reinforcement learning environment. The model attempts to generate operation plans based on the current data and agreement terms. The system automatically checks whether these plans meet the agreement requirements. Compliant plans will receive reward points, while non-compliant plans will have points deducted and the error type recorded. Then, these simulated operation results and corresponding scoring data are used.
[0060] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the rule conversion logic is configured according to the interaction model. The rule conversion logic converts the protocol-based decision rules and content output rules into code logic that can be directly executed by the machine. The code logic is then adapted to the running logic of the interaction model. Adaptation is performed by judging whether the code logic and the running logic conform to the preset execution logic. If they conform to the preset execution logic, logic fusion is performed. If they do not conform to the preset execution logic, automatic fine-tuning or manual intervention is performed. After logic fusion and adjustment, a rule-based interaction model is generated.
[0061] Step S30: Perform a data query and association between the rule interaction model and the protocol database to generate a target model, wherein the protocol database includes environmental interaction protocols.
[0062] In this embodiment, the protocol database refers to a database that stores business protocol terms and additional associated rules, including environmental interaction protocols, which are protocols involved in interaction with other devices, and interaction protocols other than the protocol learning template. Data query association refers to the process of matching relevant protocol rules from the database based on the current request. The target model refers to a practically executable decision model that integrates protocol rules and business logic.
[0063] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the rule interaction model is connected to the protocol database, the database query protocol is used to obtain the associated constraint rules, the query association between the rule interaction model and the protocol database is established, the association adaptability between the rule interaction model and the protocol database is tested, fine-tuning or manual adjustment is made according to the test adaptation structure, and finally the target model is generated and the target model is deployed.
[0064] As one implementation method for determining compatibility, the system checks whether there are any protocol clauses in the protocol database that conflict with the content output rules. If such clauses exist, the rule interaction model automatically makes minor adjustments to prioritize meeting the requirements of the content output rules. If the minor adjustments fail, manual intervention is notified.
[0065] For example, the pre-training and instruction fine-tuning approach follows the general large model training process, with an additional protocol learning step for model training after instruction fine-tuning. During the deployment phase, a protocol database is introduced to store additional protocol data not visible during model training, such as public or private APIs, locally deployed private models, and environment interaction interfaces for non-AI model types (encapsulation protocols for heterogeneous terminals). The protocol learning template contains the following fields: `<|system|>` identifies the character settings related to the large model; `<|protocol_define|>` identifies the field definitions and explanations of the global protocol; `<|user|>` identifies user input; `<|assistant|>` identifies the generated content of the model; `<|response|>` embeds the response content related to the model's interaction with humans; `<|control|>` identifies the control signals related to the model's interaction with the external environment; `<|feedback|>` identifies the feedback signals received by the model related to interactions with the external environment; `<|extra_protocol|>` identifies the protocol information retrieved from the protocol database, used to standardize the control signal content in the `<|control|>` field; `<|conversation|>` identifies the start and end of a complete multi-turn dialogue; `<|id|>` identifies the temporal relationship of micro-operations, including the following micro-operation types: `<|user|`, `<|response|`, `<|control|`, `<|feedback|>`.
[0066] By learning the adaptive design of the template through the protocol and further expanding the relevant protocol settings by combining the protocol database, the settings visible during model training converge as the training process proceeds. At the same time, the protocols invisible during model training are generalized, achieving a high degree of scalability.
[0067] Based on any of the above embodiments, in Embodiment 2 of this application, referring to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the protocol-learning-based environment interaction large model construction method of this application. Step S10 includes steps A11 to A14:
[0068] Step A11: Based on the initial model, import the protocol learning template, perform protocol learning training for the dialogue instruction task, and generate protocol learning training weights.
[0069] In this embodiment, the dialogue instruction task refers to a task that requires the AI to respond according to rules in a specific scenario. Protocol learning training weights refer to the parameter file generated after training, enabling the model to complete basic interactive tasks.
[0070] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, based on the initial model, the protocol learning template is imported into the initial model and trained on the dialogue instruction task. By inputting different dialogue instructions into the initial model, and then combining the protocol learning template to identify and respond to the relevant dialogue instructions, the protocol learning training weights are generated through the response.
[0071] Step A12: Import the protocol learning training weights into the initial model, perform protocol learning testing, and generate test results.
[0072] In this embodiment, protocol learning testing refers to the process of verifying whether the model conforms to protocol rules using simulated dialogue. The test result is an evaluation report that includes compliance rate indicators.
[0073] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the trained protocol learning weights are imported into the initial model, and the protocol learning is tested through dialogue commands to test the model's compliance with the protocol. The test results are generated based on the compliance rate.
[0074] Step A13: Based on the test results, verify the protocol learning training weights to obtain the verified protocol learning training weights.
[0075] In this embodiment, validation refers to adjusting weight parameters by analyzing erroneous cases to correct the model's misinterpretation of the rules. The validated protocol learning training weights refer to the optimized parameter file, enabling the model to more accurately comply with the protocol rules.
[0076] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the model performs self-fine-tuning based on the compliance rate and protocol conflict items displayed in the test results, and verifies and adjusts the protocol learning training weights to correct the model's misunderstanding of the rules, thus obtaining the verified protocol learning training weights.
[0077] Step A14: Import the verified protocol learning training weights into the initial model, update the pre-training weights of the initial model, and generate the interactive model.
[0078] In this embodiment, updating the pre-trained weights of the initial model means adjusting and updating the pre-trained weights using the validated protocol-learned weights.
[0079] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the optimized protocol learning weights are merged into the initial model, the original weights are gradually replaced by a parameter fusion algorithm, the original parameters are covered by the validated high-sensitivity parameters, and after the update is completed, the updated model is adaptively fine-tuned using an online learning framework to obtain the interaction model.
[0080] For example, refer to Figure 3 , Figure 3 This is a flowchart illustrating the protocol learning process for this application. By constructing protocol learning data templates for the system segment, protocol segment, and dialogue segment, the raw data corpus for training the model can be obtained. The template structure and definition are described above. The corresponding data content will be cleaned, synthesized, and constructed specifically for the vertical domain task, such as by modifying open-source datasets or synthesizing data based on existing large-scale model systems. After obtaining the raw data corpus, it is converted into tokenized data by the large-scale model's tokenizer, and then training can begin. Optional training methods include, but are not limited to, common model training methods such as pretraining, continued pretraining, instruction fine-tuning, and post-training alignment. After training is completed, the results can be tested according to the business scenario or experimental test set.
[0081] By defining the data structure through protocol templates, various fields are explicitly decoupled, allowing the introduction of control and feedback signals for environmental interaction into the structured templates, as well as unstructured natural responses. Supervised training is then used to adapt to the protocol context, thus improving the adaptability of the large model.
[0082] Based on any of the above embodiments, in Embodiment 3 of this application, referring to Figure 4 , Figure 4This is a flowchart illustrating the third embodiment of the protocol-based learning-based environment interaction large model construction method of this application. Before step S20, steps B11 to B14 are also included:
[0083] Step B11: Based on the interaction model and reinforcement learning framework, simulate the logical rules of the business scenario and generate state observations.
[0084] In this embodiment, the reinforcement learning framework refers to an AI training system that allows the model to learn through trial and error in a simulated environment, using a reward / punishment mechanism to optimize decision-making. The logical rules of the business scenario refer to the operational procedures and constraints in a specific business context. State observations refer to the real-time environmental parameters that the model references when making decisions.
[0085] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, a virtual scenario is built based on the interaction model and combined with a reinforcement learning framework. The interaction model is placed in a specific virtual scenario for reinforcement training to learn the logical rules of the business scenario. Based on the logical rules, the interaction model generates state observations for the scenario.
[0086] Step B12: The interaction model generates a feedback decision based on the state observation, and associates the state observation with the feedback decision to generate the protocolized decision rule.
[0087] In this embodiment, feedback decision refers to the response instruction generated based on the current state observation. Protocol-based decision rules refer to solidifying effective decisions into reusable business rules.
[0088] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the interaction model analyzes the state observations, identifies the characters, scenes, and tasks, generates feedback decisions based on the characters, scenes, and tasks, judges the suitability of the feedback decisions, adjusts the feedback decisions based on the judgment results, and finally associates the adjusted feedback decisions with the corresponding state observations to generate protocol-based decision rules.
[0089] Step B13: Based on the interaction model, combined with the protocol-based decision rules and preset conditions, generate content samples that meet the preset conditions, and construct an alignment verification set through a hierarchical random sampling strategy.
[0090] In this embodiment, the output content simulated by the interaction model refers to the dialogue responses or operation suggestions generated by the trained interaction model. Preset conditions refer to predefined compliance standards. Content samples refer to specific instances of the model's output. The stratified random sampling strategy refers to a method of sampling proportionally after grouping by business type or risk level. The alignment validation set refers to the set of test data used to check whether the model output matches the rule requirements.
[0091] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the interaction model processes simulated requests and outputs the corresponding content of the simulated requests according to the protocol-based decision rules. The interaction model automatically filters out content samples that meet the preset conditions based on preset conditions. It classifies business scenarios according to business scenarios through a hierarchical strategy, divides each business scenario into sub-layers according to risk level, and then randomly selects a preset proportion of samples from each sub-layer using a hierarchical sampling tool. Finally, the preset proportion of samples are combined into several aligned verification sets, which include typical compliance cases and edge cases.
[0092] Step B14: Based on the alignment verification set, perform decision tree rule extraction to generate the content output rules.
[0093] In this embodiment, decision tree rule extraction refers to analyzing data using a tree structure to find combinations of conditions that determine whether the output is compliant. Content output rules refer to the explicit requirements that the model's response must meet in terms of format and conditions.
[0094] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, decision tree rules are extracted from compliant and non-compliant samples in the validation set. Each dialogue response is decomposed into structured features, key judgment conditions are found by calculating information gain, and the effectiveness of the rules is tested using samples that did not participate in the training. If a rule is found to be misjudged, the judgment threshold is adjusted by backtracking to the corresponding node of the decision tree. Finally, the optimized rules are compiled into executable content output rules.
[0095] For example, in the scenario of constructing a large-scale environmental interaction model, 500 compliant samples and 200 non-compliant samples from the validation set are input into a decision tree algorithm. Each dialogue response is decomposed into structured features (such as whether it contains a drug name, whether it contains dosage instructions, the location of the disclaimer, etc., totaling 20 features). Information gain is used to calculate key judgment conditions—for example, it is found that "containing a drug name but lacking contraindications" is a common feature of 80% of non-compliant cases. The trained decision tree will generate branching rules such as "if the response contains a drug name → check if a contraindication warning exists → if not, determine non-compliance." These rules are translated into statements that business personnel can understand (e.g., "IF recommended drug THEN must contain the phrase 'Please follow doctor's orders'"). Next, the effectiveness of the rules is tested using 300 samples that were not used in training. If a rule is found to be misjudged (e.g., the reasonable abbreviation "Vitamin C" is misjudged as a violation), the corresponding node in the decision tree is traced back to adjust the judgment threshold (e.g., allowing specific whitelisted words). Finally, the optimized rules are compiled into standard format protocol-based decision rules and linked with the protocol database. For example, in a medical scenario, when the model generates an answer, it automatically triggers three levels of rule verification: "drug name detection → contraindication reminder verification → format compliance check". The rule hit status is visualized through the management backend (e.g., a heatmap is used to show that "missing dosage instructions" is a common problem). Business personnel can add exception clauses with one click. Finally, the optimized output rules are compiled into standard format content output rules.
[0096] By training the interaction model with decision rules and aligning the data, the model can adopt corresponding interaction methods according to specific scenarios, thus improving the interaction capabilities of the large model.
[0097] Based on any of the above embodiments, in Embodiment 4 of this application, step S20 includes steps C11 to C12:
[0098] Step C11: Import the protocol-based decision rules into the interaction model, adjust the decision parameters of the protocol-based decision, and generate the protocol-based interaction model.
[0099] In this embodiment, decision parameter adjustment refers to modifying the internal computational parameters of the model to more strictly match the rules. A protocol-based interaction model refers to a system that highly conforms to the protocol after rule reinforcement training.
[0100] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, based on the interaction model, the protocol-based decision rules are converted into parameter adjustment instructions that the model can understand. These parameter adjustment instructions are then matched and adjusted with the protocol-based decision parameters in the model to make the model comply with the protocol-based decision rules and generate a protocol-based interaction model.
[0101] Step C12: Import the content output rules into the protocol-based interaction model, adjust the content output filtering parameters, and generate the rule interaction model.
[0102] In this embodiment, adjusting the filtering parameters refers to modifying the filtering threshold and inspection intensity when the model outputs. The rule-based interaction model refers to an upgraded intelligent system that integrates protocol rules and content specifications, enabling precise control over the output.
[0103] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, based on a protocol-based interaction model, the content output rules are transformed into adjustable filtering parameters that the model can understand. These adjustable filtering parameters are then matched and adjusted with the filtering parameters and protocol-based decision parameters of the protocol-based interaction model. Prioritizing adjustments that do not violate the content output rules, the various parameters of the protocol-based interaction model are adjusted to generate a rule-based interaction model.
[0104] For example, in the scenario of constructing a large-scale environmental interaction model, protocol-based decision rules (e.g., "medical answers must include dosage instructions") are converted into parameter adjustment instructions that the model can understand. The neural network layer responsible for rule execution (e.g., a drug keyword detection layer) is located within the interaction model, and the weight parameters of these layers are fine-tuned using a gradient descent algorithm. If the model misses dosage instructions in historical data, the update intensity of the corresponding parameter for that rule is increased, essentially giving the robot's "rule memory area" a boost. During the adjustment process, a dual-channel verification is employed: on one hand, the rule hit rate is tested with 3000 simulated data points (e.g., checking if 90% of drug recommendation answers include dosage instructions); on the other hand, the quality of regular dialogue is monitored (e.g., the fluency of ordinary health consultations should not decrease). For complex rules (e.g., financial products need to be matched simultaneously with dosage instructions), further verification is performed. For user risk levels and investment periods, a parameter linkage adjuster was developed to automatically associate multiple rule parameters when the model processes complex conditions (e.g., adjusting the period verification parameter synchronously when the risk level parameter changes). Adjustable screening parameters were designed for content output rules (e.g., "all drug recommendations must include dosage, frequency, and contraindications"). For example, the sensitivity parameter for dosage description detection was adjusted from 0.7 to 0.9 (more stringent). A regular expression matcher was used to scan the model-generated answers in real time (e.g., checking if "once daily" exists in the text). If a missing item was found, the automatic completion module was triggered to insert standard wording. At the same time, a parameter control panel was developed, allowing operators to drag sliders to adjust the execution intensity of different rules (e.g., raising the priority of the "contraindication reminder" rule to the highest level) to complete the adjustment of the rule interaction model.
[0105] By integrating and matching protocol-based decision-making and content output rules, the two rules can be coordinated to eliminate conflicts between them, thereby making the interaction of the interaction model more natural and improving the interaction capability of the large model.
[0106] Based on any of the above embodiments, in Embodiment 5 of this application, step S30 includes steps D11 to D12:
[0107] Step D11: Store the data of the environmental interaction protocol in a database to build an extensible protocol database.
[0108] In this embodiment, the scalable protocol database refers to a protocol database that supports the expansion of more environmental interaction protocols and device interaction protocols.
[0109] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, a protocol database is constructed using additional protocol data, including protocol data beyond the protocol learning template, interaction protocol data of specific devices, and interaction protocol data of specific environments. Users update the protocol database according to their needs.
[0110] Step D12: Based on the rule interaction model, establish an association query table with the protocol database to generate the target model.
[0111] In this embodiment, the association query table refers to the index table that needs to query the protocol rules when defining the model decision.
[0112] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, an associated query table is established based on the protocol database and corresponding to the rule interaction model. Database query statements are automatically generated, relevant rules are retrieved from the protocol database, and a caching mechanism is established for high-frequency query scenarios to preload commonly used protocol rules into memory to obtain the target model.
[0113] For example, in the scenario of constructing a large-scale environmental interaction model, an intelligent query interface for the protocol database is developed in the rule-based interaction model. For instance, when a user requests a prescription, the system automatically generates a database query statement based on the associated query table, retrieves relevant rules from the protocol database (such as the contraindications and maximum dosage limits of the drug), establishes a caching mechanism for high-frequency query scenarios (such as financial product recommendations), preloads commonly used protocol rules into memory, allows the model to call them quickly, compresses response time through query optimization algorithms, and ensures that the model is compliant in real time when processing user requests. The generated target model records the protocol terms associated with each decision and then deploys the target model.
[0114] By enabling efficient migration of solutions for vertical scenarios through simple protocol configuration and the introduction of additional low-cost databases, the construction cost of this large model is reduced without the need for costly manual secondary development and continued training or fine-tuning.
[0115] Based on any of the above embodiments, in Embodiment Six of this application, after step S30, steps E11 to E12 are further included:
[0116] Step E11: Based on the association query table, establish a structured protocol interface between the target model and the protocol database.
[0117] In this embodiment, the structured protocol interface refers to a standard communication channel that enables interaction between the model and database protocol rules based on a unified data format.
[0118] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, a protocol interface framework is integrated into the target model. Dynamic query templates are pre-compiled based on the association query table. Through protocol mapping rules, the original rules returned by the protocol database are converted into mathematical expressions that the target model can parse. A structured protocol interface is defined using a standard interface format to establish a structured protocol interface between the target model and the protocol database.
[0119] Step E12: Based on the target model, the protocol database, the related query table, and the structured protocol interface, perform scenario deployment of the target model.
[0120] In this embodiment, scene deployment refers to fine-tuning the relevant weights of the target model based on the environmental adaptation parameters in the device or scene to complete the adaptive deployment of the target model.
[0121] As an optional implementation, in the scenario of constructing a large-scale interactive environment model, the target model is subjected to adaptability testing based on the environmental parameters of the deployment scenario. Deployment conflicts are detected, and the relevant weights of the deployment conflicts are fine-tuned to ensure that the adaptability of the target model and the deployment scenario meets the preset deployment conditions. Deployment is performed based on the target model and the protocol database. The structured protocol interfaces between the target model and the protocol database are checked according to the association query table. If any discrepancies are found between the structured protocol interfaces and the association query table, they are modified according to the association query table. If all structured protocol interfaces conform to the correspondence in the association query table, the deployment of the target model is completed.
[0122] For example, in the scenario of building a large-scale environmental interaction model, a standardized communication channel is set up in the target model. This channel can quickly query protocols from the protocol database. When the target model needs to make a decision, it first finds the rules to be queried according to the pre-prepared checklist (association query table), and then sends a query request to the protocol database through the set data format (structured protocol interface). The protocol database quickly finds the corresponding rule clauses according to the question, translates the protocol rules into calculation formulas that the model can understand, and sends them back. Then, the entire system is put into a portable Docker container and tested in a simulated environment (e.g., using real data from the past three months) to confirm the stability of the new system. Finally, the target model's system is deployed to the scenario.
[0123] By deploying models adaptively according to different scenarios, the normal operation of the target model can be guaranteed. By detecting the correspondence of the interfaces, the normal interaction of the target model is ensured, and the stability of the large model is improved.
[0124] Based on any of the above embodiments, in Embodiment 7 of this application, after step S40, steps F11 to F15 are further included:
[0125] Step F11: Based on the input message and combined with the target model, generate an instant response oriented towards natural human interaction.
[0126] In this embodiment, input messages refer to interactive requests such as text, voice, or images sent by the user. Natural interaction refers to communication methods that conform to human conversational habits. Instant response refers to real-time feedback content generated within milliseconds.
[0127] As an alternative implementation of input messages, in the application scenario of large-scale environmental interaction models, users can input text and image messages to the target model via touchscreen, or users can input voice messages via speech.
[0128] As an alternative implementation of input messages, in the application scenario of large-scale environmental interaction models, users input messages to the target model through remote communication, or users interact with the target model directly through language and actions.
[0129] As an alternative implementation of input messages, in the application scenario of large-scale environmental interaction models, the target model automatically generates input messages based on the recognition of the surrounding environment.
[0130] Step F12: If environmental interaction is involved, control signals are constructed through protocol-based decision rules.
[0131] In this embodiment, the control signal refers to the physical instructions generated by the system according to rules.
[0132] Step F13 involves parsing the control signal using hard-coded program code, making interactive requests to the associated resources, and obtaining response feedback.
[0133] In this embodiment, hard-coded program code refers to control logic directly written into the program that cannot be dynamically modified. Associated resources refer to the physical or digital devices that require interaction. An interaction request is an action that sends an operation command to the device. Response feedback refers to the status data returned by the device.
[0134] As an optional implementation, in the application scenario of large-scale environmental interaction models, if the target model involves environmental interaction, the surrounding environment is identified through the target model to obtain the surrounding scene information. The control signal is constructed based on the scene information and data through the protocolized decision rules in the target model. The instruction content of the control signal is parsed through hard-coded program code. The interaction with the associated resources is carried out through the instruction content to obtain the response feedback of the interactive resources after the interaction.
[0135] Step F14: The response feedback is fed into the target model to generate an interactive response that yields the interaction result.
[0136] In this embodiment, the interaction result refers to the conclusion generated by the target model based on the feedback data.
[0137] Step F15: Update the content of the immediate response based on the interactive response.
[0138] In this embodiment, updating the content of the real-time response refers to updating the real-time response generated by the input message through the interactive response generated by the environmental interaction in the case of environmental interaction.
[0139] As an optional implementation, in the application scenario of large-scale environmental interaction models, the response feedback generated by environmental interaction is passed to the target model. The target model generates an interactive response including the interaction result based on the response feedback, and updates the content of the timely response triggered by the input message according to the interactive response.
[0140] For example, refer to Figure 5 , Figure 5This is a diagram of the interaction system of the protocol learning model in this application. In the complete interaction steps of the protocol learning model, the predefined content of `<|system|>` and `<|protocol|>` is first defined. Then, the model accepts input from the user, forming an input message through `<|user|>`. The model's response begins with the `<|assistant|>` field, first generating an immediate response wrapped in `<|response|>` for natural human interaction. Subsequently, when interaction with the environment is required, control signals are constructed through the `<|control|>` field. If an operation type not supported by `<|protocol|>` is encountered, the relevant interaction protocol is retrieved from an external protocol database and wrapped in `<|extra_protocol|>`. Then, the control signal content is constructed in the message part of `<|control|>` according to the retrieved external protocol, and the structured signals are parsed through hard-coded program code to request interaction with resources such as associated devices or network environments. After receiving feedback, the feedback message is wrapped in the `<|feedback|>` tag and passed to the model for further iteration to generate a model `<|response|>` message reflecting the interaction result, which is then sent to the human.
[0141] By leveraging the combined effects of protocol-based decision rules and a protocol database, the target model achieves free interaction across any scenario and any device. Furthermore, the scalability of the target model is enhanced through the configuration of the protocol database.
[0142] Based on any of the above embodiments, in Embodiment 8 of this application, after step F12, steps G11 to G12 are further included:
[0143] Step G11: If there is an operation type in the environment interaction that is not in the protocol decision rules, then query the protocol database according to the association query table to obtain the interaction protocol corresponding to the operation type.
[0144] In this embodiment, operation types outside of protocol-based decision rules refer to interactive behaviors not covered by the existing rule base. An interaction protocol refers to the execution steps and constraints defined for a specific operation type.
[0145] As an optional implementation, in the application scenario of large-scale environmental interaction models, if the target model has operation types outside of the protocol-based decision rules during environmental interaction, the operation type is identified, the related query table is queried, and the interaction protocol corresponding to the operation type is found in the protocol database.
[0146] Step G12: Update the control signals constructed through the protocol-based decision rules based on the interaction protocol.
[0147] In this embodiment, "updating" refers to a technical solution that replaces decision rules without interrupting service.
[0148] As an optional implementation, in the application scenario of the large-scale environmental interaction model, the control signals constructed through protocol-based decision rules are updated and supplemented according to the corresponding interaction protocols in the protocol database to obtain complete control signals.
[0149] For example, refer to Figure 6 , Figure 6 This is an example diagram of the protocol database in this application. In personal computing scenarios deeply integrated with artificial intelligence technology, the external environment is specifically a PC terminal, whose control and response can be implemented through the reorganization of underlying source code from various third-party code libraries or service providers. Using this embodiment in conjunction with the aforementioned solution of the present invention, the external database and the protocol structure during training are optional. Generally, common fields include: Unique Identifier: used to identify the sequence number of the protocol. Index Description: used to build a fast index of the protocol content. Protocol Description: used to describe the complete protocol content. Input Format Convention: used to describe the input information transmitted by the model to the external environment. Output Format Convention: used to describe the output information of the interaction with the external environment. Exception Message Convention: used to describe the exception messages of the interaction with the external environment. Special Case Note: The index description and protocol description may share fields to simplify engineering implementation. The implementation of the database is optional, including but not limited to traditional relational databases, non-relational databases (GraphDB, KV database), vector databases, etc. The protocol database in the solution description is optional; it is a supplementary module used to expand and support the compatibility of the protocol learning model in multiple scenarios. The labels for each protocol field in the scheme description are optional. As long as the actual content and meaning referred to by the corresponding label are met, they fall within the scope of this scheme. The pre-trained weights in the scheme description are optional, including but not limited to current open-source and closed-source pre-trained weights, and including but not limited to text modality and multimodal pre-trained weights. In the training phase of the scheme description, the order and application of the remaining learning stages, except for the protocol learning step, are optional. The optimization algorithm is optional, including but not limited to optimization algorithms based on SGD, Adam, etc. The optimization scheme is optional, including but not limited to optimization of the entire model training, optimization based on LoRA (low-rank adaptation), and optimization schemes based on reinforcement learning. The training framework used is optional, including but not limited to training frameworks based on torch, tensorflow, deepspeed, and megatron.
[0150] By leveraging protocol-based decision rules and a protocol database, the target model can freely interact with any scenario and any device, enhancing the model's recognition capabilities and scalability.
[0151] Based on any of the above embodiments, in Embodiment Nine of this application, before step S10, steps H11 to H13 are further included:
[0152] Step H11: Based on the initial model, semantic features are extracted by receiving multimodal raw data, and pre-trained weight parameters containing semantic features are generated.
[0153] In this embodiment, deployment refers to the process of integrating the trained model into the business system for user use. The initial model refers to the basic neural network architecture that has not been trained for a specific task. Multimodal raw data refers to unlabeled mixed data containing various forms such as text, images, and audio. Semantic feature extraction refers to identifying information units with practical meaning from the data. Pre-trained weight parameters refer to the set of mathematical parameters stored after the model is pre-trained, used for subsequent task transfer.
[0154] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, the multimodal raw data is first cleaned. Based on the initial model, the cleaned multimodal raw data is received and semantic features are extracted. The initial model automatically associates and identifies the semantic features to generate pre-trained weight parameters that cover the semantic features.
[0155] Step H12: Based on the pre-trained weight parameters, an attention mechanism is used to map the feature space of different modal data, and the pre-trained weight parameters are optimized and aligned by minimizing the inter-modal difference function to generate target weight parameters.
[0156] In this embodiment, the attention mechanism refers to the technique of enabling the model to dynamically focus on the importance of different data components. Feature space mapping refers to the process of converting data from different modalities into a unified mathematical representation. The intermodal difference function refers to the loss function that measures the degree of feature matching between different modalities. Optimized alignment refers to adjusting parameters to make the feature representations of different modalities tend to be consistent. The target weight parameters refer to the final model parameters that, after optimization, can achieve cross-modal alignment.
[0157] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, pre-trained weight parameters are loaded into a multimodal model, paired image and text data are input, and the association mapping matrix between local image features and text word vectors in the image and text data is calculated through a cross-modal attention layer to generate a feature space mapping. Based on the feature space mapping, the initial model image and text matching is calibrated by minimizing the intermodal difference function, the pre-trained weight parameters are optimized and aligned with the data to generate target weight parameters.
[0158] Step H13: Based on the target weight parameters and combined with the vertical domain labeled data, the parameters of the initial model are updated using a contrastive learning algorithm to generate the initial model.
[0159] In this embodiment, vertically labeled data refers to labeled data specific to a particular business scenario. Contrastive learning algorithms are training methods that optimize the model by comparing the differences between positive and negative samples.
[0160] As an optional implementation, in the scenario of constructing a large-scale environmental interaction model, based on the initial model with imported target weight parameters, and combined with preprocessed vertical domain labeled data, relevant scene recognition training is performed. By constructing positive and negative sample pairs, the initial model is trained using a contrastive learning framework. Based on the matching degree between scene and label data, positive sample pairs are constructed by training the initial model. The matching degree of positive and negative sample pairs is compared. The matching degree of positive sample pairs constructed by training the initial model is higher than that of negative sample pairs. By dynamically adjusting the matching degree of negative sample pairs, the initial model is dynamically trained, the recognition parameters of the initial model are updated, and the initial model is generated.
[0161] For example, refer to Figure 7 , Figure 7 This is a flowchart of the complete training scheme. Pre-training is performed on a multimodal raw dataset (text, images, and sensor time-series data). Feature space mapping is achieved through a cross-modal contrastive learning framework to generate pre-trained weight parameters with basic semantic understanding. For vertical task requirements, supervised training is conducted using instruction fine-tuning technology. The task adaptability of the model is optimized using vertically labeled datasets (such as paired data of industrial equipment operation logs and corresponding control instructions). A protocol learning module is implemented simultaneously, encoding business rule bases (such as safety operation specifications and medical compliance clauses) into structured constraints. These constraints are injected into the model parameter update process through a regularized loss function, introducing high-quality... Align the datasets and use a contrastive learning algorithm to optimize the matching degree between the model output and the protocol rules. During the deployment phase, the fully trained model is dynamically coupled with the protocol database. A real-time rule retrieval interface is developed (based on the database query engine and vector similarity matching) to realize the protocol-based decision flow of environmental interaction requests (environmental state perception → protocol rule matching → control signal generation → execution feedback closed loop). An incremental learning mechanism is built for new interaction scenarios. Uncovered protocol cases are continuously absorbed through an online learning framework. After verification by the digital twin system, the protocol database version is updated to form a large-scale environmental interaction model architecture of "data-driven - rule constraint - dynamic evolution".
[0162] By training with vertically labeled data and multimodal raw data, the model is able to recognize various scenarios and semantic relationships, thus improving the recognition capability of the large model.
[0163] Based on any of the above embodiments, in Embodiment Ten of this application, referring to Figure 8 , Figure 8 This is a diagram of the industrial control software sub-item in the database of this application agreement.
[0164] For example, to refine the control and feedback capabilities of the model system, the external environment is specifically an industrial control system. If the industrial control system simply includes three capabilities: open, close, and run, the industrial control software protocol items will be placed in the protocol database. For "open" and "close", no explicit input content is required, and the output content is a true / false value, where the former indicates successful completion and the latter indicates failure. For abnormal messages, there should be manually hard-coded natural language messages generated by different abnormal situations. For "run", the running parameters should be described and defined according to the actual situation to help the large model organize the corresponding input information into input parameters of the corresponding operation format.
[0165] This application provides an environment interaction large model device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the protocol learning-based environment interaction large model construction method in Embodiment 1 above.
[0166] The following is for reference. Figure 9 This document illustrates a structural diagram of a large-scale interactive environmental model device suitable for implementing embodiments of this application. The large-scale interactive environmental model device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, interactive large-scale devices, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), interactive devices, etc., as well as fixed terminals such as human-computer interaction devices, desktop computers, etc. Figure 9 The large-scale interactive environment device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0167] like Figure 9As shown, the interactive large-scale device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the interactive large-scale device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the environmental interaction large model device to communicate wirelessly or wiredly with other devices to exchange data. Although environmental interaction large model devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.
[0168] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0169] The environmental interaction large-scale model device provided in this application adopts the protocol learning-based environmental interaction large-scale model construction method in the above embodiments, which can solve the technical problem of low interaction compatibility caused by the lack of scene expansion capability in related technologies in environmental interaction scenarios. Compared with the prior art, the beneficial effects of the environmental interaction large-scale model device provided in this application are the same as the beneficial effects of the protocol learning-based environmental interaction large-scale model construction method provided in the above embodiments, and other technical features in this environmental interaction large-scale model device are the same as the features disclosed in the method of the previous embodiment, and will not be repeated here.
[0170] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0171] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0172] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the protocol-based learning-based environment interaction large model construction method in the above embodiments.
[0173] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.
[0174] The aforementioned computer-readable storage medium may be included in the large-scale interactive environment device; or it may exist independently and not assembled into the large-scale interactive environment device.
[0175] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the environment interaction large model device, the environment interaction large model device performs the following actions: based on an initial model and combined with a protocol learning template, it trains a protocol learning model to generate an interaction model; based on the interaction model and combined with a reinforcement learning framework, it generates protocol-based decision rules, and based on the output content simulated by the interaction model, it trains aligned data to generate content output rules; according to the interaction model, combined with the protocol-based decision rules and the content output rules, it generates a rule-based interaction model; according to the rule-based interaction model and combined with a protocol database, it performs data query and association to generate a target model, and executes the deployment of the target model.
[0176] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0178] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0179] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described protocol-based learning-based environment interaction large model construction method. This solves the technical problem of low interaction compatibility in related technologies due to a lack of scene expansion capabilities. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the protocol-based learning-based environment interaction large model construction method provided in the above embodiments, and will not be repeated here.
[0180] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A method for constructing a large-scale environmental interaction model based on protocol learning, characterized in that, The method includes: Import the protocol learning template into the initial model, perform protocol learning on the initial model, and generate an interactive model; Based on the protocol-based decision rules and content output rules, the interaction model is modified to obtain a rule-based interaction model. The rule interaction model is associated with the protocol database through data query to generate the target model, wherein the protocol database includes environmental interaction protocols; The steps of importing the protocol learning template into the initial model, performing protocol learning on the initial model, and generating an interactive model include: Based on the initial model, a protocol learning template is imported to perform protocol learning training for dialogue command tasks, and protocol learning training weights are generated. The protocol learning training weights are imported into the initial model, the protocol learning is tested, and test results are generated. Based on the test results, the protocol learning training weights are verified to obtain the verified protocol learning training weights. The verified protocol learning training weights are imported into the initial model, the pre-training weights of the initial model are updated, and the interactive model is generated. The protocol learning template includes the following fields: <|system|>, <|protocol_define|>, <|user|>, <|assistant|>, <|response|>, <|control|>, <|feedback|>, <|extra_protocol|>, <|conversation|>, and <|id|>. The step of modifying the interaction model to obtain the rule-based interaction model according to the protocol-based decision rules and content output rules includes: Import the protocol-based decision rules into the interaction model, adjust the decision parameters of the protocol-based decision, and generate the protocol-based interaction model; Import the content output rules into the protocol-based interaction model, adjust the content output filtering parameters, and generate the rule interaction model.
2. The method for constructing a large-scale environmental interaction model based on protocol learning as described in claim 1, characterized in that, Before the step of modifying the interaction model according to the protocol-based decision rules and content output rules to obtain the rule-based interaction model, the method further includes: Based on the aforementioned interaction model and reinforcement learning framework, logical rules of business scenarios are simulated to generate state observations. The interaction model generates feedback decisions based on the state observations, and associates the state observations with the feedback decisions to generate the protocol-based decision rules; Based on the interaction model, combined with the protocol-based decision rules and preset conditions, content samples that meet the preset conditions are generated, and an alignment verification set is constructed through a hierarchical random sampling strategy. Based on the alignment validation set, decision tree rules are extracted to generate the content output rules.
3. The method for constructing a large-scale environmental interaction model based on protocol learning as described in claim 1, characterized in that, The step of querying and associating the rule interaction model with the protocol database to generate the target model, wherein the protocol database includes environmental interaction protocols, includes: The data of the environmental interaction protocol is stored in a database to construct an extensible protocol database; Based on the rule-based interaction model, an association query table is established with the protocol database to generate the target model.
4. The method for constructing a large-scale environmental interaction model based on protocol learning as described in claim 1, characterized in that, After the step of querying and associating the rule interaction model with the protocol database to generate the target model, wherein the protocol database includes the environment interaction protocol, the method further includes: Based on the association query table, a structured protocol interface is established between the target model and the protocol database; Based on the target model, the protocol database, the related query table, and the structured protocol interface, the scenario deployment of the target model is executed.
5. The method for constructing a large-scale environmental interaction model based on protocol learning as described in claim 4, characterized in that, After the step of performing scenario deployment of the target model based on the target model, the protocol database, the related query table, and the structured protocol interface, the method further includes: When environmental interaction is involved, control signals are constructed through protocol-based decision rules. The control signals are parsed through hard-coded program code to make interactive requests to the associated resources and obtain response feedback; The response feedback is fed into the target model to generate an interactive response that yields the interaction result; Update the content of the immediate response based on the interaction response.
6. The method for constructing a large-scale environmental interaction model based on protocol learning as described in claim 5, characterized in that, When environmental interaction is involved, after the step of constructing control signals through protocol-based decision rules, the method further includes: If there is an operation type in the environmental interaction that is not in accordance with the protocol decision rules, then the protocol database is queried according to the association query table to obtain the interaction protocol corresponding to the operation type. Based on the aforementioned interaction protocol, the control signals constructed through protocol-based decision rules are updated.
7. A large-scale interactive environmental model device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the protocol-based learning-based environment interaction large model construction method as described in any one of claims 1 to 6.
8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for constructing a large-scale environment interaction model based on protocol learning as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Protocol vulnerability evaluation method and device and storage medium
CN114553533A
Decision model interpretable method based on decision tree
CN117332842A
