Self-evolution method and system of agent based on autonomous decision of large model

Abstract

The application relates to the technical field of agent self-evolution, and discloses an agent self-evolution method and system based on large model autonomous decision, which comprises the following steps: performing semantic association on environment changes and user feedback in task execution through a pre-trained large language model to construct an experience memory bank; judging features of a historical decision path are extracted and weighted evaluation is carried out, if the decision quality is lower than a threshold value, error logic branches are diagnosed and corrected to generate correction rules; a decision strategy network is updated by using reinforcement learning to obtain a target decision model; the model performs situational reasoning on a current task, and analog verification is carried out in combination with the experience memory bank, and a decision result is output; the verified decision result is converted into a new experience memory unit, and a failed decision which does not pass the verification is converted into a negative experience unit together with a failure reason and is stored in the memory bank. The method can realize continuous evolution of autonomous decision-making capability.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent self-evolution technology, and in particular to an intelligent agent self-evolution method and system based on large model autonomous decision-making. Background Technology

[0002] Currently, in the field of autonomous decision-making by intelligent agents, those based on large language models are widely used in task scheduling, automated control, and other scenarios due to their powerful semantic understanding and reasoning capabilities. In smart home environment control scenarios, the agent's decisions must balance user thermal comfort with device energy consumption; balancing these multi-objectives has become a major optimization direction in this field. The agent needs to receive multimodal inputs in a dynamically changing environment, understand user intent, and generate executable decision solutions. The quality of its decisions directly affects task execution efficiency, user experience, and device energy consumption.

[0003] In existing technologies, large language model-driven agents often employ a static decision-making model based on single-inference reasoning. This means that for each task input, contextual understanding and solution generation are performed independently, without retaining the judgment logic and reasoning path from the historical decision-making process. When the same or similar task scenarios reappear, the agent still starts reasoning from scratch, lacking the ability to reuse and reflect on past successes or failures. Although reinforcement learning methods based on experience replay exist in the machine learning field, existing solutions only store state-action-reward triples, failing to incorporate structured information such as the judgment logic branches, environmental semantic relationships, and user feedback text into the experience memory. This prevents the agent from locating specific reasoning errors in erroneous decisions and making targeted corrections, and also makes it difficult to verify new decisions by analogy based on historical experience. Continuous improvement of decision-making capabilities relies on external human intervention or offline retraining.

[0004] In summary, existing technologies lack mechanisms for intelligent agents to accumulate experiential memories and conduct self-evaluation of decision quality and correction of reasoning paths, making it difficult to achieve continuous evolution of autonomous decision-making capabilities and resulting in low experience reuse rates. Summary of the Invention

[0005] This invention provides a self-evolution method and system for intelligent agents based on large-scale autonomous decision-making, so as to achieve continuous evolution of autonomous decision-making capabilities.

[0006] Firstly, to address the aforementioned technical problems, this invention provides a self-evolutionary method for intelligent agents based on large-scale model autonomous decision-making, comprising:

[0007] By semantically associating environmental state change data and user feedback text during task execution using a pre-trained large language model, interaction record data is obtained. Entity relations are extracted and semantically encoded from the interaction record data to construct an experience memory bank.

[0008] Based on the experience memory, the judgment process features of historical decision paths are extracted and weighted to obtain a decision quality assessment value. If the decision quality assessment value is lower than a preset threshold, the erroneous logical branches in the judgment process features are diagnosed and corrected, and correction rules are generated.

[0009] According to the correction rule, reinforcement learning is used to update the parameters of the decision policy network cascaded with the large language model to obtain the target decision model;

[0010] The target decision model is used to perform contextual reasoning on the current task input, and the decision result is obtained by analogy verification in combination with the experience memory bank.

[0011] Obtain the user's subjective evaluation and execution feedback corresponding to the decision result. If the user's subjective evaluation is positive or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank.

[0012] Secondly, the present invention provides an agent self-evolution system based on autonomous decision-making of a large model, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the agent self-evolution method based on autonomous decision-making of a large model as described above.

[0013] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the agent self-evolution method based on large model autonomous decision-making as described above.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] (1) This invention uses a pre-trained large language model to semantically associate environmental state change data and user feedback text during task execution to obtain interaction record data. Entity relations are extracted and semantically encoded from the interaction record data to construct an experience memory bank. The large language model maps physical state descriptions and user natural language feedback to the same vector space and constructs a contextual semantic graph. The self-attention mechanism weights and aggregates the semantic importance of each node in the entity relation graph, so that each record in the experience memory bank not only contains the decision result, but also associates the task context, reasoning path and execution feedback at the time of decision-making. This solves the problem that existing experience playback methods only store state-action-reward triples and lack semantic relation and reasoning path information.

[0016] (2) This invention extracts the judgment process features of historical decision-making paths from the experience memory bank and performs weighted calculations to obtain a decision quality assessment value. If the decision quality assessment value is lower than a preset threshold, it diagnoses and corrects the erroneous logical branches in the judgment process features and generates correction rules. By comparing the current decision with historical experience, it extracts multi-dimensional judgment process features such as temperature regulation range, response delay, and regulation energy consumption and compares them with the ideal decision benchmark, enabling the agent to quantitatively assess its own decision quality. When the assessment is substandard, the judgment process features are reorganized into a feature topology sequence according to the logical hierarchy and compared with the ideal logic tree to accurately locate the erroneous reasoning link that leads to the decision error. This realizes the agent's autonomous identification of its own cognitive bias and correction of the reasoning path, solving the problem in existing solutions where the agent cannot locate the specific reasoning error link from the erroneous decision.

[0017] (3) According to the correction rules, this invention uses reinforcement learning to update the parameters of the decision policy network cascaded with the large language model to obtain the target decision model. The scene elements covered by the triggering conditions in the correction rules are encoded as state features, and the action reward function is constructed together with the correction logic in the correction rules, the adjustment effect after the action is executed, and the energy consumption of the action execution. In the simulation environment, the execution actions corresponding to the state features are scored according to the reward function. After constructing the experience playback matrix, the parameters of the decision policy network are updated using the policy gradient algorithm, so that the correction knowledge in the correction rules is solidified in the decision policy network in the form of network weight increments. This realizes the online adaptive optimization of the agent's decision policy, solves the problem that the improvement of the agent's ability in the traditional scheme depends on offline retraining or manual intervention, and enables the agent to autonomously complete the gradual evolution of decision-making ability in continuous operation.

[0018] (4) This invention uses a target decision model to perform contextual reasoning on the current task input and combines it with an experience memory bank for analogical verification to obtain the decision result; it obtains the user's subjective evaluation and execution feedback corresponding to the decision result. If the user's subjective evaluation is positive feedback or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. The similarity between the candidate decision scheme and the historical decision record is calculated to generate a confidence score. The high-confidence scheme guides the actual execution. The real feedback after execution is verified by sentiment analysis and comparison with the expected target. The decision result that passes the verification is encoded as a positive experience memory unit, and the failed decision along with the reason for the failure is encoded as an experience memory unit with a negative label. Both are stored in the experience memory bank. The experience memory bank is continuously enriched as the task is executed, so that the agent can realize the continuous evolution of decision-making ability in the closed loop of experience accumulation and self-correction. This solves the problem that the lack of experience accumulation and self-correction mechanism of the agent in the prior art leads to slow improvement of decision quality. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the process of the agent self-evolution method based on large model autonomous decision-making provided in the first embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating the changes in decision quality assessment values ​​provided in the first embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the action reward value distribution provided in the first embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of execution feedback verification provided in the first embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Reference Figure 1 The first embodiment of the present invention provides a self-evolutionary method for intelligent agents based on autonomous decision-making of large models, including the following steps:

[0025] S11, Semantically associate the environmental state change data and user feedback text during the task execution process with a pre-trained large language model to obtain interaction record data, extract entity relations and semantically encode the interaction record data, and construct an experience memory bank.

[0026] S12, extract the judgment process features of the historical decision path according to the experience memory bank and perform weighted calculation to obtain the decision quality assessment value. If the decision quality assessment value is lower than the preset threshold, erroneous logical branches in the judgment process features are diagnosed and corrected, and correction rules are generated.

[0027] S13, According to the correction rule, reinforcement learning is used to update the parameters of the decision policy network cascaded with the large language model to obtain the target decision model;

[0028] S14, the current task input is subjected to contextual reasoning through the target decision model, and the decision result is obtained by analogy verification in combination with the experience memory bank;

[0029] S15, obtain the user's subjective evaluation and execution feedback corresponding to the decision result. If the user's subjective evaluation is positive feedback or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive feedback and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank.

[0030] In step S11, the environmental state change data and user feedback text during the task execution process are semantically associated using a pre-trained large language model to obtain interaction record data. Entity relationship extraction and semantic encoding are performed on the interaction record data to construct an experience memory bank.

[0031] In one implementation, the method includes:

[0032] The cosine similarity between the environmental state change data and the user feedback text is calculated to obtain the semantic relevance value.

[0033] If the semantic relevance value is greater than the preset relevance threshold, the environmental state change data and the user feedback text are mapped to the same vector space to obtain the relevance mapping matrix.

[0034] Key nodes are identified based on the association mapping matrix, and edge weights are determined based on the cosine distance between the key nodes to obtain the contextual semantic graph.

[0035] The large language model is used to transform the node and edge weights in the contextual semantic graph into semantic text, thereby obtaining interaction record data.

[0036] In one implementation, the method further includes:

[0037] The interaction record data is parsed using the large language model to obtain multi-source heterogeneous data;

[0038] Entity recognition and relationship extraction are performed on the multi-source heterogeneous data to obtain an entity relationship graph;

[0039] A semantic vector representation is obtained by weighted summation of each node in the entity relationship graph through a self-attention mechanism.

[0040] The semantic vector representation is associated with and stored in relation to the corresponding task context, decision-making process, and execution result to construct an experience memory bank.

[0041] This embodiment uses smart home air conditioner control as the business scenario. The large language model is the backbone model shared by all semantic processing stages in this embodiment. It adopts a decoder-structured transformer network with 32 layers, a hidden layer dimension of 4096, 32 attention heads, a vocabulary size of 65,000, a context window length of 8192 lexical units, and 7 billion parameters. After INT8 quantization and compression, it is deployed on a home edge server. The home smart gateway communicates with the edge server through the home LAN. Lightweight computations such as sentence vector encoding, self-attention encoding, decision strategy network inference, and sentiment analysis are performed locally by the home smart gateway, forming an edge-end collaborative deployment architecture.

[0042] The pre-training process of the large language model is as follows: An open-source Chinese pre-training base model is reused, and pre-training is continued using a corpus of smart home domain language (including device control command records and user dialogue records, with a scale of no less than 10 billion words). An autoregressive language modeling task is adopted, i.e., predicting the next word based on the preceding context, with cross-entropy as the loss function, and the AdamW optimizer is used. The learning rate decays according to a cosine law after warm-up, and the iteration is no less than 10,000 steps. After pre-training, supervised fine-tuning is performed using no less than 100,000 instruction data in the format of environment state description-user feedback-control command triplet, enabling the model to align environmental semantics with control actions. After fine-tuning, the model parameters are solidified for use in each step of this embodiment.

[0043] The sentence vector model is used to map any text as a whole into a fixed-length sentence-level dense vector. It adopts a two-layer transformer encoder structure with an additional mean pooling layer, with approximately 22 million parameters, and outputs a 128-dimensional sentence vector. The encoding process preserves word order and combinatorial semantics, overcoming the problem of losing word order information by word-by-word arithmetic averaging. The pre-training process of the sentence vector model is as follows: it reuses an open-source sentence-level semantic encoding base model, using at least 100 million word units of text merged from general Chinese corpus and smart home domain corpus (including device control command records and user dialogue records) as training data. It uses a contrastive learning task for fine-tuning, that is, different speech transcriptions of the same control command are mutually positive examples, and different control commands are mutually negative examples. It is optimized with a contrastive loss function and Adam optimizer, with a learning rate of 0.0001 and at least 10 iterations. After training, the model parameters are fixed, and a 128-dimensional sentence vector is directly output for any input text, ensuring that any input text can be vectorized. It should be noted that the lightweight sentence vector model is set up independently even when a large language model has been deployed. The reason is that steps S12 and S15 require similarity retrieval and comparison of each record in the experience memory, which is a high-frequency batch calculation. The computational cost of using the lightweight sentence vector encoding method to complete the encoding is much lower than the cost of calling the large language model for reasoning one record at a time, which meets the industry requirements of real-time response for smart home control.

[0044] Environmental condition change data is collected through indoor temperature and humidity sensors, a wearable wristband, and an air conditioner controller. The temperature and humidity sensors have a sampling period of 60 seconds, with a temperature measurement accuracy of ±0.5°C and a relative humidity measurement accuracy of ±3%RH. The wearable wristband has a sampling period of 30 seconds, with a body surface temperature measurement accuracy of ±0.1°C. The user's activity status is determined by the wristband's built-in accelerometer according to preset activity judgment rules, outputting whether the user is sitting or exercising. The fan speed and set temperature are read from the air conditioner controller's operation log. In this embodiment, the collected environmental condition change data shows that the indoor temperature dropped from 26°C to 22°C, the fan speed increased to a high setting, and the user was in a sitting state. The user feedback text is collected through a voice interaction panel. The user's voice is automatically transcribed into text using speech recognition. In this embodiment, two feedback texts were collected: "It's a little cold" and "The wind is giving me a headache."

[0045] Subsequently, the large language model first transcribes the environmental state change data into state description text according to a preset template: the indoor temperature drops from 26°C to 22°C, and the fan speed increases to a high level. Then, the state description text and the user feedback text are encoded as a whole using a sentence vector model, each outputting a 128-dimensional sentence vector, resulting in state feature vectors and feedback feature vectors, respectively. Finally, the cosine similarity between the two vectors is calculated. This is done by summing the products of the two vectors for each dimension as the numerator, multiplying the squares of each dimension by the square root as the denominator, dividing the numerator by the denominator to obtain the cosine similarity value, adding 1 to the cosine similarity value, dividing by 2, and then multiplying by 100 to linearly map it to the range of 0 to 100. The resulting value is the semantic relevance score. In this embodiment, the calculated semantic relevance score is 85.

[0046] The preset association threshold is 75. Under laboratory conditions, at least 100 sample pairs of environmental state change data and user feedback text are collected. Three annotators with smart home engineering experience independently label positive and negative examples based on whether the feedback is caused by the corresponding environmental change. Samples with inconsistent labeling are discarded. For each sample pair, the semantic relevance value is calculated using the above method. A receiver operating characteristic curve is plotted, and the Youden index (true positive rate minus false positive rate) is calculated. The semantic relevance value corresponding to the maximum value of the Youden index is the preset association threshold. The statistical result is 75, corresponding to a true positive rate of 0.88 and a false positive rate of 0.10. Since 85 is greater than 75, this embodiment proceeds to the construction process of the association mapping matrix. Conversely, if the semantic relevance value is less than or equal to 75, it is determined that the feedback has no meaningful association with the current environmental change (e.g., user chat text), the feedback text is discarded, and the next sampling period is returned for re-collection; no interaction record data is established.

[0047] The specific process of mapping to the same vector space is as follows: the large language model performs word segmentation and keyword extraction on the status description text and user feedback text respectively. The keyword extraction rules are to retain nouns and adjectives and verbs expressing user sensations, and to remove stop words. In this embodiment, four keywords are extracted: temperature, wind speed, cold, and headache. The sentence vector model encodes each keyword as a whole, outputting a 128-dimensional node vector. The four node vectors are stacked in rows to form a 4-row, 128-column association mapping matrix. Since both the status description text and the user feedback text are encoded by the same sentence vector model, their node vectors are located in the same vector space, eliminating the semantic gap between physical state data and natural language data.

[0048] Key nodes are the keywords corresponding to each row in the association mapping matrix. Cosine similarity is calculated pairwise for state-related nodes (temperature, wind speed) and sensation-related nodes (cold, headache). The cosine distance equals 1 minus the cosine similarity value. Edge weights are linearly mapped to the range of 0 to 100 according to the similarity level; that is, the edge weight is 100 when the cosine similarity value is 1, 50 when the cosine similarity value is 0, and 0 when the cosine similarity value is −1. In this embodiment, the edge weight between temperature and cold is 90, the edge weight between wind speed and headache is 80, and the edge weight between temperature and headache is 35. An undirected weighted graph is constructed using all key nodes as nodes and the calculated edges and weights, resulting in a contextual semantic graph. This graph intuitively reflects the logical relationships that cause user discomfort due to environmental changes.

[0049] The node list, edges, and edge weights of the contextual semantic graph are filled into the input of the large language model according to the preset prompt template. The prompt template requires the model to output a coherent causal semantic description. The model generates semantic text word by word through autoregressive decoding. In this embodiment, the output interaction record data is: Due to the indoor temperature dropping to 22°C and the wind speed being too high, the user feels cold and has a headache.

[0050] The interactive record data is input into the large language model, which performs word segmentation and part-of-speech tagging, and outputs the parsing results according to the preset parsing rules. The multi-source heterogeneous data obtained in this embodiment includes the numerical feature 22°C, the state feature excessive wind speed, and the descriptive features cold and headache.

[0051] A large language model uses named entity recognition to identify three core entities—temperature, wind speed, and user experience—from multi-source heterogeneous data. Based on the dependency syntax relations in the word segmentation results, the relationships between entities are extracted. The relationship between temperature and 22°C is a state attribute relationship, and the relationship between excessive wind speed and headache is a causal relationship. Using the identified entities as nodes and the extracted relationships as edges, an entity relationship graph containing 4 entity nodes and 2 relationship edges is constructed.

[0052] Each entity node in the entity relationship graph is represented as a 128-dimensional node vector. The query vector, key vector, and value vector for each node are calculated using a projection matrix with three dimensions of 128 rows and 128 columns. For a given node, its attention weight is obtained by scaling the inner product of the node's query vector and the key vectors of all nodes by dividing by the square root of 128, and then normalizing using softmax. All four nodes form a 4x4 attention weight matrix, with the sum of elements in each row being 1. The value vectors of each node are weighted and summed according to the attention weights of their corresponding rows to obtain the output vectors of each node. The arithmetic mean of the output vectors of the four nodes along the node dimensions is then taken to obtain the 128-dimensional semantic vector representation. In this embodiment, the normalized attention weight of the headache node to the wind speed node is 0.42, which is the maximum weight among all weights in that row, indicating that the causal relationship is strengthened during the encoding process. The pre-training process of the projection matrix parameters is as follows: training is performed on a smart home domain corpus using a masked language modeling task, i.e., randomly masking 20% ​​of the input words, recovering the masked words using cross-entropy loss, using the Adam optimizer with a learning rate of 0.001, iterating for no less than 10 rounds, and fixing the parameters after confirming the convergence of the loss through 5-fold cross-validation.

[0053] The experience memory bank is stored using a vector database with a vector nearest neighbor index. Each record contains six fields: a record number field, which is an auto-incrementing integer primary key; a task context field, which records the environmental parameters and user vital signs parameters at the time of decision-making; a decision process field, which records the sequence of control actions and the timestamps of each action; an execution result field, which records the measured environmental changes after the action was executed, the cumulative amount of air conditioning input energy, user evaluation text, feedback labels (positive or negative), and decision quality assessment value; a semantic vector field, which stores a 128-dimensional semantic vector representation, and constructs a hierarchical navigable small-world graph approximate nearest neighbor index based on the semantic vectors of all records. The maximum number of neighbors for each node is 16, and the candidate queue length during retrieval is 64. Subsequent steps retrieve historical records based on semantic similarity through this nearest neighbor index, avoiding uncontrollable retrieval overhead caused by a full table linear scan as the number of records increases; and a timestamp field.

[0054] The maximum record size of the experience memory bank is preset to 10,000 entries. When the limit is reached, records that have been merged, updated, or replaced are prioritized for deletion based on their storage timestamp. In this embodiment, the records stored in this step are as follows: the task context field records the timestamps of the indoor temperature dropping from 26°C to 22°C, the fan being at high speed, the user sitting still, and the change in environmental state; the decision process field records the timestamps of determining that the user is in a heat dissipation state after exercise, triggering strong cooling, setting the temperature to 22°C, setting the fan to high speed, and the control action being issued; the execution result field records the user's feedback of feeling cold and having a headache; and the semantic vector field stores the above 128-dimensional semantic vector representation. Each record stored in the experience memory bank with structured fields is the structured experience data used in subsequent steps.

[0055] In step S12, the judgment process features of historical decision paths are extracted from the experience memory bank and weighted to obtain a decision quality assessment value. If the decision quality assessment value is lower than a preset threshold, the erroneous logical branches in the judgment process features are diagnosed and corrected, and correction rules are generated.

[0056] In one implementation, the method includes:

[0057] The current decision state vector is obtained by parsing the historical decision records in the experience memory bank using the large language model.

[0058] Calculate the cosine similarity between the current decision state vector and the historical decision state vectors corresponding to each historical decision record, and determine the decision paths of historical decision records with cosine similarity greater than a preset similarity threshold as analogous decision paths;

[0059] Extract the judgment process features from the analogous decision path, compare the judgment process features with the preset ideal decision benchmark, and obtain the quality deviation score;

[0060] The decision quality assessment value is obtained by weighting and summing the characteristics of the judgment process based on the quality deviation score.

[0061] In one implementation, the method further includes:

[0062] If the decision quality assessment value is lower than a preset threshold, the features of the judgment process are reorganized according to time sequence and logical hierarchy to obtain a feature topology sequence.

[0063] The feature topology sequence is compared with a preset ideal logic tree to obtain erroneous logic branches that deviate from the standard path;

[0064] The large language model is used to diagnose and correct the erroneous logic branches, generating correction rules.

[0065] First, the large language model reads the task context field of the latest record and generates the current context summary text: the indoor temperature drops from 26°C to 22°C, the wind speed is at a high level, and the user is sitting still. The summary text is encoded using the same keyword extraction and self-attention encoding method as in step S11 (using the same set of projection matrices and sentence vector models) to obtain a 128-dimensional current decision state vector. The historical decision state vectors corresponding to each historical decision record are directly taken from the 128-dimensional semantic vectors stored in the semantic vector field of that record. The two are located in the same encoding space and are comparable.

[0066] Subsequently, the cosine similarity between the current decision state vector and each historical decision state vector is calculated. This cosine similarity is obtained from the approximate nearest neighbor index of the candidate experience memory semantic vector field of the historical decision state vectors, and converted to the 0-100 range using the same linear mapping as in step S11. The preset similarity threshold is 80. For no fewer than 100 record pairs in the experience memory, annotators label positive and negative examples based on whether the initial environments of the two records belong to the same scenario. The similarity is calculated for each record pair, and a receiver operation characteristic curve is plotted. The value corresponding to the maximum Youden index is taken as 80. Only historical decision records with a similarity greater than 80 are eligible to be identified as analogous decision paths. In this embodiment, the record with the highest similarity among those with a similarity greater than 80 is selected as the analogous decision path. This record corresponds to the scenario of a user sitting still and a temperature drop. If there are no historical decision records in the experience memory bank with a similarity greater than the preset similarity threshold (for example, when there are few records in the memory bank in the early stage of system operation), it is determined that there is no comparable historical experience. The quality assessment and metacognitive introspection process in this step are skipped, and there is no need to perform the parameter update in step S13. The initial model consisting of the large language model pre-trained at the time of deployment and the initial decision policy network is directly used to enter step S14 to execute the decision.

[0067] Subsequently, three types of features are extracted from the decision process field and execution result field of the record where the analog decision path is located: temperature adjustment amplitude feature, which is the absolute value of the difference between the set temperature recorded in the decision process field and the initial indoor temperature recorded in the task situation field. In this embodiment, the absolute value of the difference between the set temperature of 22°C and the initial indoor temperature of 26°C is 4°C; response delay time feature, which is the difference between the timestamp of the environmental state change in the task situation field and the timestamp of the control action issuance in the decision process field. In this embodiment, it is 160s; unit adjustment amount energy consumption feature, which is the cumulative amount of air conditioning input power during the execution of this decision recorded in the execution result field divided by the absolute value of the temperature adjustment amplitude. The cumulative amount of air conditioning input power is accumulated by the power metering unit of the air conditioning controller from the time of control action issuance to the time of adjustment end. In this embodiment, the cumulative power is 0.8kW·h divided by the adjustment amplitude of 4°C to obtain 0.2kW·h / °C.

[0068] The preset ideal decision-making benchmark is the ideal value range of each judgment process characteristic. Specifically, the ideal range for thermal comfort is determined according to the PMV-PPD index system specified in GB / T 50785 "Evaluation Standard for Indoor Thermal and Humidity Environment of Civil Buildings" and ASHRAE 55 standard. PMV, or predicted average vote value, is calculated using six parameters: indoor temperature, relative humidity, wind speed, average radiant temperature, human metabolic rate, and clothing thermal resistance, according to the formula given in the standard. PPD, or predicted percentage of dissatisfaction, is a function of PMV. A thermal comfort compliance range is defined as PMV falling between -0.5 and +0.5 and PPD not exceeding 10%. The ideal range for temperature adjustment is 0.5°C to 2°C, determined by applying different temperature conditions to 30 users under laboratory conditions. The temperature adjustment range was determined by screening the corresponding adjustment range under the dual conditions that the adjusted PMV falls within the acceptable range and the user's score on the 5-level thermal comfort scale is not lower than 4 points; the ideal range for response delay is no more than 60 seconds, which was obtained by taking the 90th percentile of the perceptible waiting time from the statistical analysis of 30 users' waiting perception experiments; the ideal range for energy consumption per unit adjustment is no more than 0.12 kWh / °C, which was determined by the 75th percentile of the energy consumption per unit adjustment under the gentle low-speed adjustment condition statistically analyzed by the laboratory energy metering device.

[0069] The comparison rule is as follows: when the feature value falls within the ideal range, the single-term normalization deviation of the feature is 0; when the feature value exceeds the ideal range, the single-term normalization deviation is equal to the excess value divided by the preset full-scale range of the feature and then multiplied by 100, and then cropped to the range of 0 to 100. The full-scale range for temperature regulation amplitude is 10°C, the full-scale range for response delay is 1000s, and the full-scale range for energy consumption per unit regulation is 0.4kW·h / °C. In this embodiment, the amount by which the temperature regulation amplitude of 4°C exceeds the ideal upper limit of 2°C is 2°C, and the single-term normalization deviation is 20; the amount by which the response delay of 160s exceeds 60s is 100s, and the single-term normalization deviation is 10; the amount by which the energy consumption per unit regulation of 0.2kW·h / °C exceeds the ideal upper limit of 0.12kW·h / °C is 0.08kW·h / °C, and the single-term normalization deviation is 20. The value range of the ideal decision benchmark is not statically fixed, but dynamically adjusted with the accumulation of experience: whenever 100 records with positive feedback labels are added to the experience memory bank, the ideal range of the corresponding feature is re-evaluated based on the 25th percentile and 75th percentile of the measured values ​​of each feature in the records of the same scenario, so that the evaluation standard evolves with the accumulation of experience and avoids the self-evolutionary upper limit being locked by the initial prior.

[0070] The preset feature weights are 0.5 for temperature regulation amplitude, 0.3 for response delay time, and 0.2 for energy consumption per unit regulation. The weights are obtained by conducting experiments with different combinations of regulation amplitude, response delay, and energy consumption on 30 users, performing a ternary linear regression with the overall satisfaction score as the dependent variable, and normalizing the standardized regression coefficients of the three features to obtain their weights. The quality deviation score is equal to the weighted sum of the three individual normalized deviations, i.e., the product of 0.5 and 20 plus the product of 0.3 and 10 plus the product of 0.2 and 20, resulting in 17. The score for each feature is equal to 100 minus the individual normalized deviation of that feature. In this embodiment, the score for temperature regulation amplitude is 80, the score for response delay time is 90, and the score for energy consumption per unit regulation is 80. The decision quality assessment value is equal to the weighted sum of the three feature scores according to the preset feature weights, i.e., the product of 0.5 and 80 plus the product of 0.3 and 90 plus the product of 0.2 and 80, resulting in 83.

[0071] The preset threshold is 90. For 50 sets of historical decisions, three experts independently labeled them as either "qualified" or "needs correction" based on whether the decisions required correction. The decision quality assessment value for each set of decisions was calculated using this method, and a receiver operating characteristic curve was plotted. The value corresponding to the maximum point of the Youden index was taken as 90. In this embodiment, if the decision quality assessment value 83 is lower than the preset threshold 90, a metacognitive introspection mechanism is triggered. Conversely, if the decision quality assessment value is greater than or equal to 90, the decision quality is deemed qualified, the metacognitive introspection mechanism is not triggered, and no correction rule is generated. Step S13 is not required, and the process proceeds directly to step S14 to execute the decision. Each calculated decision quality assessment value is written into the decision quality assessment value field of the execution result field of the evaluated record, for use as the original decision quality assessment value during subsequent experience memory fusion and updates.

[0072] The decision process field of the record where the analog decision path is located is read, and the decision links are arranged in a logical hierarchy, from first to last: perception layer, judgment layer, and execution layer. The links within the same level are then arranged in chronological order, and each link is bound to its occurrence timestamp and the feature value of the judgment process involved. The feature topology sequence obtained in this embodiment is as follows: the environmental perception link of the perception layer (the indoor temperature drops from 26°C to 22°C, and the user is sitting still), the user state judgment link of the judgment layer (misjudging the slight drop in body temperature while sitting as a need for heat dissipation after exercise), the scheme generation link of the execution layer (generating a strong wind cooling scheme), and the equipment execution link (setting the temperature to 22°C and the fan to high speed).

[0073] The pre-built ideal logic tree was constructed by experts in the field of thermal comfort based on the control technology of air conditioning equipment. The root of the tree represents the environmental state, and the second layer represents the user activity state determination nodes, which are divided into two branches: sitting and moving. Under the sitting branch, the leaf node corresponding to the decrease in body temperature is gentle heating and the airflow direction avoids people, while under the moving branch, the leaf node corresponding to the increase in body temperature is strong cooling. During comparison, the feature topology sequence is mapped to the corresponding path in the ideal logic tree step by step. In the current scenario, the user is sitting. In the ideal logic tree, the user should take the sitting branch and reach the leaf node of gentle heating and the airflow direction avoids people. However, the feature topology sequence actually enters the moving heat dissipation branch and triggers strong cooling. This deviated branch is the erroneous logic branch. Specifically, it is the branch in the user state determination stage that misjudges the slight decrease in body temperature when sitting as the heat dissipation requirement after moving.

[0074] It should be noted that the ideal logic tree is suitable for control-type task domains where decision logic can be pre-structured, such as air conditioning control in this embodiment. For open task domains where it is difficult to pre-enumerate logic branches, only decision quality assessment and erroneous logic branch diagnosis can be performed, without performing logic tree comparison. The ideal logic tree is not statically fixed, but grows with experience: whenever the number of times a decision corrected according to a certain correction rule receives positive feedback in subsequent executions reaches a preset three times, the triggering condition and correction action of the correction rule are added as new branches to the corresponding node of the ideal logic tree, enabling the agent to discover strategies superior to the initial prior strategy.

[0075] The error logic branch, along with the context (environmental parameters, user vital signs, and timestamps) of the preceding and following steps in the feature topology sequence, is input into the large language model according to a preset prompt template. The model then outputs structured correction rules through autoregressive decoding. The correction rules consist of two parts: trigger conditions and correction actions. In this embodiment, the generated correction rule is as follows: the trigger condition is that the user is in a seated state and their body surface temperature drops by more than 0.1°C within 10 minutes; the correction action is to prioritize gentle heating and adjust the airflow to avoid human movement. The process of obtaining the body surface temperature drop gradient threshold in the trigger condition involves collecting body surface temperature data from 30 seated users for at least 100 hours using a wearable wristband, statistically analyzing the natural cooling gradient in a seated state, and taking the 95th percentile to obtain a decrease of 0.1°C every 10 minutes.

[0076] For example, Figure 2 This is a schematic diagram illustrating the changes in decision quality assessment values. The diagram shows the decision quality assessment values ​​and preset thresholds for 50 historical decision records. Some records have assessment values ​​below 90 points, while the record corresponding to the patent embodiment has an assessment value of 83 points, which meets the triggering conditions of the metacognitive introspection mechanism, allowing for further diagnosis and correction of erroneous logical branches.

[0077] In step S13, according to the correction rule, reinforcement learning is used to update the parameters of the decision policy network cascaded with the large language model to obtain the target decision model. This includes:

[0078] State features are extracted according to the correction rules, and an action reward function is constructed based on the correction logic in the correction rules, the adjustment effect after the action is executed, and the energy consumption of the action execution.

[0079] The execution actions corresponding to the state features are scored according to the action reward function, and the state features, the execution actions and the corresponding reward values ​​are combined to obtain the experience replay matrix;

[0080] Batch data is extracted from the experience replay matrix and input into the decision policy network. The policy gradient algorithm is used to backpropagate the batch data to obtain the gradient update vector.

[0081] The parameters of the decision policy network are updated based on the gradient update vector to obtain the target decision model.

[0082] The dimensions of the state features are determined by the scene elements covered by the triggering conditions of the correction rules. In this embodiment, the state features are a 5-dimensional vector, with each dimension being indoor temperature, relative humidity, user body surface temperature, user activity state, and fan speed. These dimensions are linearly normalized to the range of 0 to 1. Indoor temperature is normalized to 16°C to 30°C, relative humidity to 30%RH to 90%RH, and user body surface temperature to 34°C to 38°C. For user activity states, sitting is assigned 0 and movement is assigned 1. Fan speed is normalized to 0 to 3 for four levels: silent, low speed, medium speed, and high speed, then divided by 3. The values ​​of the state features are extracted from historical interaction records for this scene in the experience memory bank. In this embodiment, the corresponding values ​​are: indoor temperature 22°C, relative humidity 55%RH, user body surface temperature 35°C, user sitting, and fan speed high.

[0083] The action space consists of seven discrete control actions: gentle heating with airflow away from people, strong cooling with direct airflow, independent dehumidification with upward airflow, natural wind mode, maintaining the status quo, only lowering the fan speed, and only adjusting the set temperature. The action reward function is composed of three parts: consistency reward, result reward, and energy consumption reward. The consistency reward is defined as follows: when the executed action is consistent with the corrected action in the correction rule (i.e., gentle heating with airflow away from people), the reward value is 10; when the action prohibited by the correction rule is executed (i.e., strong cooling with direct airflow in a seated scenario), the reward value is -15; the consistency reward value for other actions is 0.

[0084] The reward criteria are as follows: after simulating an action in the simulation environment, if the indoor temperature or relative humidity reaches the expected adjustment target within the expected time limit, an additional reward value of 8 is added; otherwise, no reward is added. This ensures that the reward signal originates from the verifiable execution result in the environment, rather than merely imitating the correction rule. The energy consumption reward criteria are as follows: when the energy consumption per unit adjustment of the simulated action does not exceed the upper limit of the ideal energy consumption range of the ideal decision benchmark (0.12 kW·h / °C), an additional reward value of 5 is added; when it exceeds the upper limit, a reward value of 2 is deducted for every 0.04 kW·h / °C exceeding the upper limit, with a lower limit of -10 for the energy consumption reward. The consistency reward magnitude is determined based on the fact that in the thermal comfort experiment, the negative rating magnitude for incorrect actions by 30 users was 1.5 times the positive rating magnitude for correct actions. Therefore, a positive reward of 10 and a negative reward of 15 are set to suppress the policy network's exploration of prohibited actions. The magnitudes of the result reward and energy consumption reward are set at 0.8 times and 0.5 times the consistency positive reward, respectively.

[0085] State characteristics are input into an air conditioning control simulation environment built based on a Markov decision process. In the simulation environment, indoor temperature and relative humidity change at a preset rate according to the executed actions. For example, under heating action, the indoor temperature increases by 0.5°C every 10 minutes, and under dehumidification action, the relative humidity decreases by 5 percentage points every 10 minutes. The state transition rate is obtained by fitting the nominal performance parameters of the air conditioning equipment with no less than 30 sets of measured cooling and dehumidification curves. The simulation environment also calculates the unit adjustment energy consumption of each simulated action according to the nominal performance parameters of the equipment, which serves as the scoring basis for energy consumption rewards. Each simulated action is scored according to the action reward function in the simulation environment, and 256 sets of interaction data are collected. Each set of interaction data consists of the current state characteristics, the executed action, the reward value, and the state characteristics after the transition. These are concatenated in rows to form a 256-row, 4-column experience playback matrix, where the first column is the number of the current state characteristics, the second column is the number of the executed action, the third column is the reward value, and the fourth column is the number of the state characteristics after the transition. The correspondence between state characteristics and numbers is stored in the buffer of the simulation environment.

[0086] The decision policy network is a three-layer fully connected neural network independent of the large language model, cascaded with the large language model. The input layer consists of 5-dimensional state features, and each of the two hidden layers contains 64 neurons activated by ReLU. The output layer is 7-dimensional and normalized by softmax to represent the execution probabilities of each control action. The pre-training process of the decision policy network involves using historical records of positive user feedback in the experience memory's execution result field as supervisory samples before agent deployment. The state features corresponding to the recorded task context are used as input, and the execution actions recorded in the decision process field are used as labels. A cross-entropy loss function and the Adam optimizer are employed, with a learning rate of 0.001. At least 1000 iterations are performed, and 5-fold cross-validation is used. Early stopping is triggered when the validation loss does not decrease for 10 consecutive iterations, and the network parameters are solidified as the initial decision policy network.

[0087] Among them, the historical records in the experience memory are accumulated during the trial run phase before the agent is deployed. During the trial run phase, the large language model that has completed supervised fine-tuning generates control actions for the task input and issues them for execution. The task context, decision-making process and execution feedback generated during the trial run are written into the experience memory according to the same field structure as in step S11, so that the experience memory already contains historical records that can be used for pre-training when the agent is deployed.

[0088] In this embodiment, the ontology parameters of the large language model are kept frozen, and the self-evolution of the agent is achieved through two channels: one is the accumulation of contextual assets such as experience memory bank, correction rules and supplementary branches of ideal logic tree, and the other is the parameter update of decision policy network.

[0089] The policy gradient algorithm weights the logarithmic execution probability of each action with the reward value, calculates the gradient of the network parameters using the weighted objective function, and obtains a gradient update vector with the same dimension as all weight parameters of the network through backpropagation. This vector indicates the direction and magnitude of parameter adjustment. The L2 norm of the gradient update vector is calculated and compared with a preset gradient threshold of 3. The preset gradient threshold of 3 is obtained by statistically analyzing the distribution of the L2 norm of the gradient update vector in each iteration during the pre-training phase of the decision policy network, taking twice the mean of the L2 norm in each iteration after the loss enters the plateau period, and rounding up to obtain 3.

[0090] If the L2 norm of the gradient update vector is greater than the preset gradient threshold of 3, it is determined that the decision policy network still has significant optimization potential. The weight matrix is ​​updated along the direction indicated by the gradient update vector with a learning rate of 0.001, and then a new batch of data is extracted for the next iteration. If the L2 norm is less than or equal to 3, the decision policy network is considered converged, and parameter updates are stopped. Iteration termination uses a dual-condition determination, with the first condition being satisfied. Firstly, the maximum number of iterations is set to 500. Secondly, on at least 100 verification states covering 7 typical action scenarios collected separately from the simulation environment, the KL divergence of the action probability distributions output in adjacent iterations is calculated. Iteration stops when the divergence is less than 0.01 or the number of iterations reaches 500. After parameter updates, the decision policy network is cascaded with a large language model to form a target decision model. The large language model is responsible for semantic understanding of the input text and outputting state features, while the decision policy network is responsible for outputting the execution probability of control actions in the action space.

[0091] For example, Figure 3 This is a schematic diagram of the action reward value distribution. The diagram shows the action reward values ​​in the experience playback matrix for gentle heating with wind direction avoiding people and strong cooling with wind direction blowing directly. Correcting actions yields stable positive rewards, while correcting actions prohibited by the rules yields negative rewards. This demonstrates the guiding role of the action reward function, jointly constructed by correction logic, adjustment effect, and energy consumption, in updating the decision-making strategy network parameters.

[0092] In step S14, the current task input is subjected to contextual reasoning using the target decision model, and analogical verification is performed using the experience memory bank to obtain the decision result. This includes:

[0093] The target decision model performs contextual reasoning on the current task input to generate a candidate decision set, which includes at least two candidate decision schemes.

[0094] Historical decision records are retrieved from the experience memory bank, and the similarity between each candidate decision scheme in the candidate decision set and the historical decision records is calculated to obtain the confidence score of each candidate decision scheme.

[0095] If the maximum value among the confidence scores is greater than a preset threshold, then the candidate decision scheme corresponding to the maximum value is determined as the decision result.

[0096] The current task input is obtained, which in this embodiment includes an indoor temperature of 28°C, a relative humidity of 75%RH (collected by a temperature and humidity sensor), a user's body surface temperature of 36°C, and the user being seated (collected by a wearable wristband). The target decision model performs contextual reasoning on the current task input, and the large language model performs semantic understanding of the current task input, outputting 5-dimensional state features according to the same normalization rules as step S13. The decision strategy network outputs 7-dimensional action execution probabilities from the state features, and the top 3 control actions with the highest execution probabilities are selected as candidate actions. In this embodiment, these are independent dehumidification with upward airflow, strong wind cooling with direct airflow, and natural wind mode. The large language model then generates specific control parameters for each candidate action, with parameter values ​​limited to the range supported by the device, forming a candidate decision set containing 3 candidate decision schemes: Scheme 1 is strong wind cooling with a set temperature of 24°C and maximum fan speed; Scheme 2 is independent dehumidification mode with a set temperature of 25°C and upward airflow; and Scheme 3 is natural wind mode. When there are fewer than two non-zero execution probabilities in the action space, actions are selected in descending order of execution probability until the number of candidate decision schemes reaches two, ensuring that the candidate decision set always includes at least two candidate decision schemes.

[0097] To enable similarity calculations between candidate decision schemes and historical decision records, each candidate decision scheme is parsed into a 4-dimensional action parameter vector. Each dimension is linearly normalized to the range of 0 to 1. The first dimension is the set temperature, normalized to 16°C to 30°C. Actions in the natural wind mode and the "maintain the status quo" category that do not include a set temperature are normalized using the current set temperature. The second dimension is the operating mode code: 0 for maintaining the status quo, 1 for heating, 2 for cooling, 3 for dehumidifying, and 4 for natural wind, then normalized by dividing by 4. The third dimension is the fan speed level, which is normalized by dividing by 3 for the four levels: silent, low, medium, and high. The fourth dimension is the wind direction code: 0 for avoiding people, 1 for upward, 2 for direct blowing, and 3 for oscillation, then normalized by dividing by 3.

[0098] Historical decision records are retrieved from the experience memory bank. A task summary text is generated from the current task input by the large language model. The current state vector is obtained using the same self-attention encoding method as step S11. Using this vector as the query vector, the hierarchical navigable small-world graph approximate nearest neighbor index of the semantic vector field of the experience memory bank is used to retrieve historical decision records with a similarity greater than a preset similarity threshold of 80. The maximum number of retrieved records is preset to 50. In this embodiment, 50 historical decision records under high temperature and high humidity scenarios in the past 30 days are obtained, of which 46 records are records with positive feedback labels and 4 records are failure records with negative labels. The decision actions of each record are also vectorized according to the above 4-dimensional rules.

[0099] The cosine similarity between the 4D vector of each candidate decision scheme and the 50 historical decision vectors in the candidate decision set is calculated, forming a 3x50 similarity matrix. For each row of the similarity matrix, a weighted average is calculated using the similarity during the corresponding historical record retrieval as the weight. The resulting mean is then linearly converted to the interval 0 to 100 by adding 1, dividing by 2, and multiplying by 100, yielding the initial confidence score for each candidate decision scheme. The confidence score is then determined according to the negative example correction rule. If the similarity between a candidate decision scheme and a failed record with a negative label is greater than a preset similarity threshold of 80, the initial confidence score of the candidate decision scheme is multiplied by a preset negative example discount factor of 0.5 to obtain the confidence score of the candidate decision scheme. Otherwise, the initial confidence score is the final confidence score. In this embodiment, the maximum similarities between Scheme 1, Scheme 2, and Scheme 3 and each failed record are 72, 65, and 58, respectively, all of which do not exceed the threshold of 80 and do not trigger discount correction. In this embodiment, the confidence score for Scheme 1 is 65, the confidence score for Scheme 2 is 92, and the confidence score for Scheme 3 is 45.

[0100] The preset threshold is 80. By using historical decisions that received positive feedback after execution in the experience memory as positive examples and those that received negative feedback as negative examples, the confidence score distribution of the candidate solutions corresponding to the positive and negative examples is statistically analyzed, and a receiver operating characteristic curve is plotted. The value corresponding to the maximum Youden index is taken as 80. The maximum confidence score is 92, which is greater than the preset threshold of 80. Solution 2 (independent dehumidification mode, set temperature 25°C, upward airflow) corresponding to the maximum confidence score is determined as the decision result, and an execution command is issued. Conversely, if the maximum confidence score is less than or equal to the preset threshold of 80, the candidate decision solution corresponding to the maximum confidence score is still determined as the decision result, and a low-confidence marker is added to the decision process field when this decision result is subsequently stored in the experience memory.

[0101] In step S15, the user's subjective evaluation and execution feedback corresponding to the decision result are obtained. If the user's subjective evaluation is positive or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank. This includes:

[0102] Obtain the user's subjective evaluation and execution feedback corresponding to the decision result, perform sentiment analysis on the user's subjective evaluation to obtain a sentiment polarity score, and if the sentiment polarity score is greater than a preset positive judgment threshold, then the user's subjective evaluation is determined to be positive feedback.

[0103] The execution feedback is compared with the expected adjustment target. If the execution feedback reaches the expected adjustment target, the execution feedback is deemed to have been successfully verified.

[0104] If the user's subjective evaluation is positive feedback or the execution feedback is successfully verified, the decision result, the task context at the time of decision-making, and the execution feedback are encapsulated into a new experience memory unit and stored in the experience memory bank.

[0105] If the user's subjective evaluation is not positive feedback and the execution feedback verification fails, the decision result, the task context at the time of decision-making, and the reason for failure are encapsulated into an experience memory unit with a negative label and stored in the experience memory bank for use as a negative example in subsequent analogy verification.

[0106] After the decision is issued and executed, the air conditioner controller transmits operating status data every 60 seconds. The voice interaction panel collects user evaluation text. In this embodiment, the user's subjective evaluation text is "The current temperature and humidity are just right". At the same time, the execution feedback is that the indoor relative humidity has dropped from 75%RH to 60%RH and the user's body surface temperature has stabilized at 36°C.

[0107] Sentiment analysis employs a pre-trained sentiment analysis model. The model's structure is as follows: input is a 128-dimensional sentence vector output from the evaluation text encoded by a sentence vector model. After passing through a hidden layer (ReLU activation) containing 64 neurons, a 2D output layer is normalized using softmax to output two probabilities: positive and negative. The sentiment polarity score is equal to the positive probability multiplied by 100, with a value ranging from 0 to 100. The pre-training process of the sentiment analysis model involves using at least 5000 user reviews from smart home customer service and voice assistant scenarios as training data. Three annotators independently and manually annotate the reviews, separating positive and negative entries. Inconsistent annotations are removed. The model uses a cross-entropy loss function and the Adam optimizer with a learning rate of 0.001, undergoing at least 500 iterations and employing 5-fold cross-validation. The test set classification accuracy is at least 92%. After training, the model parameters are fixed. In this embodiment, the positive probability of the evaluation text is 0.85, and the sentiment polarity score is 85.

[0108] The preset positive judgment threshold is 60. Emotional polarity scores are calculated for at least 200 comments labeled with positive and negative tags, and a receiver operating characteristic curve is plotted. The value corresponding to the maximum point of the Youden index (60) is taken, corresponding to a true positive rate of 0.90 and a false positive rate of 0.12. In this embodiment, an emotional polarity score of 85 is greater than 60, and the user's subjective evaluation is judged as positive feedback.

[0109] The expected adjustment target is generated by the large language model based on the parameters of the candidate decision schemes and stored in the operation log when the decision results are issued. In this embodiment, the expected adjustment target is to reduce the indoor relative humidity from 75%RH to below 65%RH within 30 minutes, that is, a reduction of no less than 10 percentage points. After the deadline for achieving the expected adjustment target expires, the execution feedback at that moment is read and compared. The judgment rule is that if the measured change is not less than the expected change, the execution feedback is judged to have achieved the expected adjustment target and the verification is successful; if the measured change is less than the expected change, the expected adjustment target is judged to have not been achieved. In this embodiment, the measured relative humidity decreased by 15 percentage points, which is not less than the expected decrease of 10 percentage points, and the user's body surface temperature stabilized at 36°C, so the execution feedback was judged to have been verified successfully.

[0110] Since the storage condition is met if either the user's subjective evaluation is positive feedback or the execution feedback is successfully verified, both conditions are met in this embodiment. The decision result, the task context at the time of decision-making, and the execution feedback are encapsulated into a new experience memory unit. The task context is an indoor temperature of 28°C, a relative humidity of 75%RH, a user's body surface temperature of 36°C, and the user is sitting still. The decision-making process is an independent dehumidification mode, a set temperature of 25°C, an upward airflow direction, and a control action issuance timestamp. The execution feedback is a 15 percentage point decrease in relative humidity, a stable user body surface temperature of 36°C, and an emotion polarity score of 85. At the same time, a summary text is generated for the current task context by a large language model, and a 128-dimensional semantic vector is obtained using the same self-attention encoding method as in step S11. This vector is encapsulated together with the self-incrementing record number and the storage timestamp. The encapsulated field structure is completely consistent with the field structure of the experience memory bank.

[0111] Conversely, if the emotional polarity score is less than or equal to the preset positive judgment threshold, and the execution feedback does not achieve the expected adjustment target, the decision result is judged as a failure sample. The decision result, the task context at the time of decision, the execution feedback, and the reasons for failure (including user subjective evaluation text and execution feedback data with insufficient measured change) are encapsulated into an experience memory unit with a negative label and stored in the experience memory bank. This unit is used as a negative example to participate in the confidence score correction of the candidate decision scheme during the analogy verification in step S14. At the same time, the decision result and the reasons for failure are written into the operation log, and the process is returned to step S12 to re-execute the quality assessment and metacognitive self-reflection of this decision, forming a dual-channel error correction loop of parallel failure memory storage and parameter update.

[0112] The operation of returning to step S12 has a preset retry limit of 3 times for the same decision task. After the failure feedback of the same task triggers the return to step S12 3 times, the operation of returning to step S12 will no longer be performed for the task. Subsequent failure decisions are still encapsulated as experience memory units with negative labels and stored in the experience memory bank and participate in the negative example correction during the analogy verification in step S14. At the same time, a mark for manual verification is recorded in the operation log of the task.

[0113] Upon storage, the large language model is triggered to update the experience memory bank: First, the large language model compares the decision process field and execution result field of the new and old records. If the two are semantically consistent and only differ in numerical value, then historical records with a semantic vector similarity greater than a preset fusion threshold of 90 are retrieved through an approximate nearest neighbor index and determined to be of the same type of decision event. The decision quality evaluation value of the corresponding historical record is then updated by weighting it according to the adaptive fusion weight. The evaluation value corresponding to the new evidence is determined by taking the sentiment polarity score for positive feedback and subtracting the sentiment polarity score from 100 for negative feedback. In this embodiment, it is positive feedback, so 85 is used. The adaptive fusion weight is determined by the confidence score of the new experience memory unit. When the confidence score is greater than 90, 0.4 is used. When the score is 80 and not greater than 90, the weight is 0.3; when it is not greater than 80, the weight is 0.2. In this embodiment, a confidence score of 92 corresponds to a fusion weight of 0.4. When the original decision quality assessment value is 80, it is updated to the product of 0.6 and 80 plus the product of 0.4 and 85, which is 82. If the two have semantic conflicts (e.g., the control actions of the old and new records are opposite), numerical fusion is not performed. The new experience memory unit is inserted into the experience memory bank as a new record, and the old record is marked as replaced. The replaced record no longer participates in subsequent analogy verification and confidence score calculation. The integration of experience memory is decided by the large language model based on the semantic comparison results rather than fixed numerical rules. If the maximum similarity found is less than or equal to 90, the new experience memory unit is inserted into the experience memory bank as a new record. The preset fusion threshold 90 is obtained by taking the 95th percentile of the distribution of similar decision events in the experience memory bank. The agent forms a closed loop of execution, feedback, reflection, and evolution in the continuous task execution, and the decision-making ability is continuously improved with the accumulation of the experience memory bank.

[0114] For example, Figure 4 This is a schematic diagram of the execution feedback verification. The diagram shows the change in indoor relative humidity in independent dehumidification mode over time after the control action is issued. The indoor relative humidity continuously decreases from approximately 75%RH, and falls below the expected adjustment target of 65%RH at about 20 minutes. By the end of 30 minutes, it has dropped to approximately 59.3%RH, indicating that the execution feedback has achieved the expected adjustment target and has been successfully verified.

[0115] In summary, this invention discloses a self-evolutionary method for intelligent agents based on large-scale model autonomous decision-making, comprising: semantically associating environmental state change data and user feedback text during task execution using a pre-trained large language model to obtain interaction record data; extracting entity relationships and semantically encoding the interaction record data to construct an experience memory bank; extracting judgment process features of historical decision paths from the experience memory bank and performing weighted calculations to obtain a decision quality assessment value; if the decision quality assessment value is lower than a preset threshold, diagnosing and correcting erroneous logical branches in the judgment process features to generate correction rules; and applying reinforcement learning to the interaction process based on the correction rules. The decision strategy network cascaded by the large language model is updated with parameters to obtain a target decision model. The target decision model is then used to perform contextual reasoning on the current task input, and analogical verification is performed using the experience memory bank to obtain a decision result. The user's subjective evaluation and execution feedback corresponding to the decision result are obtained. If the user's subjective evaluation is positive or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank. This invention achieves continuous evolution of autonomous decision-making capabilities through agent self-evolution.

[0116] The second embodiment of the present invention provides an agent self-evolution system based on autonomous decision-making of a large model, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the agent self-evolution method based on autonomous decision-making of a large model as described above.

[0117] It should be noted that the agent self-evolution system based on large model autonomous decision-making provided in this embodiment of the invention is used to execute all the process steps of the agent self-evolution method based on large model autonomous decision-making in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0118] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A self-evolutionary method for intelligent agents based on autonomous decision-making using a large model, characterized in that, include: By semantically associating environmental state change data and user feedback text during task execution using a pre-trained large language model, interaction record data is obtained. Entity relations are extracted and semantically encoded from the interaction record data to construct an experience memory bank. Based on the experience memory, the judgment process features of historical decision paths are extracted and weighted to obtain a decision quality assessment value. If the decision quality assessment value is lower than a preset threshold, the erroneous logical branches in the judgment process features are diagnosed and corrected, and correction rules are generated. According to the correction rule, reinforcement learning is used to update the parameters of the decision policy network cascaded with the large language model to obtain the target decision model; The target decision model is used to perform contextual reasoning on the current task input, and the decision result is obtained by analogy verification in combination with the experience memory bank. Obtain the user's subjective evaluation and execution feedback corresponding to the decision result. If the user's subjective evaluation is positive or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank.

2. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The interaction record data is obtained by semantically associating environmental state change data and user feedback text during task execution using a pre-trained large language model, including: The cosine similarity between the environmental state change data and the user feedback text is calculated to obtain the semantic relevance value. If the semantic relevance value is greater than the preset relevance threshold, the environmental state change data and the user feedback text are mapped to the same vector space to obtain the relevance mapping matrix. Key nodes are identified based on the association mapping matrix, and edge weights are determined based on the cosine distance between the key nodes to obtain the contextual semantic graph. The large language model is used to transform the node and edge weights in the contextual semantic graph into semantic text, thereby obtaining interaction record data.

3. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The step of extracting entity relationships and semantically encoding the interaction record data to construct an experience memory database includes: The interaction record data is parsed using the large language model to obtain multi-source heterogeneous data; Entity recognition and relationship extraction are performed on the multi-source heterogeneous data to obtain an entity relationship graph; A semantic vector representation is obtained by weighted summation of each node in the entity relationship graph through a self-attention mechanism. The semantic vector representation is associated with and stored in relation to the corresponding task context, decision-making process, and execution result to construct an experience memory bank.

4. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The step of extracting the judgment process features of historical decision-making paths from the experience memory and performing weighted calculations to obtain a decision quality assessment value includes: The current decision state vector is obtained by parsing the historical decision records in the experience memory bank using the large language model. Calculate the cosine similarity between the current decision state vector and the historical decision state vectors corresponding to each historical decision record, and determine the decision paths of historical decision records with cosine similarity greater than a preset similarity threshold as analogous decision paths; Extract the judgment process features from the analogous decision path, compare the judgment process features with the preset ideal decision benchmark, and obtain the quality deviation score; The decision quality assessment value is obtained by weighting and summing the characteristics of the judgment process based on the quality deviation score.

5. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, If the decision quality assessment value is lower than a preset threshold, then erroneous logical branches in the judgment process characteristics are diagnosed and corrected, and correction rules are generated, including: If the decision quality assessment value is lower than a preset threshold, the features of the judgment process are reorganized according to time sequence and logical hierarchy to obtain a feature topology sequence. The feature topology sequence is compared with a preset ideal logic tree to obtain erroneous logic branches that deviate from the standard path; The large language model is used to diagnose and correct the erroneous logic branches, generating correction rules.

6. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The step of updating the parameters of the decision policy network cascaded with the large language model using reinforcement learning according to the correction rule to obtain the target decision model includes: State features are extracted according to the correction rules, and an action reward function is constructed based on the correction logic in the correction rules, the adjustment effect after the action is executed, and the energy consumption of the action execution. The execution actions corresponding to the state features are scored according to the action reward function, and the state features, the execution actions and the corresponding reward values ​​are combined to obtain the experience replay matrix; Batch data is extracted from the experience replay matrix and input into the decision policy network. The policy gradient algorithm is used to backpropagate the batch data to obtain the gradient update vector. The parameters of the decision policy network are updated based on the gradient update vector to obtain the target decision model.

7. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The step of performing contextual reasoning on the current task input using the target decision model and conducting analogical verification using the experience memory base to obtain the decision result includes: The target decision model performs contextual reasoning on the current task input to generate a candidate decision set, which includes at least two candidate decision schemes. Historical decision records are retrieved from the experience memory bank, and the similarity between each candidate decision scheme in the candidate decision set and the historical decision records is calculated to obtain the confidence score of each candidate decision scheme. If the maximum value among the confidence scores is greater than a preset threshold, then the candidate decision scheme corresponding to the maximum value is determined as the decision result.

8. The agent self-evolution method based on large model autonomous decision-making according to claim 1, characterized in that, The process of obtaining the user's subjective evaluation and execution feedback corresponding to the decision result, wherein if the user's subjective evaluation is positive or the execution feedback is successfully verified, the decision result is converted into a new experience memory unit and stored in the experience memory bank; if the user's subjective evaluation is not positive and the execution feedback fails to verify, the decision result and its failure reason are converted into an experience memory unit with a negative label and stored in the experience memory bank, including: Obtain the user's subjective evaluation and execution feedback corresponding to the decision result, perform sentiment analysis on the user's subjective evaluation to obtain a sentiment polarity score, and if the sentiment polarity score is greater than a preset positive judgment threshold, then the user's subjective evaluation is determined to be positive feedback. The execution feedback is compared with the expected adjustment target. If the execution feedback reaches the expected adjustment target, the execution feedback is deemed to have been successfully verified. If the user's subjective evaluation is positive feedback or the execution feedback is successfully verified, the decision result, the task context at the time of decision-making, and the execution feedback are encapsulated into a new experience memory unit and stored in the experience memory bank. If the user's subjective evaluation is not positive feedback and the execution feedback verification fails, the decision result, the task context at the time of decision-making, and the reason for failure are encapsulated into an experience memory unit with a negative label and stored in the experience memory bank for use as a negative example in subsequent analogy verification.

9. A self-evolving intelligent agent system based on large-scale model autonomous decision-making, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the agent self-evolution method based on large model autonomous decision-making as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the agent self-evolution method based on large model autonomous decision-making as described in any one of claims 1 to 8.

Images