Online incremental learning AI chat robot response generation system

Abstract

The invention relates to the technical field of artificial intelligence and natural language processing, in particular to an online incremental learning AI chat robot response generation system which comprises a cache reweighting module, a semantic offset calculation module, an adjusting module, an updating module, a response generation module and a feedback module. According to the method, cache reweighting based on semantic similarity and time decay, a self-adaptive learning rate based on semantic offset and rollback protection are introduced, low-rank increment updating of LoRA and a weight adjustment closed loop of multi-target weighted decoding and feedback driving are adopted, so that the system can effectively improve the real-time performance of the system while preferentially utilizing aging and related historical contexts, and the real-time performance of the system is improved. And the updating strength is dynamically controlled, and stable parameters are recovered during abnormal offset, so that the problem that the response reliability is reduced due to model updating delay or parameter offset during high-frequency interaction or semantic drift caused by conflict between speed and stability in online incremental updating is effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to an online incremental learning AI chatbot response generation system. Background Technology

[0002] With the rapid development of artificial intelligence technology and the widespread application of natural language processing models in various applications, AI research workbench has become an important platform for researchers to conduct experimental design, data analysis, and model training. However, in practical use, faced with massive amounts of multi-turn dialogue data and constantly changing user needs, traditional chatbots often struggle to retain historical knowledge while achieving real-time learning and response optimization. This poses a significant challenge to the system's intelligence, personalization, and improvement of user satisfaction.

[0003] Chinese Patent Application Publication No. CN116955553A discloses a system and method for an active chatbot with behavior perception and on-demand dialogue. The system includes: an artificial intelligence platform for receiving precise question information via an application programming interface (API), inputting the precise question information into a large language model to generate answer information, and then transmitting the answer information via the API; a client host comprising: at least one sensor for continuously sensing at least one of physiological state, facial expression, and body movement to generate a user behavior state; a first non-transitory computer-readable storage medium for storing a plurality of first computer-readable instructions; and a first hardware processor electrically connected to the first non-transitory computer-readable storage medium and the sensor, for executing the plurality of first computer-readable instructions to cause the client host to continuously transmit the user behavior state and a plurality of on-demand dialogue settings, wherein the on-demand dialogue settings include time information and filtering parameters; and a server host connected to the client host to receive user... The server host, which includes user behavior status and on-demand dialogue settings, comprises: a finite state machine controller integrating multiple finite state machines; a second non-transitory computer-readable storage medium for storing multiple second computer-readable instructions; and a second hardware processor electrically connected to the second non-transitory computer-readable storage medium and the finite state machine controller for executing the multiple second computer-readable instructions. This causes the server host to perform the following actions: generate a rough query message with a natural language structure based on the received user behavior status and on-demand dialogue settings; input the rough query message into the finite state machine for parsing and state transformation to generate precise query message and transmit it to the artificial intelligence platform; receive answer information corresponding to the precise query message from the artificial intelligence platform and store the answer information in an answer list; and automatically filter answer information from the answer list that meets time information and filtering parameters as on-demand dialogue information generated based on the on-demand dialogue settings, and transmit the on-demand dialogue information to the client host for output.

[0004] Therefore, the aforementioned proactive chatbot system with behavior awareness and on-demand dialogue suffers from the following problems: the system has low accuracy in filtering and weighting historical dialogue information, making it difficult to distinguish between key rounds that are highly relevant to the current input and outdated, irrelevant rounds, resulting in a decrease in contextual coherence and dialogue personalization; the system's online incremental update mechanism presents a conflict between speed and stability, with significant delays in model parameter updates and a tendency for parameter shifts when faced with semantic drift or abnormal inputs, affecting the reliability and consistency of the response; the system's target generation strategy has a single trade-off rule, leading to an imbalance between naturalness and relevance in the generated response. Summary of the Invention

[0005] To address this, the present invention provides an AI chatbot response generation system based on online incremental learning. This system overcomes the problem in the prior art where the conflict between speed and stability in online incremental updates leads to model update delays or parameter shifts during high-frequency interactions or semantic drift, thereby reducing response reliability. This is achieved through adaptive learning rate adjustment based on semantic shift, low-rank incremental updates, and parameter snapshot rollback protection mechanisms.

[0006] To achieve the above objectives, the present invention provides an online incremental learning AI chatbot response generation system, comprising: The cache reweighting module is used to calculate the importance weight of each round in the current user input query and historical dialogue cache based on semantic similarity and time decay factor, and to integrate the first preset number of historical dialogue records after sorting the importance weights to obtain a historical dialogue list. A semantic offset calculation module, which is connected to the cache reweighting module, is used to calculate the semantic offset value based on the vector representation of the user's current input query and the reference knowledge vector in the preset language model. An adjustment module, connected to the semantic offset calculation module, is used to adjust the preset learning rate of the preset language model according to the semantic offset value, a preset first offset threshold, and a preset second offset threshold, so as to obtain the adjusted learning rate. An update module, which is connected to the cache reweighting module and the adjustment module respectively, is used to update the preset attention layer parameters and preset output layer parameters in the preset language model according to the historical dialogue list and the adjusted learning rate, so as to obtain the updated parameters; A response generation module, which is connected to the update module and the cache reweighting module respectively, is used to generate a text response result based on the update parameters and the historical dialogue list; The feedback module, which is connected to the response generation module and the cache reweighting module respectively, is used to adjust the importance weights based on the user's feedback signals on the text response results.

[0007] Furthermore, the cache reweighting module is used to obtain preset first balancing hyperparameters and preset second balancing hyperparameters, and to calculate the time interval between the time of each round and the time of the user's current input query based on the semantic similarity between the input query of each round in the historical dialogue cache and the input query of the user, and to calculate the time decay factor based on the preset second balancing hyperparameters, and to calculate the importance weight of each round based on the product of the preset first balancing hyperparameters and the semantic similarity, the preset first balancing hyperparameters and the time decay factor.

[0008] Furthermore, the preset first balancing hyperparameter is used to adjust the contribution ratio of semantic similarity in the total weight, and the preset second balancing hyperparameter is used to control the rate of time decay.

[0009] Furthermore, the adjustment module is used to reduce the preset learning rate by a preset adjustment ratio when the semantic offset value is greater than a preset first offset threshold and less than a preset second offset threshold, so as to obtain the adjusted learning rate.

[0010] Furthermore, the adjustment module is also used to roll back the updated parameters in the preset language model to the previously saved stable parameter snapshot state when the semantic offset value is greater than a preset second offset threshold.

[0011] Furthermore, the update module is used to incrementally update the parameter subsets in the preset attention layer and preset output layer using a low-rank approximation technique to obtain the updated parameters.

[0012] Furthermore, the response generation module is used to integrate and splice all the historical dialogue records in the historical dialogue list in chronological order to form a coherent contextual prompt text, and to splice the contextual prompt text with the user's current input query to obtain an input sequence, and to input the input sequence into the preset language model loaded with the updated parameters for decoding, and to use a weighted strategy to perform multi-objective optimization on response fluency, semantic matching degree between response text and input sequence and predicted user satisfaction during the decoding process, and to generate the text response result based on the decoding strategy of Top-k sampling and temperature adaptive adjustment.

[0013] Furthermore, the feedback module is used to calculate positive feedback weight and negative feedback weight respectively based on the explicit and implicit signals in the feedback signal, and to increase the importance weight of the corresponding historical dialogue record based on the positive feedback weight, and to decrease the importance weight of the corresponding historical dialogue record based on the negative feedback weight.

[0014] Furthermore, the feedback module is also used to statistically analyze the distribution of the feedback signal during a dialogue with a preset number of judgment rounds, determine the user's current dialogue satisfaction trend based on the distribution, and trigger the adjustment module to recalculate the adjustment learning rate when the satisfaction trend is less than a preset satisfaction threshold.

[0015] Furthermore, the semantic offset calculation module is used to denote the vector representation of the user's current input query obtained through the text embedding model as the current vector, and to calculate the cosine distance between the current vector and the reference knowledge vector to obtain the semantic offset value.

[0016] Compared with existing technologies, the advantages of this invention are as follows: by introducing a cache reweighting mechanism based on semantic similarity and time decay factor, historical dialogues that are closer to the user's current input and have higher semantic relevance are prioritized, thereby ensuring the timeliness and semantic continuity of the input context; the semantic offset calculation module quantifies the difference between the current input and the model's reference knowledge, and uses this as the basis for dynamic adjustment of the learning rate, so that the model reduces the update magnitude when the input semantics change significantly, avoiding catastrophic forgetting, and accelerates the convergence speed when the semantics are stable; the update module adopts a low-rank approximation strategy for incremental updates, which improves training efficiency and reduces the destruction of the original parameter distribution of the model. The response generation module performs multi-objective optimization decoding based on the concatenated contextual prompt text and updated parameters, simultaneously improving the generated response in terms of fluency, semantic matching, and user satisfaction. The feedback module maps explicit and implicit user feedback signals to positive and negative weights, and adjusts the importance weights of historical dialogues in real time, forming a closed loop of input-update-output-feedback. This enables the system to continuously optimize response quality and learning strategies in multi-turn dialogues, thereby significantly improving the stability and interactive experience of online incremental learning. It effectively solves the problem of reduced response reliability caused by the conflict between speed and stability in online incremental updates, which leads to model update delays or parameter shifts during high-frequency interactions or semantic drift.

[0017] Furthermore, by multiplying the preset first balancing hyperparameter with the semantic similarity and combining it with the time decay factor calculated by the preset second balancing hyperparameter, the importance weight can reflect the semantic similarity between the historical dialogue content and the current input, and can also dynamically decay the dialogue rounds that are older. In the end, the calculation result takes into account both contextual relevance and timeliness, so as to prioritize the historical records that have the greatest impact on the current query and are still of reference value, avoid outdated information from interfering with model updates, and improve the accuracy and contextual coherence of response generation.

[0018] Furthermore, by using a preset first balancing hyperparameter to adjust the contribution ratio of semantic similarity in the total weight, and using a preset second balancing hyperparameter to control the time decay rate, a dynamic balance between semantic relevance and temporal freshness can be achieved between historical dialogues and current input. This allows the system to fully utilize contextual information that is highly relevant to the current input while effectively suppressing the interference of outdated information on response generation, thereby improving the accuracy and real-time performance of the generated results.

[0019] Furthermore, by reducing the learning rate by a preset ratio when the semantic offset value is between a preset first offset threshold and a preset second offset threshold, the system can dynamically adjust the model update magnitude according to the degree of difference between the current input and the reference knowledge. This ensures that when the semantic offset is small but there are certain changes, the model update will not excessively disturb the existing knowledge, but can also appropriately adapt to the new semantic information, thereby achieving a balanced incremental learning effect and improving the stability and accuracy of response generation.

[0020] Furthermore, by rolling back the updated parameters when the semantic offset value exceeds the second offset threshold, the system can prevent the model from making unstable parameter updates when encountering extreme or abnormal inputs, thereby maintaining the overall consistency and response reliability of the language model. At the same time, it ensures the effective retention of historical learning information, enabling the model to quickly adapt to new semantic information in continuous interactions while avoiding excessive deviation from the existing knowledge structure.

[0021] Furthermore, by incrementally updating the parameter subsets of the attention layer and output layer using a low-rank approximation technique, the model can fine-tune only the parameters most sensitive to response generation while maintaining the overall parameter structure stability. This effectively captures subtle semantic changes between historical dialogues and the current input. This incremental update method reduces computational and storage overhead while maintaining the consistency and coherence of the model output, ensuring that the parameters after each update can balance the flexibility and stability of the response, thereby improving the model's adaptability and generation quality in continuous dialogues.

[0022] Furthermore, by integrating historical dialogue records chronologically to form contextual prompts and concatenating them with the user's current input query, complete semantic continuity can be preserved in the input sequence. This allows the model to fully utilize the correlation between historical information and current semantics during decoding, thereby achieving multi-objective optimization of response fluency, semantic matching accuracy, and predicted user satisfaction. A weighted strategy balances the influence between various objectives, while Top-k sampling and temperature adaptive adjustment further control the diversity and stability of the generated text, ensuring that the generated response not only conforms to user intent but also maintains natural and coherent language, effectively improving the overall interactive experience and model adaptability.

[0023] Furthermore, by quantifying explicit and implicit feedback into positive and negative feedback weights respectively, and dynamically adjusting them in conjunction with the importance weight of historical dialogue records, semantic similarity, and time decay factor, high-value information is prioritized in the response generation process, while the importance of low-value or mismatched information is reduced, thereby achieving intelligent filtering and optimization of historical dialogues. This mechanism can adaptively reflect changes in user preferences and interaction behaviors, improve the relevance, coherence, and user satisfaction of generated text, while ensuring that the impact of feedback signals on model updates is controllable and stable.

[0024] Furthermore, by statistically analyzing the distribution of user feedback signals within a preset number of rounds, the trend of user satisfaction is dynamically determined, and the trend results are compared with a preset satisfaction threshold. This triggers a readjustment of the learning rate when satisfaction declines, enabling the model to optimize the weight allocation of historical dialogues and current inputs in a timely manner. This mechanism can adaptively reflect changes in user interaction experience, balance the relationship between the magnitude of model updates and response quality, effectively improve the accuracy, consistency, and user satisfaction of responses, while ensuring the stability and continuity of the model update process.

[0025] Furthermore, by mapping the user's current input query to a vector representation and calculating the cosine distance with the reference knowledge vector, the semantic difference between the input content and existing knowledge can be accurately quantified. This provides a reliable basis for subsequent learning rate adjustments and model parameter updates, helping the model to effectively adapt to new inputs while retaining historical semantic information, improving the accuracy and relevance of the response, and maintaining the coherence of contextual understanding. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the AI ​​chatbot response generation system using online incremental learning in this embodiment; Figure 2 This is a logic diagram for the adjustment module to calculate the adjustment learning rate in this embodiment; Figure 3 This is a logic diagram for determining the rollback of the adjustment module update parameters in this embodiment; Figure 4 This is the logic diagram for determining the recalculation and adjustment of the learning rate in the feedback module of this embodiment. Detailed Implementation

[0027] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0028] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0029] Please see Figure 1 As shown, this is a schematic diagram of the AI ​​chatbot response generation system based on online incremental learning in this embodiment. This embodiment provides an AI chatbot response generation system based on online incremental learning, including: The cache reweighting module is used to calculate the importance weight of each round in the current user input query and historical dialogue cache based on semantic similarity and time decay factor, and to integrate the first preset number of historical dialogue records after sorting the importance weights to obtain a historical dialogue list. A semantic offset calculation module, which is connected to the cache reweighting module, is used to calculate the semantic offset value based on the vector representation of the user's current input query and the reference knowledge vector in the preset language model. An adjustment module, connected to the semantic offset calculation module, is used to adjust the preset learning rate of the preset language model according to the semantic offset value, a preset first offset threshold, and a preset second offset threshold, so as to obtain the adjusted learning rate. An update module, which is connected to the cache reweighting module and the adjustment module respectively, is used to update the preset attention layer parameters and preset output layer parameters in the preset language model according to the historical dialogue list and the adjusted learning rate, so as to obtain the updated parameters; A response generation module, which is connected to the update module and the cache reweighting module respectively, is used to generate a text response result based on the update parameters and the historical dialogue list; The feedback module, which is connected to the response generation module and the cache reweighting module respectively, is used to adjust the importance weights based on the user's feedback signals on the text response results.

[0030] In this embodiment, the system is deployed in an AI-powered online research service platform. Facing a large number of multi-turn text interactions (including but not limited to research Q&A), the system continuously receives current input and historical dialogues during real-time user conversations and collects explicit / implicit feedback. While maintaining basic language capabilities, the system needs to optimize the response generation strategy online by weighting the timeliness and semantic relevance of historical contexts, adjusting the learning rate based on semantic shifts, and performing low-rank incremental updates. This forms a closed loop of input-update-generation-feedback, thereby continuously improving the coherence, accuracy, and user satisfaction of responses in multi-turn interactions. The system is applicable to both research model debugging and production-level deployment workflows.

[0031] The preset number is the number of historical dialogue records selected, which depends on the model context window length and the average number of words in the current input query. It is usually set between 5 and 20 records. In this embodiment, it is set to 10 records, which can ensure semantic continuity while avoiding the increase in computational overhead caused by excessively long context.

[0032] The pre-trained language model refers to a deep neural network model that has been pre-trained on a large-scale text corpus and has basic language understanding and generation capabilities. In this embodiment, an autoregressive language model with the Transformer architecture is selected, which includes an embedding layer, several multi-head attention layers, a feedforward network layer, and an output layer. The embedding layer is used to transform the input text sequence into a high-dimensional vector representation, the attention layer is used to capture the dependencies between different positions in the sequence, the feedforward network layer is used for non-linear feature transformation, and the output layer is used to generate word probability distributions to decode the text. This model is loaded as the basic model in this system, and the low-rank adaptation (LoRA) module is introduced into the parameters of the attention layer and the output layer through incremental learning to update them online, so as to quickly adapt to user dialogue scenarios while retaining the original language capabilities.

[0033] In this embodiment, the preset learning rate refers to a scalar hyperparameter used to control the step size of parameter updates during online incremental fine-tuning, determining the magnitude of weight changes in each gradient descent. For scenarios that only update the LoRA / low-rank adapter, a commonly used baseline learning rate is typically 1e. -5 up to 1e -3 Order of magnitude (typical value 1e) -4 up to 5e -4 ), while the finer adjustments are even smaller when fine-tuning the entire base model (e.g., 1e). -6 up to 1e -5The learning rate should be chosen in a way that balances convergence speed and stability: an excessively large learning rate can cause sudden changes in model output and forgetting of existing capabilities, while an excessively small learning rate will lead to slow adaptation. The preset attention layer parameters usually refer to the set of weights in the multi-head attention sub-layer of the Transformer (including the query / key / value linear transformation matrices Wq, Wk, Wv and the head merging / output projection matrix Wo, as well as the corresponding bias and layer normalization scaling / bias terms). Its function is to map the hidden state to attention key-value pairs and complete the aggregation of information at different positions, which directly affects the model's ability to model contextual dependencies. The preset output layer parameters usually refer to the linear transformation matrix (and bias) that projects the model's hidden representation onto the vocabulary probability distribution, which determines the final generation distribution and style of the lexical units. In online incremental learning, the original large weights are not usually modified directly. Instead, a LoRA-type low-rank increment matrix ΔW = A×B is introduced on these layers (A and B are low-rank matrices, and their rank r is much smaller than the dimension of the original matrix). Only A, B or their scaling factors are trained, thereby achieving functional adjustment with a small number of parameters. It is recommended that these "update parameters" be initialized with zero or small values ​​at the beginning, and the update magnitude is controlled by combining snapshot saving, gradient pruning and small weight decay, so as to minimize the destruction of the original behavior of the base model while improving context adaptability and generation quality.

[0034] The preset first offset threshold is a key parameter used to determine whether semantic shift or gradient update intensity triggers learning rate adjustment. It depends on the model's baseline performance and semantic similarity distribution on the initial task, and is usually set between 0.2 and 0.5. In this embodiment, it is set to 0.3, which can suppress over-updates and keep the model output stable when a small input semantic shift is detected. The preset second offset threshold is a parameter used to determine whether to enable large-scale parameter updates or switch to high-intensity adaptation mode. It depends on the magnitude of changes in the training corpus and the sensitivity of the target task to new knowledge, and is usually set between 0.6 and 0.8. In this embodiment, it is set to 0.7, which can trigger faster weight adjustments when the semantic shift is significant, thereby quickly converging to the new task distribution.

[0035] By introducing a cache reweighting mechanism based on semantic similarity and time decay factors, historical dialogues that are closer to the user's current input and have higher semantic relevance are prioritized, thus ensuring the timeliness and semantic continuity of the input context. A semantic offset calculation module quantifies the difference between the current input and the model's reference knowledge, using this as the basis for dynamic adjustment of the learning rate. This allows the model to reduce the update magnitude when the input semantics change significantly, avoiding catastrophic forgetting, and accelerate convergence when the semantics are stable. The update module uses a low-rank approximation strategy for incremental updates, improving training efficiency while minimizing disruption to the original parameter distribution of the model. The response generation module is based on... The subsequent contextual prompts and updated parameters are decoded using multi-objective optimization, simultaneously improving the fluency, semantic matching, and user satisfaction of the generated response. The feedback module maps explicit and implicit user feedback signals to positive and negative weights, adjusting the importance weights of historical dialogues in real time to form a closed loop of input-update-output-feedback. This allows the system to continuously optimize response quality and learning strategies in multi-turn dialogues, significantly improving the stability and interactive experience of online incremental learning. It effectively solves the problem of reduced response reliability caused by the conflict between speed and stability in online incremental updates, leading to model update delays or parameter shifts during high-frequency interactions or semantic drift.

[0036] Specifically, the cache reweighting module is used to obtain preset first balancing hyperparameters and preset second balancing hyperparameters, and to calculate the time interval between the time of each round and the time of the user's current input query based on the semantic similarity between the input queries of each round in the historical dialogue cache and the input query of the user, and to calculate the time decay factor based on the preset second balancing hyperparameters, and to calculate the importance weight of each round based on the product of the preset first balancing hyperparameters and the semantic similarity, and the preset first balancing hyperparameters and the time decay factor. , Among them, w i Let x be the importance weight for the i-th round, α be the preset first balancing hyperparameter, β be the preset second balancing hyperparameter, sim() be the cosine similarity function, and x be the weight for the i-th round. i For the input query in the i-th round, x current For the user's current input query, Δt i This is the time interval between the time of each round and the time of the user's current input query.

[0037] The preset first balancing hyperparameter is a coefficient used to control the proportion of semantic similarity in the importance weight calculation. It depends on the model's need to balance historical context dependence and current input sensitivity, and is usually set between 0.5 and 0.9. In this embodiment, it is set to 0.7, which can enhance the weight of historical records that are semantically close to the current input, thereby improving the context relevance of the response. The preset second balancing hyperparameter is a coefficient used to control the time decay rate. It depends on the system's requirements for the timeliness of historical dialogues, and is usually set between 0.01 and 0.1. In this embodiment, it is set to 0.05, which can moderately reduce the weight of older dialogue rounds and avoid outdated information interfering with the model's learning and response generation.

[0038] By multiplying the preset first balancing hyperparameter with the semantic similarity and combining it with the time decay factor calculated by the preset second balancing hyperparameter, the importance weight can reflect the semantic similarity between the historical dialogue content and the current input, and can also dynamically decay the dialogue rounds that are older. In the end, the calculation result takes into account both contextual relevance and timeliness, so as to prioritize the historical records that have the greatest impact on the current query and are still of reference value, avoid outdated information from interfering with model updates, and improve the accuracy of response generation and contextual coherence.

[0039] Specifically, the preset first balancing hyperparameter is used to adjust the contribution ratio of semantic similarity in the total weight, and the preset second balancing hyperparameter is used to control the rate of time decay.

[0040] By using a preset first balancing hyperparameter to adjust the contribution ratio of semantic similarity to the total weight, and a preset second balancing hyperparameter to control the time decay rate, a dynamic balance between semantic relevance and temporal freshness can be achieved between historical dialogues and current input. This allows the system to fully utilize contextual information that is highly relevant to the current input while effectively suppressing the interference of outdated information on response generation, thereby improving the accuracy and real-time performance of the generated results.

[0041] Please see Figure 2 As shown, it is a logic diagram for the adjustment module to calculate the adjustment learning rate in this embodiment. In this embodiment, the adjustment module is used to reduce the preset learning rate by a preset adjustment ratio when the semantic offset value is greater than a preset first offset threshold and less than a preset second offset threshold, so as to obtain the adjustment learning rate.

[0042] The preset adjustment ratio is a coefficient used to control the reduction of the learning rate. It depends on the relative size of the semantic offset value and the threshold, and is usually set between 0.1 and 0.9. In this embodiment, it is set to 0.5, which can effectively reduce the learning rate when the semantic offset is moderate, thus balancing the stability and adaptability of the model.

[0043] By reducing the learning rate by a preset ratio when the semantic offset value is between a preset first offset threshold and a preset second offset threshold, the system can dynamically adjust the model update magnitude according to the degree of difference between the current input and the reference knowledge. This ensures that when the semantic offset is small but there are certain changes, the model update will not excessively disturb the existing knowledge, but can also appropriately adapt to the new semantic information, thereby achieving a balanced incremental learning effect and improving the stability and accuracy of response generation.

[0044] Please see Figure 3 As shown, it is the logic diagram for determining the rollback of the update parameters of the adjustment module in this embodiment. In this embodiment, the adjustment module is also used to roll back the update parameters in the preset language model to the previously saved stable parameter snapshot state when the semantic offset value is greater than the preset second offset threshold.

[0045] In this embodiment, rolling back the updated parameters to the previously saved stable parameter snapshot state means that when the semantic offset value of the current input is detected to exceed the preset second offset threshold, the system will restore all incremental update parameters of the attention layer and output layer in the preset language model to the previously saved stable version, that is, replace the current parameters with the previously saved LoRA weight snapshot. Through this operation, the model can undo all unstable adjustments caused by abnormal or deviated inputs, ensuring that the subsequent response generation is still based on verified, stable and reliable model parameters, thereby avoiding abrupt changes or error accumulation in the output content.

[0046] By rolling back parameter updates when the semantic offset value exceeds the second offset threshold, the system can prevent unstable parameter updates when the model encounters extreme or abnormal inputs, thereby maintaining the overall consistency and response reliability of the language model. At the same time, it ensures the effective retention of historical learning information, enabling the model to quickly adapt to new semantic information in continuous interactions while avoiding excessive deviation from the existing knowledge structure.

[0047] Specifically, the update module is used to incrementally update the parameter subsets in the preset attention layer and preset output layer using a low-rank approximation technique to obtain the updated parameters.

[0048] In this embodiment, the low-rank approximation technique is implemented using the LoRA (Low-Rank Adaptation) structure. Specifically, the parameters of the attention and output layers of the pre-defined language model are kept frozen, and only low-rank matrix pairs (usually denoted as A and B) are inserted into these layers, where A is the input mapping matrix and B is the output mapping matrix, and their ranks are much lower than the original weight matrices. During incremental learning, the model only updates the parameters of A and B, while the original weights remain unchanged. In this way, LoRA can maintain the original representational capabilities of the model while efficiently adapting to new semantic information through the low-rank subspace, achieving incremental updates, significantly reducing training computation and storage overhead, and ensuring the stability and consistency of the model's generated responses.

[0049] By incrementally updating a subset of parameters in the attention and output layers using a low-rank approximation technique, the model can fine-tune only the parameters most sensitive to response generation while maintaining the overall parameter structure stability. This effectively captures subtle semantic changes between historical dialogues and the current input. This incremental update method reduces computational and storage overhead while maintaining the consistency and coherence of the model output, ensuring that the parameters after each update can balance the flexibility and stability of the response, thereby improving the model's adaptability and generation quality in continuous dialogues.

[0050] Specifically, the response generation module integrates and splices all the historical dialogue records in the historical dialogue list in chronological order to form a coherent contextual prompt text; it also splices the contextual prompt text with the user's current input query to obtain an input sequence; it inputs the input sequence into the preset language model loaded with the updated parameters for decoding; and during the decoding process, it uses a weighted strategy to perform multi-objective optimization on response fluency, semantic matching degree between the response text and the input sequence, and predicted user satisfaction, and generates the text response result based on a decoding strategy of Top-k sampling and temperature adaptive adjustment.

[0051] In this embodiment, the response generation module integrates and concatenates the historical dialogue list in chronological order to form contextual prompt text, and concatenates it with the user's current input query to obtain an input sequence. This input sequence is then input into a preset language model with updated parameters for decoding. During the decoding process, the response fluency, semantic matching degree, and predicted user satisfaction are calculated for each generated candidate word or sentence. These are then weighted and combined according to preset weights (fluency 0.4, semantic matching degree 0.4, user satisfaction 0.2) to obtain a comprehensive score. Simultaneously, the temperature is adaptively adjusted based on the matching degree between the current input query and the context (high matching degree T=0.7, medium matching degree T=1.0, low matching degree T=1.2). Combined with a Top-k sampling (k=50) strategy, the optimal generation result is selected from the candidate words, thereby generating a text response that is both fluent and natural, semantically appropriate, and meets the user's expected satisfaction.

[0052] By integrating historical dialogue records chronologically to form contextual prompts and concatenating them with the user's current input query, complete semantic continuity can be preserved in the input sequence. This allows the model to fully utilize the correlation between historical information and current semantics during decoding, thereby achieving multi-objective optimization of response fluency, semantic matching accuracy, and predicted user satisfaction. A weighted strategy balances the influence between various objectives, while Top-k sampling and temperature adaptive adjustment further control the diversity and stability of the generated text. This ensures that the generated response not only conforms to user intent but also maintains natural and coherent language, effectively improving the overall interactive experience and model adaptability.

[0053] Specifically, the feedback module is used to calculate positive feedback weight and negative feedback weight based on the explicit and implicit signals in the feedback signal, and to increase the importance weight of the corresponding historical dialogue record based on the positive feedback weight, and to decrease the importance weight of the corresponding historical dialogue record based on the negative feedback weight.

[0054] In this embodiment, after receiving the user's feedback signal on the text response, the feedback module first distinguishes between explicit and implicit signals and calculates the corresponding positive and negative feedback weights respectively. Subsequently, the importance of relevant historical dialogue records is increased based on the positive feedback weight, so that these high-value information are given priority when generating subsequent responses. At the same time, the importance of low-value or mismatched historical records is reduced based on the negative feedback weight, thereby achieving dynamic optimization of the historical dialogue list. In this process, the adjustment range of each weight works together with the strength of the user feedback signal, the semantic similarity between the historical dialogue and the current input, and the time decay factor, so that the selection of historical information can adaptively respond to changes in user preferences, improving the relevance, coherence, and user satisfaction of the model's generated response.

[0055] By quantifying explicit and implicit feedback into positive and negative feedback weights respectively, and dynamically adjusting them in conjunction with the importance weight of historical dialogue records, semantic similarity, and time decay factor, high-value information is prioritized in the response generation process, while the importance of low-value or mismatched information is reduced, thereby achieving intelligent filtering and optimization of historical dialogues. This mechanism can adaptively reflect changes in user preferences and interaction behaviors, improve the relevance, coherence, and user satisfaction of generated text, while ensuring that the impact of feedback signals on model updates is controllable and stable.

[0056] Please see Figure 4 As shown, it is the judgment logic diagram of the feedback module recalculating the adjustment learning rate in this embodiment. In this embodiment, the feedback module is also used to statistically analyze the distribution of the feedback signal during the dialogue process of a preset number of judgment rounds, and to determine the satisfaction trend of the user's current dialogue based on the distribution. When the satisfaction trend is less than a preset satisfaction threshold, the adjustment module is triggered to recalculate the adjustment learning rate.

[0057] In this embodiment, the feedback module statistically analyzes the distribution of feedback signals and determines the trend of user satisfaction in the following manner: Statistical Feedback Signal Distribution: During the dialogue process with a preset number of judgment rounds, the feedback module records the feedback signals for each round, including explicit signals (such as user clicks on the "Satisfied / Dissatisfied" button and ratings) and implicit signals (including but not limited to user message length, response delay, and input repetition rate). These signals are accumulated by round to form a feedback data sequence. For explicit signals, positive and negative feedback are counted separately; for implicit signals, each round of feedback is mapped to a numerical weight according to a predefined rating function, and then the frequency distribution of each weight is statistically analyzed to obtain the probability distribution or weighted average distribution of the overall feedback signal.

[0058] Determine the satisfaction trend: Based on the statistically obtained distribution, calculate user satisfaction indicators, such as using a weighted average or moving average method to obtain the satisfaction score for each round of conversation. Then, compare the satisfaction score of the current round with the scores of previous rounds and calculate the trend value (such as the increase / decrease rate or difference) to determine whether the user's satisfaction with the current conversation is increasing, remaining stable, or decreasing.

[0059] Triggering mechanism: When the calculated satisfaction trend is less than the preset satisfaction threshold, the system considers the user's current dialogue satisfaction to have decreased, thereby triggering the adjustment module to recalculate the learning rate and model parameter update strategy to enhance the responsiveness to the user's current input and improve the overall dialogue quality.

[0060] The preset satisfaction threshold is a reference value used to determine whether the user's satisfaction with the current conversation has decreased. It depends on the feedback habits of the target user group and the system response requirements, and is usually set between 0 and 1. In this embodiment, it is set to 0.7, which can trigger model adjustment in time when the user's satisfaction is lower than this threshold, thereby improving the adaptability and accuracy of subsequent responses.

[0061] By statistically analyzing the distribution of user feedback signals within a preset number of rounds, the trend of user satisfaction is dynamically determined. The trend results are then compared with a preset satisfaction threshold, triggering a readjustment of the learning rate when satisfaction declines. This allows the model to optimize the weight allocation of historical dialogues and current inputs in a timely manner. This mechanism can adaptively reflect changes in user interaction experience, balance the relationship between the magnitude of model updates and response quality, effectively improve the accuracy, consistency, and user satisfaction of responses, while ensuring the stability and continuity of the model update process.

[0062] Specifically, the semantic offset calculation module is used to denote the vector representation of the user's current input query obtained through the text embedding model as the current vector, and to calculate the cosine distance between the current vector and the reference knowledge vector to obtain the semantic offset value. , Where d is the semantic offset value, v current For the current vector, v ref For reference knowledge vectors.

[0063] Text embedding models are models that map natural language text into fixed-dimensional vectors. They are typically trained based on deep neural networks (such as the Transformer architecture). By capturing the semantic relationships between words, phrases, and sentences, semantically similar texts are placed closer together in the vector space. This enables semantic comparison, retrieval, clustering, or as the input representation for downstream tasks. In this embodiment, the vector representation is used to calculate the cosine distance between the user input and the reference knowledge to obtain a semantic offset value, which provides a basis for adjusting the learning rate, optimizing model updates, and generating responses that conform to the context.

[0064] By mapping the user's current input query to a vector representation and calculating the cosine distance with the reference knowledge vector, the semantic difference between the input content and existing knowledge can be accurately quantified. This provides a reliable basis for subsequent learning rate adjustments and model parameter updates, helping the model to effectively adapt to new inputs while retaining historical semantic information, improving the accuracy and relevance of the response, and maintaining the coherence of contextual understanding.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An online incremental learning AI chatbot response generation system, characterized in that, include: The cache reweighting module is used to calculate the importance weight of each round in the current user input query and historical dialogue cache based on semantic similarity and time decay factor, and to integrate the first preset number of historical dialogue records after sorting the importance weights to obtain a historical dialogue list. A semantic offset calculation module, which is connected to the cache reweighting module, is used to calculate the semantic offset value based on the vector representation of the user's current input query and the reference knowledge vector in the preset language model. An adjustment module, connected to the semantic offset calculation module, is used to adjust the preset learning rate of the preset language model according to the semantic offset value, a preset first offset threshold, and a preset second offset threshold, so as to obtain the adjusted learning rate. An update module, which is connected to the cache reweighting module and the adjustment module respectively, is used to update the preset attention layer parameters and preset output layer parameters in the preset language model according to the historical dialogue list and the adjusted learning rate, so as to obtain the updated parameters; A response generation module, which is connected to the update module and the cache reweighting module respectively, is used to generate a text response result based on the update parameters and the historical dialogue list; The feedback module is connected to both the response generation module and the cache reweighting module to adjust the importance weights based on user feedback signals regarding the text response results.

2. The AI ​​chatbot response generation system for online incremental learning according to claim 1, characterized in that, The cache reweighting module is used to obtain preset first balancing hyperparameters and preset second balancing hyperparameters, and to calculate the time interval between the time of each round and the time of the user's current input query based on the semantic similarity between the input query of each round in the historical dialogue cache and the input query of the user, and to calculate the time decay factor based on the preset second balancing hyperparameters, and to calculate the importance weight of each round based on the product of the preset first balancing hyperparameters and the semantic similarity, and the preset first balancing hyperparameters and the time decay factor.

3. The AI ​​chatbot response generation system for online incremental learning according to claim 2, characterized in that, The preset first balancing hyperparameter is used to adjust the contribution ratio of semantic similarity in the total weight, and the preset second balancing hyperparameter is used to control the rate of time decay.

4. The AI ​​chatbot response generation system for online incremental learning according to claim 3, characterized in that, The adjustment module is used to reduce the preset learning rate by a preset adjustment ratio when the semantic offset value is greater than a preset first offset threshold and less than a preset second offset threshold, so as to obtain the adjusted learning rate.

5. The AI ​​chatbot response generation system for online incremental learning according to claim 4, characterized in that, The adjustment module is also used to roll back the updated parameters in the preset language model to the previously saved stable parameter snapshot state when the semantic offset value is greater than the preset second offset threshold.

6. The AI ​​chatbot response generation system for online incremental learning according to claim 5, characterized in that, The update module is used to incrementally update the parameter subsets in the preset attention layer and preset output layer using a low-rank approximation technique to obtain the updated parameters.

7. The AI ​​chatbot response generation system for online incremental learning according to claim 6, characterized in that, The response generation module is used to integrate and splice all the historical dialogue records in the historical dialogue list in chronological order to form a coherent contextual prompt text, and to splice the contextual prompt text with the user's current input query to obtain an input sequence, and to input the input sequence into the preset language model loaded with the updated parameters for decoding, and to use a weighted strategy to perform multi-objective optimization on response fluency, semantic matching degree between response text and input sequence and predicted user satisfaction during the decoding process, and to generate the text response result based on the decoding strategy of Top-k sampling and temperature adaptive adjustment.

8. The AI ​​chatbot response generation system for online incremental learning according to claim 7, characterized in that, The feedback module is used to calculate positive feedback weight and negative feedback weight based on the explicit and implicit signals in the feedback signal, and to increase the importance weight of the corresponding historical dialogue record based on the positive feedback weight, and to decrease the importance weight of the corresponding historical dialogue record based on the negative feedback weight.

9. The AI ​​chatbot response generation system for online incremental learning according to claim 8, characterized in that, The feedback module is also used to statistically analyze the distribution of the feedback signal during a dialogue with a preset number of rounds, determine the user's current dialogue satisfaction trend based on the distribution, and trigger the adjustment module to recalculate the adjustment learning rate when the satisfaction trend is less than a preset satisfaction threshold.

10. The AI ​​chatbot response generation system for online incremental learning according to claim 9, characterized in that, The semantic offset calculation module is used to denote the vector representation of the user's current input query obtained through the text embedding model as the current vector, and to calculate the cosine distance between the current vector and the reference knowledge vector to obtain the semantic offset value.

Images