AIAgent online learning adaptive optimization method

Abstract

The invention relates to the technical field of natural language processing, and particularly discloses an AIAgent online learning adaptive optimization method, which comprises the following steps of: performing text analysis on first input information input into an AIAgent in real time to determine a real-time environment condition in a current preset time window; the method comprises the following steps: acquiring a real-time environment condition in a current preset time window and a historical environment condition in a previous preset time window, and acquiring memory association background information and second environment application information of an initial language model based on a memory mechanism of AIAgent so as to determine the historical environment condition in the previous preset time window, so that the real-time environment condition in the current preset time window and the historical environment condition in the previous preset time window are acquired; therefore, the intelligent agent-oriented online learning optimization strategy is generated according to the difference between the online associated background information and the memory associated background information in combination with the first environment application information, the second environment application information and the initial language model, so that the real-time environment fitness of the intelligent agent can be effectively improved, the scene task processing efficiency is improved, and the user experience is improved. And the real-time application requirement of the user is met.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an AI Agent online learning adaptive optimization method. Background Art

[0002] An AIAgent, or artificial intelligence agent, is an intelligent entity capable of environmental perception, autonomous decision-making, and action. Unlike traditional AI systems, which passively respond, it can proactively plan and execute complex task chains, evolving from a mere information processor to a proactive, goal-driven executor. Currently, AI agents generally function well due to their learned experience and the support of historical big data models. However, in rapidly changing real-time environments, AI agents often fail to adapt effectively and timely, leading to delays or even inability to complete current tasks, making it difficult to meet users' real-time application needs. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the present invention proposes an AI Agent online learning adaptive optimization method that can effectively improve the real-time environment adaptability of the agent to meet the real-time application needs of users.

[0004] An embodiment of the present invention provides an AI Agent online learning adaptive optimization method, wherein the AI ​​Agent configures an initial language model, and the method includes:

[0005] Performing text parsing on first input information input online into the initial language model to obtain online associated background information and first environment application information, wherein the first input information corresponds to a current preset time window;

[0006] Based on the memory mechanism of the AIAgent, acquiring the memory-associated background information and the second environment application information of the initial language model, wherein the second environment application information corresponds to the previous preset time window;

[0007] According to the difference between the online associated background information and the memory associated background information, combined with the first environment application information, the second environment application information and the initial language model, an online learning optimization strategy for the AI ​​Agent is generated.

[0008] Optionally, in one embodiment of the present invention, generating an online learning optimization strategy for the AI ​​Agent based on the difference between the online associated context information and the memory associated context information, in combination with the first environment application information, the second environment application information, and the initial language model, includes:

[0009] When it is determined that there is a difference between the online associated context information and the memory associated context information, obtaining an online training loss of the initial language model, and performing forgetting processing on the memory associated context information in the initial language model to obtain a first language model;

[0010] Training the first language model according to the online associated background information to obtain a second language model, and obtaining an online training loss of the second language model;

[0011] Obtaining a difference between the online training loss of the second language model and the online training loss of the initial language model to obtain a difference loss value;

[0012] When the difference loss value is less than or equal to a preset difference threshold, an online learning optimization strategy for the AI ​​Agent is generated according to the second language model, the first environment application information, and the second environment application information.

[0013] Optionally, in one embodiment of the present invention, generating an online learning optimization strategy for the AI ​​Agent based on the second language model, the first environment application information, and the second environment application information includes:

[0014] training the second language model at a preset learning rate according to the first environment application information and the second environment application information;

[0015] During the training process of the second language model, respectively obtaining the distribution of the first environment application information and the second environment application information relative to the training data of the second language model to obtain a first distribution value and a second distribution value;

[0016] When the absolute value of the difference between the first distribution value and the second distribution value is less than a preset distribution threshold, modifying the preset learning rate to obtain an optimized learning rate;

[0017] The second language model is trained at the optimized learning rate according to the first environment application information to obtain a third language model.

[0018] Optionally, in one embodiment of the present invention, the modifying the preset learning rate to obtain the optimized learning rate includes:

[0019] When the first distribution value is greater than or equal to the second distribution value, obtaining a difference between the preset learning rate and a preconfigured learning rate adjustment variable to obtain an optimized learning rate, wherein the learning rate adjustment variable is less than the preset learning rate;

[0020] or,

[0021] When the first distribution value is less than the second distribution value, the sum of the preset learning rate and a preconfigured learning rate adjustment variable is obtained to obtain an optimized learning rate.

[0022] Optionally, in one embodiment of the present invention, generating an online learning optimization strategy for the AI ​​Agent based on the difference between the online associated context information and the memory associated context information, in combination with the first environment application information, the second environment application information, and the initial language model, includes:

[0023] When it is determined that there is no difference between the online associated context information and the memory associated context information, respectively obtaining label categories of the first environment application information and the second environment application information;

[0024] Based on the initial language model, the first environment application information and the second environment application information, combined with the label category of the first environment application information and the label category of the second environment application information, an online learning optimization strategy for the AI ​​Agent is generated.

[0025] Optionally, in one embodiment of the present invention, generating an online learning optimization strategy for the AI ​​Agent based on the initial language model, the first environment application information, and the second environment application information, in combination with a label category of the first environment application information and a label category of the second environment application information, includes:

[0026] When the label category of the first environment application information is the same as the label category of the second environment application information, training the initial language model according to the first environment application information to obtain response result data output by the initial language model;

[0027] Constructing a first reward model based on a self-attention model and a long short-term memory network, and performing supervised fine-tuning on the first reward model according to the response result data to obtain a second reward model;

[0028] In a case where a plurality of response scores of the response result data are outputted by the second reward model, reinforcement learning is used to optimize the initial language model based on the plurality of response scores to obtain a fourth language model.

[0029] Optionally, in one embodiment of the present invention, generating an online learning optimization strategy for the AI ​​Agent based on the initial language model, the first environment application information, and the second environment application information, in combination with a label category of the first environment application information and a label category of the second environment application information, includes:

[0030] When the label category of the first environment application information is different from the label category of the second environment application information, performing forgetting training on the initial language model according to the second environment application information to obtain a fifth language model;

[0031] When the online training loss of the fifth language model is less than or equal to a minimum preset loss threshold, determining that the training of the fifth language model is completed;

[0032] The pre-trained fifth language model is locally trained according to the first environment application information to obtain a sixth language model.

[0033] Optionally, in one embodiment of the present invention, performing text parsing on the first input information input online into the initial language model to obtain online associated background information and first environment application information includes:

[0034] The first input information input online into the initial language model is sequentially subjected to data cleaning, word segmentation processing, feature extraction and cluster analysis to obtain online associated background information and first environmental application information.

[0035] Optionally, in one embodiment of the present invention, performing forgetting processing on the memory-associated background information to obtain a first language model includes:

[0036] Based on the memory mechanism of the AI ​​Agent, searching for additional background information different from the online associated background information and the memory associated background information;

[0037] When the additional context information is found, the initial language model is trained according to the additional context information and the memory-associated context information to obtain a first language model; or, when the additional context information is not found, the initial language model is partially trained according to the memory-associated context information to obtain a first language model.

[0038] The present invention proposes an AI Agent online learning adaptive optimization method, which performs text parsing on the first input information in the real-time input AI Agent to determine the real-time environment situation within the current preset time window, and obtains the memory-related background information and second environment application information of the initial language model based on the memory mechanism of the AI ​​Agent to determine the historical environment situation within the previous preset time window. Since the real-time environment situation within the current preset time window and the historical environment situation within the previous preset time window are known, the two can be compared and analyzed, that is, based on the difference between the online-related background information and the memory-related background information, the first environment application information, the second environment application information and the initial language model are taken into consideration, thereby generating an online learning optimization strategy for the intelligent agent. Based on the online learning optimization strategy, the real-time environment adaptability of the intelligent agent can be effectively improved, the scene task processing efficiency can be improved, and the real-time application needs of users can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of an AIAgent online learning adaptive optimization method provided by one embodiment of the present invention;

[0040] Figure 2 yes Figure 1 A flow chart of step S3 in FIG.

[0041] Figure 3 yes Figure 2 A partial flow chart of the step S31 in step “forgetting the memory-associated background information to obtain a first language model”;

[0042] Figure 4 yes Figure 2 Flowchart of step S34 in FIG.

[0043] Figure 5 yes Figure 1 Another flow chart of step S3 in ;

[0044] Figure 6 yes Figure 5 A flow chart of step S36 in FIG.

[0045] Figure 7 yes Figure 5 Another flow chart of step S36 in ;

[0046] Figure 8 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] like Figure 1As shown, an embodiment of the present invention provides an AIAgent online learning adaptive optimization method, which may specifically include but is not limited to steps S1 to S3.

[0048] Step S1: performing text parsing on first input information input online into the initial language model to obtain online associated background information and first environment application information, wherein the first input information corresponds to the current preset time window. There is no restriction on the specific model of the initial language model. An existing network model on the market, such as GPT, Codex, Pangu, etc., can be used, or a network model can be set by itself, and the specific setting needs to be made accordingly according to the actual application scenario;

[0049] Step S2: Based on the memory mechanism of the AI ​​Agent, the memory-related background information and the second environment application information of the initial language model are obtained, wherein the second environment application information corresponds to the previous preset time window;

[0050] It should be noted that the length of the preset time window can be selected and set according to the actual scenario and is not limited here. The preset time window can also be understood to include multiple preset time windows with shorter time lengths. In other words, the first input information can also correspond to the current multiple preset time windows. Then, the second environment application information to be collected at this time corresponds to the previous multiple preset time windows. It is only necessary to ensure that the number of the multiple preset time windows corresponding to the first input information is the same as the number of the multiple preset time windows corresponding to the second input information.

[0051] It can be understood that the online associated context information is used to characterize the environmental application context within the current preset time window. Different from the online associated context information, the first environmental application information is used to characterize the environmental application content within the current preset time window. For example, through text analysis, it can be determined that a customer service call is in progress within the current preset time window. A user is asking the customer service, and the content of the inquiry is "Help me check the weather conditions in place A for the next week." Then the online associated context information at this time represents the application of the current preset time window to daily inquiry scenarios. The first environmental application information represents the inquiry content information "Help me check the weather conditions in place A for the next week" in the daily inquiry scenario. Similarly, the presentation form of the memory associated context information is the same as that of the online associated context information. The presentation form of the second environmental application information is the same as that of the first environmental application information. The only difference is that the corresponding preset time windows are different, which will not be elaborated here.

[0052] Step S3: Based on the difference between the online associated background information and the memory associated background information, combined with the first environment application information, the second environment application information and the initial language model, an online learning optimization strategy for the AI ​​Agent is generated.

[0053] In this step, the first input information in the real-time input AIAgent is subjected to text parsing to determine the real-time environment situation within the current preset time window, and the memory-related background information and the second environment application information of the initial language model are obtained based on the memory mechanism of the AIAgent to determine the historical environment situation within the previous preset time window. Since the real-time environment situation within the current preset time window and the historical environment situation within the previous preset time window are known, a comparative analysis can be performed between the two, that is, based on the difference between the online-related background information and the memory-related background information, the first environment application information, the second environment application information and the initial language model are taken into consideration to generate an online learning optimization strategy for the intelligent agent. Based on the online learning optimization strategy, the real-time environment adaptability of the intelligent agent can be effectively improved, the efficiency of scene task processing can be improved, and the real-time application needs of users can be met.

[0054] In one embodiment, since the intelligent agent is able to store experience and knowledge and support long-term learning, it can at least use the internal memory to store short-term memory, such as but not limited to basic information such as interaction details and scene features during the previous task. It can also store some long-term memory according to specific needs, such as but not limited to information such as the task execution status and personal characteristic information of the user corresponding to each segment of short-term memory. Therefore, according to the memory mechanism of the intelligent agent, the memory-related background information and the second environment application information of the initial language model within the previous preset time window can be obtained; if this part of the memory data is presented in the form of overall text information, it can still be processed in the manner of "text parsing the first input information input online into the initial language model" in the above embodiment, so as to obtain the required memory-related background information and second environment application information.

[0055] It should be noted that the specific contents of the memory-related background information, online-related background information, first environment application information and second environment application information all need to be determined accordingly according to their actual scenarios. There is no restriction here. It is precisely because of the possible differences between the above-mentioned associated information that the embodiments of the present invention can achieve more effective and reliable online learning optimization, thereby effectively improving the real-time environmental adaptability of the intelligent body, improving the efficiency of scene task processing, and meeting the real-time application needs of users.

[0056] In one embodiment of the present invention, step S1 may include, but is not limited to, the following steps:

[0057] Step S11: performing data cleaning, word segmentation, feature extraction, and cluster analysis on the first input information input online into the initial language model in sequence to obtain online associated background information and first environmental application information.

[0058] In this step, the first input information is first cleaned to remove impurities in the first input information and ensure the accuracy and reliability of subsequent data analysis; further, word segmentation is performed to segment the cleaned first input information according to pre-introduced or set characters or character strings to reduce the difficulty of subsequent data analysis; further, feature extraction is performed, for example, it can be achieved through but not limited to tf-idf in the visualization module, which takes into account both the importance of the extracted vocabulary in the first input information and the prevalence of the extracted vocabulary in the corpus; further, cluster analysis is performed, that is, clustering and merging analysis is performed on some of the selected keywords, and they are effectively displayed under certain conditions to determine the vocabulary association in the first input information; through the above analysis, the associated information in the first input information can be extracted one by one, and the online associated background information and the first environmental application information can be obtained after analysis; of course, the above data processing means are not the only limitation, and more relevant data processing means can be added on this basis, which will not be elaborated here.

[0059] like Figure 2 As shown, in one embodiment of the present invention, step S3 may include, but is not limited to, the following steps:

[0060] Step S31: When it is determined that there is a difference between the online associated context information and the memory associated context information, the online training loss of the initial language model is obtained, and the memory associated context information is forgotten in the initial language model to obtain a first language model;

[0061] Step S32: training the first language model based on the online associated background information to obtain a second language model, and obtaining the online training loss of the second language model;

[0062] Step S33: Obtain the difference between the online training loss of the second language model and the online training loss of the initial language model to obtain a difference loss value;

[0063] Step S34: When the difference loss value is less than or equal to the preset difference threshold, an online learning optimization strategy for the AI ​​Agent is generated according to the second language model, the first environment application information, and the second environment application information.

[0064] In this step, when it is determined that there is a difference between the online associated background information and the memory associated background information, it means that the application scenario in the current preset time window is different from the application scenario in the previous preset time window. If the application scenario in the previous preset time window is still directly analyzed, the result will be less accurate and may cause new problems. Therefore, further optimization is performed based on replacing the memory associated background information with the online associated background information; first, in the initial language model, the memory associated background information is forgotten, with the aim of deleting the memory associated background information and its associated content from the initial language model, so as to lay a good foundation for subsequent retraining; further, the first language model is trained according to the online associated background information to obtain the second language model, with the aim of combining the online associated background information with the memory associated background information after The first language model of the forgotten training is matched to obtain a second language model associated with the online associated background information, and then the difference between the online training loss of the second language model and the online training loss of the initial language model is obtained to determine the loss difference of the second language model relative to the initial language model. When the difference loss value between the two is less than or equal to the preset difference threshold, it indicates that the preliminary training of the second language model based on the online associated background information is completed; since the application scenario has been replaced, it is necessary to further consider the specific impact of the first environment application information in the application scenario, that is, when the difference loss value is less than or equal to the preset difference threshold, further generate an online learning optimization strategy for the AI ​​Agent based on the second language model, the first environment application information and the second environment application information.

[0065] In one embodiment, there is a difference between the online associated context information and the memory associated context information, which can be presented as, but not limited to: there is a difference in the application scenarios between the two. For example, referring to the description in the aforementioned embodiment that "the online associated context information at this time represents that the current preset time window is applied to the daily inquiry scenario", while the corresponding memory associated context information represents that the previous preset time window is applied to the intelligent customer service scenario, it can be determined that there is a difference between the two.

[0066] It should be noted that the specific numerical value of the preset difference threshold can be multiple, which is not limited here and can be determined accordingly according to the actual application scenario; the specific steps and processes such as model training based on relevant data and calculation of training loss are well known to those skilled in the art and will not be described here to avoid redundancy.

[0067] like Figure 3 As shown, in one embodiment of the present invention, the step of "forgetting the memory-related background information to obtain the first language model" in step S31 may include, but is not limited to, the following steps:

[0068] Step S311: Based on the memory mechanism of the AIAgent, search whether there is additional background information different from the online associated background information and the memory associated background information;

[0069] Step S312: When additional context information is found, the initial language model is trained based on the additional context information and the memory-associated context information to obtain a first language model. Alternatively, when no additional context information is found, the initial language model is partially trained based on the memory-associated context information to obtain a first language model.

[0070] In this step, consider introducing additional background information different from online associated background information and memory associated background information as new training data. If the information can be found through the memory mechanism of AIAgent, it means that the initial language model has been associated with untrained additional background information before. In this case, training the initial language model based on the additional background information and the memory associated background information can achieve a good forgetting training effect. It can be understood that if no additional background information is found, it means that the initial language model has not been associated with untrained additional background information before, and the initial language model is locally trained based on the memory associated background information alone.

[0071] like Figure 4 As shown, in one embodiment of the present invention, step S34 may include, but is not limited to, the following steps:

[0072] Step S341: training a second language model at a preset learning rate according to the first environment application information and the second environment application information;

[0073] Step S342: During the training process of the second language model, respectively obtain the distribution of the first environment application information and the second environment application information relative to the training data of the second language model to obtain a first distribution value and a second distribution value;

[0074] Step S343: When the absolute value of the difference between the first distribution value and the second distribution value is less than the preset distribution threshold, the preset learning rate is modified to obtain an optimized learning rate;

[0075] Step S344: Train the second language model with an optimized learning rate according to the first environment application information to obtain a third language model.

[0076] In this step, since the second language model is obtained based on the differential structure between the online associated background information and the memory associated background information, it is necessary to further train again based on the first environment application information and the second environment application information to obtain the required language model. Specifically, first, the second language model is trained at a preset learning rate based on the first input information, with the aim of introducing the first input information into the second language model and strengthening the influence of the first input information in the current application scenario; further, in the training process of the second language model, the distribution degree of the first environment application information and the second environment application information relative to the training data of the second language model is obtained respectively, and the distribution degree represents the training The probability distribution measurement of the first environment application information during training or the first environment application information relative to the training data of the second language model itself can be used to measure the information loss during training through distribution degree measurement. Therefore, when the absolute value of the difference between the first distribution value and the second distribution value is less than the preset distribution threshold, it means that the training data of the second language model meets the requirements. At this time, the preset learning rate is corrected to obtain the optimized learning rate. The optimized learning rate is the learning rate that meets the scenario requirements. Finally, the second language model is trained with the optimized learning rate according to the first environment application information, and the required third language model that meets the scenario requirements is obtained, thereby realizing online learning optimization for the intelligent agent.

[0077] In one embodiment, a specific method for obtaining the degree of distribution may be, but is not limited to, calculating the difference between the proportion distribution of the first environment application information and the proportion distribution of the training data of the second language model, which may be obtained by using a KL divergence operation. The KL divergence is well known to those skilled in the art and will not be described in detail here.

[0078] It should be noted that the initial preset learning rate can be set accordingly according to actual conditions and is not limited here.

[0079] In one embodiment of the present invention, step S343 may include, but is not limited to, the following steps:

[0080] Step S3431: When the first distribution value is greater than or equal to the second distribution value, obtain the difference between the preset learning rate and the preconfigured learning rate adjustment variable to obtain the optimized learning rate, wherein the learning rate adjustment variable is less than the preset learning rate. The learning rate adjustment variable can be set accordingly according to actual conditions and is not limited here;

[0081] or,

[0082] Step S3432: When the first distribution value is less than the second distribution value, obtain the sum of the preset learning rate and the preconfigured learning rate adjustment variable to obtain the optimized learning rate.

[0083] In this step, when the first distribution value is greater than or equal to the second distribution value, it means that the distribution difference corresponding to the first environment application information is greater, that is, the information loss corresponding to the current application scenario is greater. In order to more effectively improve the above information loss, it is necessary to reduce the learning rate to a certain extent and extend the optimization time; conversely, it means that the distribution difference corresponding to the first environment application information is smaller. In this case, increasing the learning rate will help to find the convergence point of the second language model more quickly. In either case, the required third language model can be obtained through training.

[0084] like Figure 5 As shown, in one embodiment of the present invention, step S3 may include, but is not limited to, the following steps:

[0085] Step S35: when it is determined that there is no difference between the online associated context information and the memory associated context information, obtaining the label categories of the first environment application information and the second environment application information respectively;

[0086] Step S36: Based on the initial language model, the first environment application information and the second environment application information, combined with the label category of the first environment application information and the label category of the second environment application information, generate an online learning optimization strategy for the AI ​​Agent.

[0087] In this step, if it is determined that there is no difference between the online associated background information and the memory associated background information, it means that the application scenario of the current preset time window is the same as that of the previous preset time window. Then, at this time, the difference in the specific application situation under the application scenario is mainly considered, that is, by respectively obtaining the label category of the first environment application information and the second environment application information, the label category reflects the specific application content classification of each environment application information, and then based on the initial language model, the first environment application information and the second environment application information, combined with the label category of the first environment application information and the label category of the second environment application information, online learning optimization can be performed.

[0088] like Figure 6 As shown, in one embodiment of the present invention, step S36 may include, but is not limited to, the following steps:

[0089] Step S361: When the label category of the first environment application information is the same as the label category of the second environment application information, an initial language model is trained based on the first environment application information to obtain response result data output by the initial language model;

[0090] Step S362: construct a first reward model based on the self-attention model and the long short-term memory network, and perform supervised fine-tuning on the first reward model according to the response result data to obtain a second reward model;

[0091] Step S363: When multiple response scores of the response result data are outputted by the second reward model, the initial language model is optimized using reinforcement learning based on the multiple response scores to obtain a fourth language model.

[0092] In this step, if the label category of the first environment application information is the same as the label category of the second environment application information, it means that the training framework of the initial language model corresponding to the previous second environment application information is available. On this basis, it is only necessary to introduce the first environment application information to further train the initial language model. Taking into account the further optimization of the application scenarios corresponding to the first environment application information and the second environment application information, the first reward model is constructed to further supervise the learning of the first reward model based on the response result data output by the training initial language model, that is, a more accurate second reward model is obtained based on feedback training through high-quality candidate response data, so that multiple response scores of the response result data are output through the second reward model. Reinforcement learning is used to further optimize the initial language model based on the diversified response scores, which can improve the generation quality and training accuracy of the initial language model and obtain the required fourth language model.

[0093] In one embodiment, the method of constructing the first reward model based on the self-attention model and the long short-term memory network belongs to the relevant existing technology in this field and is not described here to avoid redundancy. The self-attention model can also be replaced by a pointer network, etc.; supervised fine-tuning of the first reward model can be achieved by but not limited to the SFT training method of the current large model.

[0094] In one embodiment, determining whether the tag category of the first environmental application information and the tag category of the second environmental application information are the same can be determined based on actual scenarios. This example is provided here for illustration, but is not intended to be the sole limitation. For example, referring to the first environmental application information in the aforementioned embodiment, which represents the query content information in the daily query scenario "Help me check the weather conditions in place A for the next week", if the second environmental application information represents the query content information "Help me check whether there will be heavy rain in place B on X month X day", that is, the corresponding query content information of the two is different, but both are inquiries about the weather conditions of a certain place, so the AI ​​Agent will determine that the tag categories of the two are the same. In another case, if the second environmental application information represents the query content information "Help me check the airfare from place A to place B for the next week", that is, the corresponding query content information of the two is neither the same nor belongs to the same query category (direction), one is about weather conditions and the other is about airfare, so the AI ​​Agent will determine that the tag categories of the two are different.

[0095] like Figure 7 As shown, in one embodiment of the present invention, step S36 may include, but is not limited to, the following steps:

[0096] Step S364: When the label category of the first environment application information is different from the label category of the second environment application information, forget training is performed on the initial language model according to the second environment application information to obtain a fifth language model;

[0097] Step S365: When the online training loss of the fifth language model is less than or equal to the minimum preset loss threshold, it is determined that the training of the fifth language model is completed;

[0098] Step S366: Perform local training on the pre-trained fifth language model according to the first environment application information to obtain a sixth language model.

[0099] In this step, if the label category of the first environment application information is different from the label category of the second environment application information, it means that the previous training framework of the second environment application information corresponding to the initial language model is unavailable, and the second environment application information needs to be forgotten in the training framework of the initial language model, that is, the initial language model is trained for forgetting according to the second environment application information to obtain a forgotten model, namely the fifth language model. Furthermore, during the training process of the fifth language model, the online training loss of the fifth language model is continuously detected to see whether it reaches the minimum preset loss threshold. When it reaches the minimum preset loss threshold, it can be determined that the training of the fifth language model is completed, and the fifth language model in this case is used as the basic training framework to further introduce the first environment application information for training, thereby obtaining a sixth language model that meets the scenario requirements.

[0100] Figure 8 FIG. 1 is a schematic diagram of the structure of an electronic device 1000 provided by an embodiment of the present invention. Figure 8 As shown, the electronic device 1000 includes a memory 1100 and a processor 1200. The number of the memory 1100 and the processor 1200 can be one or more. Figure 8 In the embodiment, a memory 1100 and a processor 1200 are taken as an example; the memory 1100 and the processor 1200 in the device can be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0101] Memory 1100, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the AIAgent online learning adaptive optimization method provided in any embodiment of the present invention. Processor 1200 implements the AIAgent online learning adaptive optimization method by executing the software programs, instructions, and modules stored in memory 1100.

[0102] The memory 1100 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function. In addition, the memory 1100 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1100 may further include a memory remotely located relative to the processor 1200, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for executing the AIAgent online learning adaptive optimization method provided in any embodiment of the present invention.

[0104] An embodiment of the present invention further provides a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device performs the AI ​​Agent online learning adaptive optimization method provided in any embodiment of the present invention.

[0105] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0106] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0107] As used in this specification, the terms "component," "module," "system," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components can reside in a process or execution thread, and a component can be located on a single computer or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

Claims

1. An AIAgent online learning adaptive optimization method, characterized in that: The AI ​​Agent configures an initial language model, and the method includes: Performing text parsing on first input information input online into the initial language model to obtain online associated background information and first environment application information, wherein the first input information corresponds to a current preset time window; Based on the memory mechanism of the AIAgent, acquiring the memory-associated background information and the second environment application information of the initial language model, wherein the second environment application information corresponds to the previous preset time window; According to the difference between the online associated background information and the memory associated background information, combined with the first environment application information, the second environment application information and the initial language model, an online learning optimization strategy for the AI ​​Agent is generated.

2. The AIAgent online learning adaptive optimization method according to claim 1, characterized in that: The generating of an online learning optimization strategy for the AI ​​Agent based on the difference between the online associated context information and the memory associated context information, in combination with the first environment application information, the second environment application information, and the initial language model, includes: When it is determined that there is a difference between the online associated context information and the memory associated context information, obtaining an online training loss of the initial language model, and performing forgetting processing on the memory associated context information in the initial language model to obtain a first language model; Training the first language model according to the online associated background information to obtain a second language model, and obtaining an online training loss of the second language model; Obtaining a difference between the online training loss of the second language model and the online training loss of the initial language model to obtain a difference loss value; When the difference loss value is less than or equal to a preset difference threshold, an online learning optimization strategy for the AI ​​Agent is generated according to the second language model, the first environment application information, and the second environment application information.

3. The AIAgent online learning adaptive optimization method according to claim 2, characterized in that: The generating of an online learning optimization strategy for the AI ​​Agent based on the second language model, the first environment application information, and the second environment application information includes: training the second language model at a preset learning rate according to the first environment application information and the second environment application information; During the training process of the second language model, respectively obtaining the distribution of the first environment application information and the second environment application information relative to the training data of the second language model to obtain a first distribution value and a second distribution value; When the absolute value of the difference between the first distribution value and the second distribution value is less than a preset distribution threshold, modifying the preset learning rate to obtain an optimized learning rate; The second language model is trained at the optimized learning rate according to the first environment application information to obtain a third language model.

4. The AIAgent online learning adaptive optimization method according to claim 3, characterized in that: The step of modifying the preset learning rate to obtain an optimized learning rate includes: When the first distribution value is greater than or equal to the second distribution value, obtaining a difference between the preset learning rate and a preconfigured learning rate adjustment variable to obtain an optimized learning rate, wherein the learning rate adjustment variable is less than the preset learning rate; or, When the first distribution value is less than the second distribution value, the sum of the preset learning rate and a preconfigured learning rate adjustment variable is obtained to obtain an optimized learning rate.

5. The AIAgent online learning adaptive optimization method according to claim 1, characterized in that: The generating of an online learning optimization strategy for the AI ​​Agent based on the difference between the online associated context information and the memory associated context information, in combination with the first environment application information, the second environment application information, and the initial language model, includes: When it is determined that there is no difference between the online associated context information and the memory associated context information, respectively obtaining label categories of the first environment application information and the second environment application information; Based on the initial language model, the first environment application information and the second environment application information, combined with the label category of the first environment application information and the label category of the second environment application information, an online learning optimization strategy for the AI ​​Agent is generated.

6. The AIAgent online learning adaptive optimization method according to claim 5, characterized in that: The generating of an online learning optimization strategy for the AI ​​Agent based on the initial language model, the first environment application information, and the second environment application information, in combination with the label category of the first environment application information and the label category of the second environment application information, includes: When the label category of the first environment application information is the same as the label category of the second environment application information, training the initial language model according to the first environment application information to obtain response result data output by the initial language model; Constructing a first reward model based on a self-attention model and a long short-term memory network, and performing supervised fine-tuning on the first reward model according to the response result data to obtain a second reward model; In a case where a plurality of response scores of the response result data are outputted by the second reward model, reinforcement learning is used to optimize the initial language model based on the plurality of response scores to obtain a fourth language model.

7. The AIAgent online learning adaptive optimization method according to claim 5, characterized in that: The generating of an online learning optimization strategy for the AI ​​Agent based on the initial language model, the first environment application information, and the second environment application information, in combination with the label category of the first environment application information and the label category of the second environment application information, includes: When the label category of the first environment application information is different from the label category of the second environment application information, performing forgetting training on the initial language model according to the second environment application information to obtain a fifth language model; When the online training loss of the fifth language model is less than or equal to a minimum preset loss threshold, determining that the training of the fifth language model is completed; The pre-trained fifth language model is locally trained according to the first environment application information to obtain a sixth language model.

8. The AIAgent online learning adaptive optimization method according to claim 1, characterized in that: The performing text parsing on the first input information input online into the initial language model to obtain online associated background information and first environment application information includes: The first input information input online into the initial language model is sequentially subjected to data cleaning, word segmentation processing, feature extraction and cluster analysis to obtain online associated background information and first environmental application information.

9. The AIAgent online learning adaptive optimization method according to claim 2, characterized in that: The forgetting process of the memory-associated background information to obtain a first language model includes: Based on the memory mechanism of the AI ​​Agent, searching for additional background information different from the online associated background information and the memory associated background information; When the additional context information is found, the initial language model is trained according to the additional context information and the memory-associated context information to obtain a first language model; or, when the additional context information is not found, the initial language model is partially trained according to the memory-associated context information to obtain a first language model.

Images