Model updating method and device based on artificial intelligence, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a model updating method and device based on artificial intelligence, equipment and a medium, and the method comprises the steps: obtaining to-be-added data, and carrying out the data recall processing from an original model database according to the to-be-added data, so as to obtain an incremental data set containing the to-be-added data and the recall data; training the incremental data set by using a fine tuning training algorithm to update the original model; and performing preference optimization processing on the updated original model to obtain a target model preferred to the to-be-added data. Through incremental training and preference optimization processing, when the model is updated, forgetting of other old knowledge can be avoided, the old knowledge can be completely replaced for specific knowledge of the to-be-added data, the model does not need to be retrained, and training time and training resources can be effectively saved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and natural language processing technology, and in particular to an artificial intelligence-based model updating method, device, equipment and medium. Background Art

[0002] In recent years, with the development of big data and the advancement of deep learning, artificial intelligence has gradually entered the public eye and gradually penetrated into people's lives. At present, based on the development of the field of artificial intelligence, intelligent information interaction platforms based on natural language processing (NLP) and deep learning (DL) have been applied to various fields of life. As we all know, with the development of time, the knowledge in the model database of the intelligent information interaction platform needs to be updated. For example, at the end of August 22, the age of the approved population for HPV vaccine was expanded from the original 16-26 years old to 9-45 years old. However, for the models trained before August 22, the knowledge in the model always remains at the original approved age, which brings inconvenience to users in actual use.

[0003] In the prior art, when a model is updated, the old knowledge in the original training data is usually replaced with the new knowledge data to be added, and the large model is retrained. This method requires a lot of training resources and is time-consuming, which is a huge burden for most companies. Summary of the invention

[0004] The present invention provides an artificial intelligence model updating method, device, computer equipment and medium to solve the technical problem that the existing model needs to be retrained when updating, which consumes a lot of training resources and time.

[0005] In a first aspect, a model updating method based on artificial intelligence is provided, comprising:

[0006] Acquire the data to be added, and perform data recall processing from the original model database according to the data to be added, so as to obtain an incremental data set including the data to be added and the recalled data;

[0007] Using a fine-tuning training algorithm to train the incremental data set to update the original model;

[0008] The updated original model is subjected to preference optimization processing to obtain a target model that is preferred over the data to be added.

[0009] In a second aspect, a model updating device based on artificial intelligence is provided, comprising:

[0010] A recall module is used to perform data recall processing from the original model database according to the acquired data to be added, so as to obtain an incremental data set including the data to be added and the recalled data;

[0011] A fine-tuning update module, used to train the incremental data set using a fine-tuning training algorithm to update the original model;

[0012] The preference optimization module is used to perform preference optimization processing on the updated original model to obtain a target model that is preferred to the data to be added.

[0013] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned model updating method when executing the computer program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned model updating method are implemented.

[0015] In the scheme implemented by the above-mentioned artificial intelligence-based model updating method, device, computer equipment and storage medium, data recall processing can be performed from the original model database according to the acquired data to be added to obtain an incremental data set containing the data to be added and the recalled data; and the incremental data set is trained using a fine-tuning training algorithm to update the original model; and the updated original model is subjected to preference optimization processing to obtain a target model that prefers the data to be added. In the present invention, when updating the knowledge of business models such as finance, insurance and medical care, training time and training resources are saved through incremental training. First, data recall processing is performed on the data to be added from the original model database, and then incremental fine-tuning training is performed on the incremental data set containing the data to be added and the recalled data obtained after the data recall processing to update the original model to obtain a model containing the incremental data, which can avoid the original other old knowledge from being forgotten, and then the model is subjected to preference optimization processing to obtain a target model that prefers the data to be added, so as to ensure that after the knowledge is updated, the model completely replaces the old knowledge in the specific knowledge of the data to be added, so that the model does not need to be retrained, which can effectively save training time and training resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 is a schematic diagram of an application environment of a model updating method in one embodiment of the present invention;

[0018] Figure 2 is a flow chart of a model updating method in one embodiment of the present invention;

[0019] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S10;

[0020] Figure 4 is a structural schematic diagram of a model updating device in one embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0022] Figure 6 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The artificial intelligence-based model updating method provided in the embodiment of the present invention can be applied in Figure 1In an application environment, a client communicates with a server through a network. The server can periodically obtain the data to be added from the network, or obtain the data to be added through the client. The server can perform data recall processing from the original model database according to the data to be added to obtain an incremental data set containing the data to be added and the recalled data; and use a fine-tuning training algorithm to train the incremental data set to update the original model to obtain a model containing the incremental data; the model containing the incremental data is subjected to preference optimization processing to obtain a target model that prefers the data to be added. In the present invention, when updating the knowledge of the business model in the intelligent information interaction platform such as finance, insurance and medical care, the training time and training resources can be saved through incremental training. First, the data to be added is processed from the original model database for data recall processing, and then the incremental data set obtained after the data recall processing is subjected to incremental fine-tuning training to obtain a model containing the incremental data, which can avoid forgetting the original old knowledge, and then the model is subjected to preference optimization processing to obtain a target model that prefers the data to be added, so that the target model can completely replace the old knowledge in the specific knowledge of the data to be added, so that the model does not need to be retrained, which can effectively save training time and training resources and improve training efficiency. Understandably, after the model is updated, the server can also receive user question and answer information through the client, and display the corresponding answer knowledge through the client. In the present invention, the client can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0025] See also Figure 2 As shown, Figure 2 A flowchart of a model updating method based on artificial intelligence provided by an embodiment of the present invention includes the following steps S10-S30:

[0026] S10: Acquire the data to be added, and perform data recall processing from the original model database according to the data to be added to obtain an incremental data set including the data to be added and the recalled data.

[0027] The model updating method provided by the present invention can be applied to the intelligent information interaction platform in various application scenarios, and is usually implemented through a server, which can obtain the data to be added in real time. For example, in the field of banking applications, users often consult through online chats, and often need to use the intelligent information interaction platform to reply to some questions about financial products and time deposit interest rates of customers, and these issues may be different in each stage as time goes by. For example, the time deposit interest rate fluctuates every year, and the intelligent information interaction platform needs to obtain the data to be added in real time, and perform data recall processing on the data to be added to obtain an incremental data set, so as to train the incremental data set later, saving resources and time. Among them, the data to be added can be the latest data of the relevant knowledge in the model, and the data recall processing refers to the use of a preset recall algorithm to perform recall processing from the database of the original model according to the data to be added, and obtain an incremental data set containing the data to be added and the recalled data.

[0028] It should be understood that the data recall here refers to the process of finding data whose correlation with the data to be added meets the preset standard from the database of the original model, and the screened data whose correlation with the data to be added meets the preset standard is used as the recalled data, and the original model is the current model. The model update method of the present invention can be used to update the knowledge in the current model to improve the user experience. Preferably, in this embodiment, after obtaining the data to be added, the data to be added is preprocessed to obtain data for training. Specifically, prompt technology is used to convert each piece of data to be added into multiple similar pieces of data to be added to form a training set, covering more possibilities. Subsequent incremental fine-tuning training and other training are performed on the data to be added in the training set to improve the accuracy of model updating.

[0029] It should be noted that prompt technology is a technology based on artificial intelligence instructions, which can generate text that meets user needs through the output of a clear and specific guiding language model. In the present invention, the pre-trained prompt model is used to pre-process each piece of data to be added and convert it into multiple similar pieces of data to be added. It can be understood that the prompt model can be pre-trained through a oneshot (single sample learning) method by giving an example of converting one piece of data into multiple similar training data.

[0030] Among them, Figure 3 As shown, in step S10, data recall processing is performed from the original model database according to the data to be added to obtain an incremental data set containing the data to be added and the recalled data, including the following steps S11-S13:

[0031] S11: performing vector conversion processing on the data to be added to obtain a corresponding vector.

[0032] In this step, the BERT algorithm can be used to vectorize the data to be added to obtain a corresponding vector, which can be a semantic vector. It can be understood that in some other embodiments, other vectorization algorithms based on neural networks (Neural Networks) can also be used to realize the vectorization of the data to be added.

[0033] S12: Compare the vector with the vector corresponding to the data in the original model database, so as to recall the data in the original model database whose similarity meets a preset threshold as the recalled data.

[0034] In this step, specifically, the recall similarity between the vector and the vector corresponding to each data in the original model database is calculated by using the pre-trained similarity retrieval model. When the recall similarity is greater than a preset threshold, the corresponding data in the original model database is used as the recall data. In this embodiment, the similarity retrieval model can adopt the Faiss model, which improves efficiency through its efficient index structure and search algorithm, so as to quickly find multiple vectors in the original model database that are most similar to the corresponding vector obtained in the above step S11.

[0035] S13: Taking the union of the data to be added and the recalled data to obtain the incremental data set.

[0036] For steps S11-S13, after obtaining the data to be added, the data to be added must first be vectorized to obtain the vector of the data to be added, and the similarity between the vector of the data to be added and the vector corresponding to each data in the original model database is calculated to obtain the recall similarity. When the recall similarity is greater than the preset threshold, the corresponding data in the original model database is used as the recall data, and the data to be added and the recall data are combined to form an incremental data set. Among them, the original model database is a database that is continuously accumulated according to the business scenario, including various types of knowledge in the business scenario, such as the application field of the insurance business scenario. The database may include various insurance questions and related answers in the insurance business scenario, for example, the insurance question may be: "What is the commission of Ping An Ping An Fu 23", "What protection content does Ping An Ping An Fu 23 include", etc. The related answer may refer to the answer written by the expert to the question, or the intelligent answer generated by the question and answer in the previous intelligent information interaction platform, which is not limited here. Among them, the above-mentioned Ping An Ping An Fu refers to the name of an insurance product.

[0037] S20: Train the incremental data set using a fine-tuning training algorithm to update the original model.

[0038] After obtaining the incremental data set, in order to ensure that other old knowledge is not forgotten, to achieve new knowledge update, and to replace the old knowledge to the greatest extent, the present invention performs incremental fine-tuning training on the incremental data set. Specifically, in some embodiments, according to the resource situation and time consumption, Qlora, lora or full-sft fine-tuning large model is selected to train the incremental data set to update the original model, wherein the resource situation refers to the resource situation of the training resources, and the training resources specifically refer to the hardware (such as running memory, etc.) and software version of the GPU, CPU, etc. When the fine-tuning large model is running, the running memory of the GPU and CPU is consumed. The consumption of the GPU and CPU is arranged from large to small as full-sft fine-tuning large model>lora fine-tuning large model>Qlora fine-tuning large model. The above-mentioned different fine-tuning large models can be selected to train the incremental data set according to the actual resource situation and the specific requirements for the running time.

[0039] S30: Performing preference optimization processing on the updated original model to obtain a target model that is preferred to the data to be added.

[0040] After completing the incremental fine-tuning training in the previous step, compared to the original model that can only generate answers to old knowledge, the current model trained through incremental fine-tuning has both old knowledge and new knowledge corresponding to the newly added data. However, in actual use, the updated original model has a certain probability of generating both old knowledge and new knowledge during the generation process. For example, in medical and health application scenarios, the age of the approved applicable population for HPV vaccines was expanded from 16-26 years old to 9-45 years old at the end of August 22. After updating the model using the above steps, the updated original model has a certain probability of generating two knowledge responses of approved applicable population ages 16-26 and 9-45 years old when asking and answering questions, which brings inconvenience to users in actual use. Therefore, in an embodiment of the present invention, after incremental fine-tuning training of the incremental data set, in order to ensure that the replacement of old and new knowledge is completely completed, the original model updated by incremental fine-tuning training is also subjected to preference optimization processing.

[0041] Specifically, in some embodiments, the updated original model is subjected to preference optimization processing using a direct preference optimization algorithm until the loss function converges to obtain a target model that is preferred to the data to be added. In this embodiment, the direct preference optimization algorithm uses a DPO (Direct Preference Optimization) algorithm, and the loss function is:

[0042]

[0043] In the above formula, π θ is the original model after the above update, π ref is the reference model, which can be the initial SFT model, πθ (y w |x) is the probability of the updated original model generating a new knowledge answer when the input is x, π θ (y l |x) is the probability of the updated original model generating an old knowledge answer when the input is x, π ref (y w |x) is the probability of the reference model generating a new knowledge answer when the input is x, π ref (y l |x) is the probability of the reference model generating an old knowledge answer when the input is x, is the reward function, D is a dataset containing input and preference pairs, and β is a regularization parameter that controls how much the policy deviates from the reference policy.

[0044] After the above-mentioned direct preference optimization training, it can be ensured that the target model prefers new knowledge in the subsequent question and answer generation, and the purpose of completely replacing old knowledge with new knowledge can be achieved in the specific knowledge of the data to be added. That is, corresponding to the question of the age of the approved applicable population for the HPV vaccine in the above-mentioned medical and health application scenario, the target model updated by the model updating method of the present invention will generate the latest knowledge response, that is, generate the knowledge response that the approved applicable population age is 9-45 years old. It can be understood that in some other embodiments, preference optimization algorithms such as KTO can also be used to perform preference optimization processing on the updated original model to achieve complete replacement of new and old knowledge.

[0045] It can be seen that in the above scheme, when updating the knowledge of business models in intelligent information interaction platforms such as finance, insurance and medical care, the data to be added is first recalled from the original model database, and then the incremental data set containing the data to be added and the recalled data obtained after the data recall processing is trained using the fine-tuning training algorithm to update the original model. By performing preference optimization processing on the updated original model, a target model that prefers the data to be added is obtained, so that when the model is updated, the forgetting of other old knowledge can be avoided, and the specific knowledge of the data to be added can be completely replaced by the old knowledge, and there is no need to retrain the model, which can effectively save training time and training resources.

[0046] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0047] In one embodiment, a model updating device based on artificial intelligence is provided, and the model updating device based on artificial intelligence corresponds to the model updating method based on artificial intelligence in the above embodiment. Figure 4As shown, the model updating device includes a recall module 101, a fine-tuning update module 102 and a preference optimization module 103. The functional modules are described in detail as follows:

[0048] The recall module 101 is used to perform data recall processing from the original model database according to the acquired data to be added, so as to obtain an incremental data set containing the data to be added and the recalled data; wherein, the original model is the current model, and the data to be added can be the latest data of the relevant knowledge in the model, and the data recall processing refers to using a preset recall algorithm to perform recall processing from the database of the current model according to the data to be added, so as to obtain an incremental data set containing the data to be added and the recalled data. It should be understood that the data recall here refers to the process of finding data whose correlation with the data to be added meets the preset criteria from the database of the current model, and using the screened data whose correlation with the data to be added meets the preset criteria as the recalled data, to obtain an incremental data set for incremental fine-tuning training, so as to facilitate subsequent training on the incremental data set, saving resources and time;

[0049] A fine-tuning update module 102, configured to train the incremental data set using a fine-tuning training algorithm to update the original model;

[0050] The preference optimization module 103 is used to perform preference optimization processing on the updated original model to obtain a target model that is preferred to the data to be added.

[0051] In one embodiment, the recall module 101 is specifically configured to:

[0052] Performing vector conversion processing on the data to be added to obtain a corresponding vector;

[0053] Compare the vector with the vector corresponding to the data in the original model database, and use the data in the original model database whose recall similarity meets the preset threshold as the recall data; specifically, use the pre-trained similarity retrieval model to traverse and calculate the recall similarity between the vector and the vector corresponding to each data in the original model database, and when the recall similarity is greater than the preset threshold, the corresponding data in the original model database is used as the recall data;

[0054] The union of the data to be added and the recalled data is taken to obtain the incremental data set.

[0055] In one embodiment, the recall module 101 is further used for:

[0056] Use prompt technology to convert each piece of data to be added into multiple similar pieces of data to be added.

[0057] In one embodiment, the fine-tuning update module 102 is specifically configured to:

[0058] According to the resource situation and time consumption, Qlora, lora or full-sft is selected to fine-tune the large model and train the incremental data set to update the original model.

[0059] In one embodiment, the preference optimization module 103 is specifically configured to:

[0060] The updated original model is subjected to preference optimization processing using a direct preference optimization algorithm until the loss function converges to obtain a target model that is preferred to the data to be added; wherein the loss function is:

[0061]

[0062] In the above formula, π θ is the original model after the above update, π ref is the reference model, which can be the initial SFT model, π θ (y w |x) is the probability of the updated original model generating a new knowledge answer when the input is x, π θ (y l |x) is the probability of the updated original model generating an old knowledge answer when the input is x, π ref (y w |x) is the probability of the reference model generating a new knowledge answer when the input is x, π ref (y l |x) is the probability of the reference model generating an old knowledge answer when the input is x, is the reward function, D is a dataset containing input and preference pairs, and β is a regularization parameter that controls how much the policy deviates from the reference policy.

[0063] The present invention provides a model updating device, which first performs data recall processing from an original model database according to data to be added, and then uses a fine-tuning training algorithm to train an incremental data set containing data to be added and recalled data obtained after the data recall processing, so as to update the original model and obtain a model containing incremental data, which can avoid forgetting of original old knowledge, and then obtain a target model that prefers the data to be added by performing preference optimization processing on the updated original model, so that the target model can completely replace the old knowledge in the specific knowledge of the data to be added, thereby achieving the purpose of avoiding the forgetting of other old knowledge when updating the model, and completely replacing the old knowledge in the specific knowledge of the data to be added, and there is no need to retrain the model, which greatly saves training time and training resources.

[0064] For the specific definition of the model updating device, please refer to the definition of the model updating method above, which will not be repeated here. Each module in the above-mentioned model updating device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0065] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a model update method based on artificial intelligence.

[0066] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a model updating method based on artificial intelligence.

[0067] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0068] Acquire the data to be added, and perform data recall processing from the original model database according to the data to be added, so as to obtain an incremental data set including the data to be added and the recalled data;

[0069] Using a fine-tuning training algorithm to train the incremental data set to update the original model;

[0070] The updated original model is subjected to preference optimization processing to obtain a target model that is preferred over the data to be added.

[0071] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0072] Acquire the data to be added, and perform data recall processing from the original model database according to the data to be added, so as to obtain an incremental data set including the data to be added and the recalled data;

[0073] Using a fine-tuning training algorithm to train the incremental data set to update the original model;

[0074] The updated original model is subjected to preference optimization processing to obtain a target model that is preferred over the data to be added.

[0075] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0077] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0078] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A model updating method based on artificial intelligence, characterized in that: include: Acquire the data to be added, and perform data recall processing from the original model database according to the data to be added, so as to obtain an incremental data set including the data to be added and the recalled data; Using a fine-tuning training algorithm to train the incremental data set to update the original model; The updated original model is subjected to preference optimization processing to obtain a target model that is preferred over the data to be added.

2. The artificial intelligence-based model updating method according to claim 1, characterized in that: The data recall processing is performed from the original model database according to the data to be added to obtain an incremental data set containing the data to be added and the recalled data, including: Performing vector conversion processing on the data to be added to obtain a corresponding vector; Compare the vector with the vector corresponding to the data in the original model database to recall the data in the original model database whose similarity meets a preset threshold as the recalled data; The union of the data to be added and the recalled data is taken to obtain the incremental data set.

3. The artificial intelligence-based model updating method according to claim 2, characterized in that: The step of comparing the vector with the vector corresponding to the data in the original model database to recall the data in the original model database whose similarity meets a preset threshold as the recalled data includes: The pre-trained similarity retrieval model is used to traverse and calculate the recall similarity between the vector and the vector corresponding to each data in the original model database. When the recall similarity is greater than a preset threshold, the corresponding data in the original model database is used as the recalled data.

4. The artificial intelligence-based model updating method according to claim 1, characterized in that: After obtaining the data to be added, the method further includes: Use prompt technology to convert each piece of data to be added into multiple similar pieces of data to be added.

5. The artificial intelligence-based model updating method according to claim 1, characterized in that: The performing preference optimization processing on the updated original model to obtain a target model that is preferred over the data to be added includes: The updated original model is optimized using a direct preference optimization algorithm until the loss function converges to obtain a target model that is more favorable to the data to be added.

6. The artificial intelligence-based model updating method according to claim 5, characterized in that: The convergence function is: Among them, π θ is the updated original model, π ref is the reference model, π θ (y w |x) is the probability of the updated original model generating a new knowledge answer when the input is x, π θ (y l |x) is the probability of the updated original model generating an old knowledge answer when the input is x, π ref (y w |x) is the probability of the reference model generating a new knowledge answer when the input is x, π ref (y l |x) is the probability of the reference model generating an old knowledge answer when the input is x, is the reward function and β is the regularization parameter.

7. The artificial intelligence-based model updating method according to claim 1, characterized in that: The step of training the incremental data set using a fine-tuning training algorithm to update the original model includes: According to the resource situation and time consumption, Qlora, Lora or full-sft is selected to fine-tune the large model and train the incremental data set to update the original model.

8. A model updating device based on artificial intelligence, characterized in that: include: A recall module is used to perform data recall processing from the original model database according to the acquired data to be added, so as to obtain an incremental data set including the data to be added and the recalled data; A fine-tuning update module, used to train the incremental data set using a fine-tuning training algorithm to update the original model; The preference optimization module is used to perform preference optimization processing on the updated original model to obtain a target model that is preferred to the data to be added.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the model updating method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the model updating method according to any one of claims 1 to 7 are implemented.

Images