Dialogue model training method and device, equipment and medium

Abstract

The invention discloses a dialogue model training method and device, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: adjusting the importance weight of each current memory unit in a preset memory library through a memory module based on a dynamic weight strategy, filtering the memory units with low weights according to the magnitude order of the importance weights, and obtaining a dialogue model training result; updating the preset memory bank to obtain a current preset memory bank; when the pre-training model receives a training request initiated by a user, outputting a plurality of target similarity calculation results meeting a similarity condition; taking a memory unit corresponding to each target similarity calculation result as a target memory unit, performing weighted aggregation on each target memory unit and the query vector to train a memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and inputting the aggregated memory information into a pre-training model, and the pre-training model generates response information corresponding to the query request according to the aggregated memory information and the query vector.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a dialogue model training method, device, equipment and medium. Background Art

[0002] Large language models are usually trained based on static, predefined data sets, and model parameters are fixed after training. This design faces significant challenges in an era of rapid knowledge updates, including dynamic changes in data, tasks, and user preferences. For example, information in fields such as news, scientific research, or technical specifications is updated every day, and traditional large dialogue models have difficulty adapting to these changes quickly.

[0003] Existing knowledge updating methods mainly rely on retraining or incremental fine-tuning, but both methods have significant defects. Retraining requires huge computing resources and time costs. In addition, frequent parameter updates can easily lead to "catastrophic forgetting", that is, the model loses old knowledge when learning new knowledge. Although incremental fine-tuning has lower resource requirements, it usually requires a large amount of task-specific data, and its adaptability and knowledge retention capabilities are limited.

[0004] In summary, how to improve the knowledge adaptability of the model, reduce training costs and avoid catastrophic forgetting are technical problems to be solved in this field. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a dialogue model training method, device, equipment and medium, which can improve the knowledge adaptability of the model, reduce training costs and avoid catastrophic forgetting. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a dialogue model training method, comprising:

[0007] The importance weight of each current memory unit in the preset memory bank is adjusted through the memory module of the pre-trained model and based on the dynamic weight strategy, and the memory units with low weight are filtered in the order of the importance weight to update the preset memory bank and obtain the current preset memory bank;

[0008] When the pre-trained model receives a training request initiated by a user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen a number of target similarity calculation results that meet the similarity condition;

[0009] The memory unit corresponding to each target similarity calculation result is used as the target memory unit, and the target memory unit and the query vector are weightedly aggregated through a cross-attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and the aggregated memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregated memory information and the query vector.

[0010] Optionally, before adjusting the importance weight of each current memory unit in the preset memory bank based on the dynamic weight strategy through the memory module of the pre-trained model, the method further includes:

[0011] Each document used for training the model is compressed into a compact vector one by one, and each of the compact vectors is stored as a memory unit in a preset memory bank.

[0012] Optionally, compressing each document used for training the model into a compact vector one by one includes:

[0013] Obtaining a number of documents for training the model, and mapping characters of each of the documents through word vector mapping to obtain mapped documents;

[0014] Each of the mapped documents and the corresponding preset vector markers are compressed by a compression module to obtain a compact vector with a preset length; wherein the preset vector marker is a vector used to enhance the document expression capability of the mapped document.

[0015] Optionally, before adjusting the importance weight of each current memory unit in the preset memory bank based on the dynamic weight strategy through the memory module of the pre-trained model, the method further includes:

[0016] A dynamic weight strategy is constructed according to the similarity between the current memory unit and the current query vector, the corresponding first hyperparameter, the time information of the current memory unit, and the corresponding second hyperparameter.

[0017] Optionally, the weighted aggregation of each of the target memory units and the query vector by a cross attention mechanism to train the memory module to output corresponding aggregated memory information based on each of the target memory units and the query vector includes:

[0018] Aggregate all the target memory units through a cross-attention mechanism to obtain a global aggregation result;

[0019] Aggregate all the target memory units according to the same field or the same subject through a cross-attention mechanism to obtain a local aggregation result;

[0020] The global aggregation result and the local aggregation result are linearly weighted combined to obtain aggregated memory information for training the memory module.

[0021] Optionally, after the aggregate memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregate memory information and the query vector, the method further includes:

[0022] When the user determines that the response information is an error response, the corresponding error information is fed back to the pre-trained model to notify the memory module to locate the relevant memory unit corresponding to the response information;

[0023] According to the inference process of generating the response information based on the pre-trained model, it is determined that there is an erroneous memory unit in the relevant memory unit, and the erroneous memory unit is deleted from the current preset memory library to update the current preset memory library.

[0024] Optionally, the dialogue model training method further includes:

[0025] If there are multiple query requests at the same time, extract the memory units related to each query request from the current preset memory bank respectively;

[0026] Predicting responses to each query request using the pre-trained model and based on the memory units corresponding to each query request, so as to output corresponding predicted response information;

[0027] Calculate the loss value between the predicted response information and the corresponding true response information according to the task loss function of the pre-trained model;

[0028] The model parameters of the pre-trained model are trained and optimized with the minimum weighted sum of the loss values ​​of each query request as the optimization goal until the training stop condition is met to obtain the target model.

[0029] In a second aspect, the present application discloses a dialogue model training device, comprising:

[0030] A library update module, used to adjust the importance weight of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, filter the low-weight memory units in the order of the importance weight, so as to update the preset memory library and obtain the current preset memory library;

[0031] A similarity calculation module, configured to convert the query request information in the training request into a query vector when the pre-trained model receives a training request initiated by the user, and use the pre-trained model to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen a number of target similarity calculation results that meet the similarity condition;

[0032] A training module is used to use the memory unit corresponding to each target similarity calculation result as a target memory unit, and weightedly aggregate each target memory unit and the query vector through a cross-attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and input the aggregated memory information into the pre-trained model so that the pre-trained model generates response information corresponding to the query request based on the aggregated memory information and the query vector.

[0033] In a third aspect, the present application discloses an electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed dialogue model training method.

[0036] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed dialogue model training method are implemented.

[0037] It can be seen that the present application discloses a dialogue model training method, including: adjusting the importance weights of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, filtering the memory units with low weights in the order of the importance weights to update the preset memory library and obtain the current preset memory library; when the pre-trained model receives a training request initiated by the user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen several target similarity calculation results that meet the similarity condition; the memory unit corresponding to each of the target similarity calculation results is used as the target memory unit, and the cross-attention mechanism is used to weightedly aggregate each of the target memory units and the query vector to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and the aggregated memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregated memory information and the query vector. It can be seen that by adjusting the importance weights of memory units and filtering low-weight memory units through dynamic weight strategies, the preset memory library can always maintain the most valuable knowledge for the current task. The query request information in the user training request is converted into a query vector, and then the similarity with each memory unit in the updated memory library is calculated to screen the target similarity result, which improves the accuracy of retrieval. The target memory unit and the query vector are weightedly aggregated using the cross-attention mechanism, and the training memory module outputs aggregated memory information, integrating memory units of different granularities and sources. Finally, the continuous training process promotes the coordinated optimization of the memory module and the pre-trained model. The memory module continuously learns to optimize the memory storage, retrieval and aggregation strategies. The pre-trained model generates more accurate responses based on high-quality aggregated memory. The two feedback and adjust each other, gradually reducing the probability of generating wrong answers, improving the overall accuracy, reliability and practicality of the model, and allowing the model to always maintain good performance in long-term operation and knowledge updating. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flow chart of a dialogue model training method disclosed in this application;

[0040] Figure 2A flow chart of a large language model continuous learning method based on compressed memory training disclosed in this application;

[0041] Figure 3 A structural schematic diagram of a dialogue model training device disclosed in this application;

[0042] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] Large language models are usually trained based on static, predefined data sets, and model parameters are fixed after training. This design faces significant challenges in an era of rapid knowledge updates, including dynamic changes in data, tasks, and user preferences. For example, information in fields such as news, scientific research, or technical specifications is updated every day, and traditional large dialogue models have difficulty adapting to these changes quickly.

[0045] Existing knowledge updating methods mainly rely on retraining or incremental fine-tuning, but both methods have significant defects. Retraining requires huge computing resources and time costs. In addition, frequent parameter updates can easily lead to "catastrophic forgetting", that is, the model loses old knowledge when learning new knowledge. Although incremental fine-tuning has lower resource requirements, it usually requires a large amount of task-specific data, and its adaptability and knowledge retention capabilities are limited.

[0046] To this end, the present invention provides a dialogue model training scheme, which can improve the knowledge adaptability of the model, reduce training costs and avoid catastrophic forgetting.

[0047] Reference Figure 1 As shown, an embodiment of the present invention discloses a dialogue model training method, comprising:

[0048] Step S11: adjust the importance weights of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, filter the low-weight memory units in order of the importance weights to update the preset memory library and obtain the current preset memory library.

[0049] In this embodiment, before the importance weight of each current memory unit in the preset memory bank is adjusted based on the dynamic weight strategy through the memory module of the pre-trained model, it also includes: compressing each document used for training the model into a compact vector one by one, and storing each compact vector as a memory unit in the preset memory bank. Specifically, a number of documents used for training the model are obtained, and the characters of each document are mapped through word vector mapping to obtain a mapped document; each mapped document and the corresponding preset vector mark are compressed through a compression module to obtain a compact vector of a preset length; wherein the preset vector mark is a vector used to enhance the document expression ability of the mapped document.

[0050] It is understandable that the input document is ,Include A word or symbol, ,in, For the document A word or symbol, Indicates the total number of words or symbols in a document. Then use word embedding technology to map each word to a vector space of fixed dimension, so that each word There are corresponding word embedding representations , these words are embedded as the initial representation of the document, so the document after mapping is At the same time, in order to enhance the expressiveness of the document, a set of learnable virtual tags is added , the virtual tag representation is a learnable vector of fixed length. In this way, because the document is represented by the original word embedding in the document alone, there will be information missing or insufficient representation. Virtual tags can be regarded as a kind of supplementary information, which can capture some potential semantic relationships or structural information in the document that are difficult to reflect through word embedding, so as to accurately describe the document content for better subsequent compression.

[0051] Furthermore, the mapped documents and virtual tags are jointly compressed by introducing a compression module. Specifically, the compression module is a trainable module, and the internal structure of the compression module is implemented based on neural network architectures. For example, it contains multiple fully connected layers, attention mechanisms, etc., and extracts and integrates key information by performing linear transformation, nonlinear activation, and feature interaction operations on the input word embedding and virtual tags.

[0052] Specifically, after the compression module After the internal layers are operated, the goal is to convert the document information originally represented by many word embeddings and virtual tags into a compact vector of fixed length. , the conversion formula is:

[0053] ;

[0054] in, Indicates The dimension of this compact vector is much smaller than the word embedding representation dimension of the original document, that is, ,in, represents the compressed length of the document, and , Represents the dimensions of the compact vector.

[0055] For example, after word embedding, the original document is a vector sequence with a large dimension and a length that depends on the number of words in the document. After being processed by the compression module, it becomes a compact vector with a fixed length (such as 50 or 100 dimensions, depending on the model design). This compact vector retains the key semantic information of the document while greatly reducing the amount of data, making it easier to store, operate, and use in the memory library. The compression module realizes the process of compressing the input new document information into a compact vector representation.

[0056] In this embodiment, each compact vector obtained after compression is stored as a memory unit in a preset memory bank. It is composed of memory units obtained from a series of compressed documents. The preset memory library supports a dynamic update mechanism and can optimize the storage content according to the importance and freshness of each document. Therefore, the update of the memory library is dynamic, and new document compression representations will be continuously added. ,in, Indicates The memory unit of a document, Is the number of documents currently stored.

[0057] In this embodiment, before adjusting the importance weight of each current memory unit in the preset memory library based on the dynamic weight strategy through the memory module of the pre-trained model, it also includes: constructing a dynamic weight strategy according to the similarity between the current memory unit and the current query vector, the corresponding first hyperparameter, the time information of the current memory unit and the corresponding second hyperparameter. It can be understood that whenever a new document is input, the document will generate a new memory unit through the compression module. , and add it to the memory. When the memory reaches the storage limit, some mechanism is used to decide which document's memory representation to store: .

[0058] Specifically, the memory update strategy based on dynamic weights optimizes the storage efficiency of the memory bank by dynamically adjusting the importance weights of memory units.

[0059] For each memory cell in the memory bank , assign a dynamic weight , which is used to represent the relevance and importance of the memory unit. Therefore, the calculation formula of the importance weight is:

[0060] ;

[0061] in, Indicates the similarity between the memory unit and the current query, represents the temporal freshness of the memory unit, and They represent the first and second hyperparameters, respectively, which are used to balance the effects of similarity and temporal freshness.

[0062] It should be noted that if the memory bank exceeds the storage capacity, the memory elimination mechanism will be implemented according to the importance weight. The memory units with the lowest importance weight are eliminated in order of size, thereby maintaining efficient storage utilization.

[0063] Step S12: When the pre-trained model receives a training request initiated by a user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen several target similarity calculation results that meet the similarity conditions.

[0064] In this embodiment, when a user makes a query request or when a simulated user sends a query request to a pre-trained model, memory units related to the query are retrieved from the current memory bank, and an enhanced context representation is generated through an aggregation mechanism to provide support for model reasoning. Specifically, when a user makes a query , and want to answer questions based on the documents currently in memory. First, query Converted into a query vector Then, the most relevant memory cells are retrieved by calculating the similarity between the query and each document in the memory. Specifically, the cosine similarity measure is used to measure the query vector and memory unit For example, if the query request is "Who is the person in charge of the X Games in a certain year", it will involve memory units corresponding to multiple documents such as knowledge related to the organizational structure of the X Games, knowledge related to the sports management agency in a certain region, and recent news on the preparations for the X Games. Therefore, by calculating the similarity, several target similarity calculation results are screened out, that is, the memory units of several documents related to the query request are determined.

[0065] Step S13: The memory unit corresponding to each target similarity calculation result is used as the target memory unit, and each target memory unit and the query vector are weightedly aggregated through a cross-attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and the aggregated memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregated memory information and the query vector.

[0066] In this embodiment, the above-mentioned memory unit is used as the target memory unit, and then subsequent aggregation processing is performed. Specifically, all the target memory units are aggregated through the cross-attention mechanism to obtain a global aggregation result; all the target memory units are aggregated according to the same field or the same subject through the cross-attention mechanism to obtain a local aggregation result; the global aggregation result and the local aggregation result are linearly weighted combined to obtain aggregated memory information for training the memory module. It can be understood that the introduction of global and local memory aggregation mechanisms can fuse knowledge of different granularities related to the query to improve the accuracy and diversity of answer generation.

[0067] Specifically, for the most relevant memory vectors retrieved, a weighted aggregation method is used to synthesize them into a comprehensive representation. The present invention uses a cross-attention mechanism to aggregate queries and memories. Specifically, a multi-level memory aggregation mechanism, global aggregation and local aggregation are used to improve the integration capability of knowledge of different granularities through multi-level memory aggregation.

[0068] Global aggregation: retrieve the most relevant memories from all memory units for aggregation to form a global representation (global aggregation result):

[0069] ;

[0070] in, It is the criss-cross attention mechanism, which is a mechanism used in deep learning models to calculate the correlation between two input information and weight the aggregated information.

[0071] Local aggregation: Aggregate memory units for specific fields or topics to form local representations (partial aggregation results):

[0072] ;

[0073] in, Represents the collection of memory units in the current memory bank that are related to the subject of the query request.

[0074] The global representation and the local representation are further fused to generate the final memory-enhanced representation (aggregate memory information):

[0075] ;

[0076] in, is a hyperparameter that controls the weight of global aggregation results and local aggregation results.

[0077] For the query "Who is the person in charge of the X Games in X-Y?", global aggregation retrieves all relevant memories, while local aggregation only retrieves memories related to the theme of X Games. The final fused representation can generate answers more accurately. The representation will be passed to the pre-trained model for subsequent inference.

[0078] In this way, knowledge at different levels can be integrated through global and local aggregation mechanisms. Global aggregation can retrieve the most relevant memories from the overall memory bank to grasp macro knowledge; local aggregation aggregates memory units in specific fields or topics to obtain more detailed knowledge. The combination of knowledge at different granularities can provide more ideas and angles for generating answers. This aggregation method can avoid the answer being too single, so that the generated answer can respond to queries from multiple dimensions and provide richer information. Weighted aggregation can also reasonably combine knowledge according to the importance of different memory units (reflected by weights), so that the final generated answer achieves a better balance between content richness and accuracy.

[0079] In this embodiment, the pre-trained model generates answers based on the aggregated aggregate memory information and the query request. The model parameters remain fixed during this process to avoid catastrophic forgetting. It should be noted that since the model parameters of the pre-trained model remain fixed during memory training, the pre-trained model is a trained model that can perform reasoning based on query requests and memory cells of the memory bank.

[0080] Specifically, the parameters of the core model remain fixed throughout the training and reasoning process, and only the parameters of the memory module are updated with the input of new documents. Therefore, through the collaborative working mode between the pre-trained model and the memory module, the memory module provides it with processed knowledge (such as aggregated memory information). The pre-trained model can effectively use this knowledge to output prediction results. This method effectively avoids the problem of catastrophic forgetting, that is, the new knowledge will not overwrite the original knowledge when it is updated.

[0081] Using memory robustness and error repair mechanisms, the system's ability to resist interference with the memory bank is improved in the presence of irrelevant or noisy documents, and erroneous memories are dynamically repaired. In a specific implementation, during the memory retrieval phase, the retrieved memory units are evaluated for quality and noisy memories are removed: ,in, is the similarity threshold.

[0082] In another specific implementation, when the user determines that the response information is an error response, the corresponding error information is fed back to the pre-trained model to notify the memory module to locate the relevant memory unit corresponding to the response information; according to the reasoning process of the pre-trained model based on the generation of the response information, it is determined that there is an error memory unit in the relevant memory unit, and the error memory unit is deleted from the current preset memory library to update the current preset memory library. It can be understood that based on the user's feedback on the response information, the memory unit corresponding to the error response that the user feedback is an error is updated or deleted in the memory library to improve the accuracy of the system.

[0083] In this embodiment, compared with executing one query task at a time, the present invention is also applicable to multi-task queries. The specific multi-task adaptation and joint optimization methods are as follows:

[0084] If there are multiple query requests at the same time, the memory units related to each query request are extracted from the current preset memory library respectively; the response prediction is performed for each query request through the pre-trained model and based on the memory units corresponding to each query request to output the corresponding predicted response information; the loss value between the predicted response information and the corresponding true response information is calculated according to the task loss function of the pre-trained model; the model parameters of the pre-trained model are trained and optimized with the weighted sum of each loss value of each query request being minimized as the optimization goal until the training stop condition is met to obtain the target model. It can be understood that by sharing compressed memory to multiple tasks (such as question answering, summary generation, etc.), knowledge sharing between tasks is achieved, and the adaptability and generalization of the pre-trained model in multi-task scenarios are improved. Specifically, in multi-task scenarios, the memory library The stored document is a compressed representation, and each task can be processed independently based on the same memory library. Set the task collection to , Indicates the total number of tasks, each task Include query request set and the corresponding label set , Represents the total number of elements in the query request set and the corresponding label set. For each task , the system uses memory and query Retrieving relevant memories , and then generate prediction results through the pre-trained model. The loss function of the task is defined as:

[0085] ;

[0086] in, It's a task No. queries, is the correct answer. is the memory bank. The overall optimization goal of the system is the weighted sum of all task losses: .in, It's a task The weight can be adjusted dynamically according to the importance of the task.

[0087] Specifically, the weights are assigned according to the importance of the task. To make dynamic adjustments:

[0088] ;

[0089] in, Indicates The importance of a task is measured by a comprehensive consideration of multiple factors. It is not a fixed value but can be dynamically adjusted according to actual conditions. Indicates the first A measure of the importance of a task (used to traverse all tasks to perform operations such as summation).

[0090] For example, in the sports-related multitasking scenario mentioned above, It may be a sports event quiz task, It may be a task to generate a summary of sports event news, etc. They each have corresponding importance measurement indicators to participate in the calculation of weights.

[0091] Reference Figure 2 As shown, a method for continuous learning of a large language model through compressed memory training is disclosed, and the specific steps are as follows:

[0092] The first is document compression and memory representation generation, which mainly involves compressing the input document and generating corresponding memory units.

[0093] The second is memory storage and updating. The generated memory units are stored, and as new information (new documents) are input, the stored memory units are updated to ensure the timeliness and accuracy of the memory units.

[0094] Then comes memory retrieval and aggregation. When there is a task requirement, relevant information is retrieved from the memory bank and aggregated to provide more comprehensive and accurate knowledge support for generating answers.

[0095] The second is answer generation, which generates answers to specific questions or tasks based on the retrieved and aggregated memory information.

[0096] Then comes robustness enhancement and error correction, which involves processing the generated answers to enhance their robustness, that is, improving the stability and reliability of the answers in the face of various situations, while fixing existing errors to ensure the quality of the answers.

[0097] The last step is multi-task adaptation and joint optimization, which enables the entire process to adapt to a variety of different tasks and jointly optimize each link to improve overall performance and efficiency.

[0098] Through this series of steps, the process from document input to high-quality answer output is realized, and it can be continuously optimized and adapted to different task requirements.

[0099] It can be seen that the present application discloses a dialogue model training method, including: adjusting the importance weights of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, filtering the memory units with low weights in the order of the importance weights to update the preset memory library and obtain the current preset memory library; when the pre-trained model receives a training request initiated by the user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen several target similarity calculation results that meet the similarity condition; the memory unit corresponding to each of the target similarity calculation results is used as the target memory unit, and the cross-attention mechanism is used to weightedly aggregate each of the target memory units and the query vector to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and the aggregated memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregated memory information and the query vector. It can be seen that by adjusting the importance weights of memory units and filtering low-weight memory units through dynamic weight strategies, the preset memory library can always maintain the most valuable knowledge for the current task. The query request information in the user training request is converted into a query vector, and then the similarity with each memory unit in the updated memory library is calculated to screen the target similarity result, which improves the accuracy of retrieval. The target memory unit and the query vector are weightedly aggregated using the cross-attention mechanism, and the training memory module outputs aggregated memory information, integrating memory units of different granularities and sources. Finally, the continuous training process promotes the coordinated optimization of the memory module and the pre-trained model. The memory module continuously learns to optimize the memory storage, retrieval and aggregation strategies. The pre-trained model generates more accurate responses based on high-quality aggregated memory. The two feedback and adjust each other, gradually reducing the probability of generating wrong answers, improving the overall accuracy, reliability and practicality of the model, and allowing the model to always maintain good performance in long-term operation and knowledge updating.

[0100] Reference Figure 3 As shown, the present invention also discloses a dialogue model training device, including:

[0101] A library update module 11 is used to adjust the importance weight of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, and filter the memory units with low weights in the order of the importance weights to update the preset memory library and obtain the current preset memory library;

[0102] A similarity calculation module 12 is used for converting the query request information in the training request into a query vector when the pre-trained model receives a training request initiated by the user, and using the pre-trained model to calculate the similarity between the query vector and each memory unit in the current preset memory library, so as to screen a number of target similarity calculation results that meet the similarity condition;

[0103] The training module 13 is used to use the memory unit corresponding to each target similarity calculation result as the target memory unit, and weightedly aggregate each target memory unit and the query vector through a cross-attention mechanism to train the memory module to output corresponding aggregate memory information based on each target memory unit and the query vector, and input the aggregate memory information into the pre-trained model so that the pre-trained model generates response information corresponding to the query request based on the aggregate memory information and the query vector.

[0104] It can be seen that the present application discloses a method of adjusting the importance weights of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, filtering the memory units with low weights in the order of the importance weights to update the preset memory library and obtain the current preset memory library; when the pre-trained model receives a training request initiated by the user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen several target similarity calculation results that meet the similarity condition; the memory unit corresponding to each target similarity calculation result is used as the target memory unit, and the target memory unit and the query vector are weightedly aggregated through the cross-attention mechanism to train the memory module to output the corresponding aggregate memory information based on each target memory unit and the query vector, and the aggregate memory information is input into the pre-trained model so that the pre-trained model generates the response information corresponding to the query request according to the aggregate memory information and the query vector. It can be seen that by adjusting the importance weights of the memory units and filtering the low-weight memory units through the dynamic weight strategy, the preset memory library can always maintain the most valuable knowledge for the current task. The query request information in the user training request is converted into a query vector, and then the similarity with each memory unit in the updated memory library is calculated to select the target similarity result, thereby improving the accuracy of retrieval. The target memory unit and the query vector are weightedly aggregated using the cross-attention mechanism, and the training memory module outputs the aggregated memory information, integrating memory units of different granularities and sources. Finally, the continuous training process promotes the coordinated optimization of the memory module and the pre-trained model. The memory module continuously learns to optimize the memory storage, retrieval and aggregation strategies. The pre-trained model generates more accurate responses based on high-quality aggregated memory. The two feedback and adjust each other, gradually reducing the probability of generating wrong answers, improving the overall accuracy, reliability and practicality of the model, and allowing the model to always maintain good performance in long-term operation and knowledge updating.

[0105] Furthermore, the present application also discloses an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0106] Figure 4A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the dialogue model training method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0107] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0108] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0109] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0110] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the dialogue model training method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0111] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed dialogue model training method is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0112] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0113] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (ElectricErasable Programmable Read Only Memory), register, hard disk, removable disk, CD-ROM (CompactDisc-Read Only Memory), or any other form of storage medium known in the technical field.

[0114] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0115] The scheme provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A dialogue model training method, characterized in that: include: The importance weight of each current memory unit in the preset memory bank is adjusted through the memory module of the pre-trained model and based on the dynamic weight strategy, and the memory units with low weight are filtered in the order of the importance weight to update the preset memory bank and obtain the current preset memory bank; When the pre-trained model receives a training request initiated by a user, the query request information in the training request is converted into a query vector, and the pre-trained model is used to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen a number of target similarity calculation results that meet the similarity condition; The memory unit corresponding to each target similarity calculation result is used as the target memory unit, and the target memory unit and the query vector are weightedly aggregated through a cross-attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and the aggregated memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregated memory information and the query vector.

2. The method for training a dialogue model according to claim 1, characterized in that: Before adjusting the importance weight of each current memory unit in the preset memory bank based on the dynamic weight strategy through the memory module of the pre-trained model, the method further includes: Each document used for training the model is compressed into a compact vector one by one, and each of the compact vectors is stored as a memory unit in a preset memory bank.

3. The dialogue model training method according to claim 2, characterized in that: The step of compressing each document used for training the model into a compact vector one by one includes: Obtaining a number of documents for training the model, and mapping characters of each of the documents through word vector mapping to obtain mapped documents; Each of the mapped documents and the corresponding preset vector markers are compressed by a compression module to obtain a compact vector with a preset length; wherein the preset vector marker is a vector used to enhance the document expression capability of the mapped document.

4. The method for training a dialogue model according to claim 1, characterized in that: Before adjusting the importance weight of each current memory unit in the preset memory bank based on the dynamic weight strategy through the memory module of the pre-trained model, the method further includes: A dynamic weight strategy is constructed according to the similarity between the current memory unit and the current query vector, the corresponding first hyperparameter, the time information of the current memory unit, and the corresponding second hyperparameter.

5. The method for training a dialogue model according to claim 1, characterized in that: The weighted aggregation of each target memory unit and the query vector by the cross attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector includes: Aggregate all the target memory units through a cross-attention mechanism to obtain a global aggregation result; Aggregate all the target memory units according to the same field or the same subject through a cross-attention mechanism to obtain a local aggregation result; The global aggregation result and the local aggregation result are linearly weighted combined to obtain aggregated memory information for training the memory module.

6. The method for training a dialogue model according to claim 1, characterized in that: After the aggregate memory information is input into the pre-trained model so that the pre-trained model generates response information corresponding to the query request according to the aggregate memory information and the query vector, the method further includes: When the user determines that the response information is an error response, the corresponding error information is fed back to the pre-trained model to notify the memory module to locate the relevant memory unit corresponding to the response information; According to the inference process of generating the response information based on the pre-trained model, it is determined that there is an erroneous memory unit in the relevant memory unit, and the erroneous memory unit is deleted from the current preset memory library to update the current preset memory library.

7. The method for training a dialogue model according to any one of claims 1 to 6, characterized in that: Also includes: If there are multiple query requests at the same time, extract the memory units related to each query request from the current preset memory bank respectively; Predicting responses to each query request using the pre-trained model and based on the memory units corresponding to each query request, so as to output corresponding predicted response information; Calculate the loss value between the predicted response information and the corresponding true response information according to the task loss function of the pre-trained model; The model parameters of the pre-trained model are trained and optimized with the minimum weighted sum of the loss values ​​of each query request as the optimization goal until the training stop condition is met to obtain the target model.

8. A dialogue model training device, characterized in that: include: A library update module, used to adjust the importance weight of each current memory unit in the preset memory library through the memory module of the pre-trained model and based on the dynamic weight strategy, and filter the low-weight memory units in the order of the importance weight to update the preset memory library and obtain the current preset memory library; A similarity calculation module, configured to convert the query request information in the training request into a query vector when the pre-trained model receives a training request initiated by a user, and use the pre-trained model to calculate the similarity between the query vector and each memory unit in the current preset memory library to screen a number of target similarity calculation results that meet the similarity condition; A training module is used to use the memory unit corresponding to each target similarity calculation result as a target memory unit, and weightedly aggregate each target memory unit and the query vector through a cross-attention mechanism to train the memory module to output corresponding aggregated memory information based on each target memory unit and the query vector, and input the aggregated memory information into the pre-trained model so that the pre-trained model generates response information corresponding to the query request based on the aggregated memory information and the query vector.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the dialogue model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the dialogue model training method as described in any one of claims 1 to 7 are implemented.

Images