Pre-training model adjusting method and device, storage medium and computing equipment

Abstract

This application provides a method, apparatus, storage medium, and computing device for adjusting a pre-trained model. The method includes: acquiring initial samples, which include multiple question-answer pairs, and calculating the sentence embedding corresponding to each question-answer pair, where each question-answer pair includes a question and an answer; clustering each question-answer pair using the sentence embeddings to obtain multiple clusters, where each cluster includes multiple points, and each point corresponds to a sentence embedding; selecting multiple points in each cluster based on their maximum distance to other points in the same cluster to obtain core samples; and training and adjusting the pre-trained model using the core samples. This application can improve the performance of the pre-trained model by preventing the model from forgetting existing knowledge while ensuring the effectiveness of model training.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a pre-trained model adjustment method and apparatus, storage medium, and computing device. Background Technology

[0002] With the increasing popularity of large-scale pre-trained language models, they have demonstrated their powerful capabilities in many scenarios and tasks. For a specific model, its training data is fixed during training. If the model cannot learn online, its internal knowledge becomes outdated. Furthermore, due to various problems in the training data (e.g., unclean data), the internal knowledge may also contain errors after training. Retraining the model is costly, time-consuming, and labor-intensive. Therefore, many researchers have begun to explore how to edit large models to modify their knowledge. Simultaneously, they strive to maintain consistency between the unmodified knowledge and the model's previous outputs.

[0003] Generally, training large models involves two steps: 1. Unsupervised training, which uses a large amount of training data to give the model general capabilities; 2. Fine-tuning, which allows the model to better align with downstream tasks and user interaction habits. Existing constrained fine-tuning methods typically involve updating the model's internal knowledge with new data, i.e., using new data to fine-tune the model.

[0004] However, existing technologies suffer from the problem of pre-trained models forgetting existing knowledge when adjusting pre-trained models, which affects the performance of the models. Summary of the Invention

[0005] This application can improve the performance of pre-trained models by preventing them from forgetting existing knowledge while ensuring the effectiveness of model training.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] Firstly, a pre-trained model adjustment method is provided, comprising: acquiring initial samples, the initial samples including multiple question-answer pairs, and calculating the sentence embedding corresponding to each question-answer pair, each question-answer pair including a question and an answer; clustering each question-answer pair using the sentence embeddings corresponding to each question-answer pair to obtain multiple clusters, each cluster including multiple points, each point corresponding to a sentence embedding; selecting multiple points in each cluster according to the maximum distance to other points in the same cluster to obtain core samples; and training and adjusting the pre-trained model using the core samples.

[0008] Optionally, selecting multiple points in each cluster based on the maximum distance to other points in the same cluster includes: calculating the Euclidean distance between every two points in each cluster and determining the maximum distance for each point; sorting the points in each cluster in ascending order of maximum distance and selecting the top-ranked points as the core samples.

[0009] Optionally, each cluster has a center point. After clustering each question-answer pair using the sentence embeddings corresponding to each question-answer pair, the method further includes: calculating the cosine similarity between each point in each cluster and the center point of that cluster; and selecting points in each cluster whose cosine similarity is greater than a threshold value for use in the selection of the core samples.

[0010] Optionally, obtaining the initial sample includes: obtaining an initial question-answer pair from a knowledge base, the knowledge base including knowledge entries, the knowledge entries including the initial question-answer pair; updating the initial question-answer pair using the pre-trained model to obtain the initial sample, the language style of the initial sample being consistent with the language style output by the pre-trained model.

[0011] Optionally, the pre-trained model adjustment method further includes: obtaining user feedback on the output of the pre-trained model, the feedback including questions and updated answers; and adding the feedback to the core samples.

[0012] Optionally, adding the feedback result to the core sample includes: adding the feedback result to the core sample every preset iteration during the training of the pre-trained model.

[0013] Optionally, the pre-trained model adjustment method further includes: updating the feedback results in a knowledge base, wherein the knowledge base includes knowledge entries, and the knowledge entries include multiple question-answer pairs.

[0014] Optionally, the pre-trained model includes an adapter, and the step of training and adjusting the pre-trained model using the core samples includes: training the adapter using the core samples to update the parameters of the adapter, wherein the parameters of the adapter are used to calculate the output of the pre-trained model.

[0015] Secondly, this application also discloses a pre-trained model adjustment device, which includes: an acquisition module for acquiring initial samples, the initial samples including multiple question-answer pairs, and calculating the sentence embedding corresponding to each question-answer pair, each question-answer pair including a question and an answer; a clustering module for clustering each question-answer pair using the sentence embeddings corresponding to each question-answer pair to obtain multiple clusters, each cluster including multiple points, each point corresponding to a sentence embedding; a selection module for selecting multiple points in each cluster according to the maximum distance to other points in the same cluster to obtain core samples; and an adjustment module for training and adjusting the pre-trained model using the core samples.

[0016] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to perform any of the methods provided in the first aspect.

[0017] Fourthly, a computer program product is provided, on which a computer program is stored, the computer program being executed by a processor to perform any of the methods provided in the first aspect.

[0018] Fifthly, embodiments of this application also provide a chip that stores a computer program, which, when executed by the chip, implements the steps of the above-described method.

[0019] In a sixth aspect, embodiments of this application also provide a system chip for use in a terminal. The chip system includes at least one processor and an interface circuit, the interface circuit and the at least one processor being interconnected via a line. The at least one processor is used to execute instructions to perform the method provided in the first aspect.

[0020] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0021] In this application's technical solution, initial samples are obtained, comprising multiple question-answer pairs. Sentence embeddings are calculated for each question-answer pair, each pair including a question and an answer. These question-answer pairs are then clustered using their corresponding sentence embeddings to obtain multiple clusters. Each cluster contains multiple points, each point corresponding to a sentence embedding. Multiple points within each cluster are selected based on their maximum distance to other points in the same cluster to obtain core samples. These core samples are then used to train and adjust the pre-trained model. In this application, the core samples effectively represent the initial samples, meaning a small number of samples represent a large amount of data, resulting in high-quality core samples. Training the pre-trained model with a small number of core samples ensures effective model training while preventing the model from forgetting existing knowledge. This allows for domain-specific reinforcement training while maintaining the model's generality, further improving the performance of the pre-trained model.

[0022] Furthermore, the cosine similarity between each point in each cluster and the center point of that cluster is calculated. Points with a cosine similarity greater than a threshold value are selected from each cluster for core sample selection. Specifically, the Euclidean distance between every two points in each cluster is calculated, and the maximum distance for each point is determined. Within each cluster, points are sorted in ascending order of maximum distance, and the top-ranked points are selected as core samples. This application reduces the computational load of core sample selection through a two-step screening process, saving computational resources.

[0023] Furthermore, by using feedback results, the convergence speed of the pre-trained model during training can be improved, enabling the model to converge faster and saving training time. Attached Figure Description

[0024] Figure 1 This is a flowchart of a pre-trained model adjustment method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of a pre-trained model adjustment architecture provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of a pre-trained model adjustment device provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the hardware structure of a pre-trained model adjustment device provided in an embodiment of this application. Detailed Implementation

[0028] As described in the background section, existing technologies suffer from the problem of pre-trained models forgetting existing knowledge when adjusting pre-trained models, which affects the performance of the models.

[0029] In addition, how to update the parameters of the pre-trained model by updating as few model parameters as possible or only updating part of the model structure can save model computing resources and training time.

[0030] In this application, the core samples can represent multiple question-answer pairs, that is, a small number of samples are used to represent a large amount of data, and the core samples have high quality. By training the pre-trained model with a small number of core samples, the pre-trained model avoids forgetting the original knowledge while ensuring the effectiveness of model training. Thus, the model is reinforced in a specific domain while ensuring its generality, which further improves the performance of the pre-trained model.

[0031] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0032] See Figure 1The method provided in this application specifically includes the following steps:

[0033] Step 101: Obtain initial samples, which include multiple question-answer pairs, and calculate the sentence embedding corresponding to each question-answer pair. Each question-answer pair includes a question and an answer.

[0034] Step 102: Cluster each question-answer pair using the sentence embeddings corresponding to each question-answer pair to obtain multiple clusters. Each cluster includes multiple points, and each point corresponds to a sentence embedding.

[0035] Step 103: Select multiple points in each cluster based on their maximum distance from other points in the same cluster to obtain core samples;

[0036] Step 104: Train and adjust the pre-trained model using core samples.

[0037] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0038] It is understood that, in specific implementations, the pre-trained model adjustment method can be implemented using software programs, which run in the processor integrated within the chip or chip module. This method can also be implemented using a combination of software and hardware; this application does not impose any limitations on this approach.

[0039] This embodiment can sample the most useful core samples from existing data to help the pre-trained model learn the knowledge required for downstream tasks, and achieve good performance of the pre-trained model with only a small number of samples. In this embodiment, question-answer pairs refer to data in the format of questions and answer pairs with descriptive instructions.

[0040] In one specific implementation of step 101, initial samples can be directly obtained from the knowledge base. The question-answer pairs in the knowledge base will directly participate in the adjustment of the subsequent pre-trained model. Specifically, the knowledge base is pre-built by collecting accurate and reliable knowledge entries. Knowledge entries can be in the form of question-answer pairs.

[0041] In another specific implementation of step 101, initial question-answer pairs can be obtained from a knowledge base, where the knowledge entries include the initial question-answer pairs; then, the initial question-answer pairs can be updated using a pre-trained model to obtain initial samples, the language style of which is consistent with the language style of the output of the pre-trained model.

[0042] This embodiment updates and rewrites the initial question-answer pairs in the knowledge base using a pre-trained model, ensuring that the initial samples are consistent with the style generated by the model. The updated question-answer pairs will then be used to adjust the pre-trained model, thereby guaranteeing the consistency of the language style output by the adjusted pre-trained model and improving the user experience.

[0043] More specifically, the knowledge entries in the knowledge base can be cleaned web page data or parsed document data, such as document data in Word, PDF and TXT formats; then these data are sliced ​​into document blocks of appropriate size and length; then a pre-trained model is used to generate question-answer pairs, or question-answer pairs, from these document blocks.

[0044] To obtain core samples, the question-answer pairs need to be vectorized first. Specifically, the Bidirectional Encoder Representation from Transformers (BERT) model can be used to vectorize the question-answer pairs to obtain the sentence embeddings corresponding to the question-answer pairs. Specifically, after inputting each question-answer pair into the BERT model, the cls word vectors of each question-answer pair are extracted to represent the sentence embedding of that question-answer pair.

[0045] It should be noted that other algorithms can also be used to calculate sentence embeddings, such as WORD2VECTOR, and this application does not impose any restrictions on this.

[0046] In step 102, each question-answer pair is clustered using its corresponding sentence embedding. Specifically, unsupervised clustering can be performed using the aforementioned sentence embedding. Specifically, the Density-Based Spatial Clustering of Applications with Noise (DBSCANDBSCAN) algorithm is used to perform unsupervised clustering in the embedding space to obtain multiple clusters. The center point of the cluster with the highest frequency in each cluster is selected as the distribution center point of that cluster.

[0047] In step 103, multiple points are selected in each cluster according to the maximum distance to other points in the same cluster to obtain core samples.

[0048] In a non-limiting embodiment, the Euclidean distance between every two points in each cluster is calculated, and the maximum distance for each point is determined. Within each cluster, points are sorted in ascending order of their maximum distances, and the top-ranked points are selected as core samples.

[0049] By employing the selection method described above, we can ensure that the maximum distance between each point and other points in the final core samples is small, meaning that the selected core samples can effectively represent the overall initial samples. Furthermore, the number of core samples is significantly lower than the number of initial samples.

[0050] In this invention, embodiments can select diverse and high-quality core samples to align with downstream tasks and user interaction habits. Knowledge updates are completed by adjusting the pre-trained model using a small number of samples, saving model computing resources and training time.

[0051] Furthermore, the cosine similarity between each point in each cluster and the center point of that cluster is calculated; points with a cosine similarity greater than a threshold value are selected in each cluster for use in the selection of core samples.

[0052] As mentioned earlier, selecting core samples requires calculating the Euclidean distance between every two points, which can be computationally intensive when the number of points is large. To reduce computational load and conserve computing resources, this embodiment of the invention employs a two-step filtering process. Points with a cosine similarity greater than a threshold between each point and the cluster's center point are selected, and only these selected points are used to calculate the Euclidean distance, thereby reducing unnecessary computation.

[0053] Specifically, after obtaining the distribution center points corresponding to the clusters, the cosine similarity between each point in each cluster and the center point is calculated. The samples within each cluster are then sorted based on this similarity, and samples with similarity values ​​exceeding a threshold are selected. This process results in a large number of selected samples, requiring further filtering. Specifically, the KCentergreedy core set algorithm can be used for each cluster to select core samples based on the required number of samples and clusters. For example, if 1000 core samples are needed and there are 20 clusters, then 50 points need to be selected from each cluster.

[0054] Furthermore, since the number of core samples is relatively small, manual review of the core samples is possible, specifically to assess and review the quality and diversity of the core samples and make modifications accordingly.

[0055] In step 104, the pre-trained model is trained and adjusted using core samples. Since core samples can represent the initial samples, and the number of core samples is much smaller than the number of initial samples, training the pre-trained model with a small number of core samples ensures the effectiveness of model training while preventing the pre-trained model from forgetting the original knowledge. This allows for domain-specific reinforcement training of the model while maintaining its generality, further improving the performance of the pre-trained model.

[0056] In this embodiment of the invention, core samples are used to fine-tune the pre-trained model, enabling the pre-trained model to have the ability to understand knowledge in a specific domain.

[0057] Specifically, model fine-tuning aims to adjust the pre-trained model fW (where W represents the model's parameters) using the edit descriptor (xe, ye). The ultimate goal is to create an edited model, denoted as fWe (where the parameter size of We can be greater than or equal to W). In particular, given an edit descriptor that includes the edit input xe and the edit label ye, such that the original model fθ(xe) ≠ ye, the output of the edited model will be fθe(xe) = ye.

[0058] In one non-limiting embodiment, the pre-trained model includes an adapter. The adapter is trained using core samples to update its parameters, which are used to compute the output of the pre-trained model.

[0059] Specifically, fine-tuning for specific tasks is achieved by adding an adapter to the pre-trained model. During fine-tuning, only the adapter's parameters are updated, not the pre-trained model's parameters. These adapters occupy only a small fraction of the original model size, making the pre-trained model train faster and requiring less memory. Adapters can be trained for multiple tasks and then fused into the pre-trained model to perform new tasks.

[0060] The model fine-tuning method used in this embodiment of the invention does not perform full fine-tuning of the model parameters, but rather achieves effective fine-tuning of the pre-trained language model by efficiently fine-tuning the parameters, while minimizing the required parameters and computational resources.

[0061] In a specific example, LoRA can be used for model fine-tuning.

[0062] For example, fine-tuning a pre-trained language model (such as GPT-3.5) in a downstream task requires updating the pre-trained model parameters, expressed as: W + ΔW, where θ is the pre-trained model parameter and ΔW is the parameter to be updated. LoRA's fine-tuning method adds a bypass to the original pre-trained Large Scale Language Model (LLM), performing a dimensionality reduction and subsequent dimensionality increase operation. During training, the LLM parameters are fixed, and only the dimensionality reduction matrix A and the dimensionality increase matrix B are trained. The input and output dimensions of the model remain unchanged; the output is achieved by superimposing the bypass parameters with the LLM parameters.

[0063] Figure 2 An architecture for adjusting a pre-trained model is illustrated. A knowledge base 201 can provide knowledge entries, and initial samples are selected from the knowledge base. The model fine-tuning module 202 can perform step 104 described above, that is, train and adjust the pre-trained model using core samples.

[0064] In this embodiment, the knowledge editor 204 can be used to obtain feedback results from the user on the output of the pre-trained model, including questions and updated answers; the feedback module 203 is used to add the feedback results to the core samples.

[0065] By providing feedback results, this invention can improve the convergence speed of the pre-trained model during training, enabling the model to converge faster and saving training time.

[0066] Specifically, through the knowledge editing module 204, human experts review, edit, and modify the knowledge generated by the pre-trained model. The knowledge editing module 204 provides a user interface, which includes input text boxes, a display area, a feedback button, a knowledge entry browsing area, and a save button.

[0067] More specifically, an input text box is a text box used to input editing instructions, in which human experts can enter editing instructions, such as instructions to modify, supplement, or delete specific knowledge content.

[0068] The display area is used to show the knowledge content generated by the pre-trained model, and human experts can view the knowledge content generated by the pre-trained model in the display area.

[0069] The feedback button is a button that human experts can click to provide feedback, such as pointing out errors or inaccuracies in the generated results, or giving suggestions for improvement.

[0070] The knowledge entry browsing area is a region used to browse existing knowledge entries in the knowledge base. Human experts can browse existing knowledge content to assist in the editing process.

[0071] The save button is a button that human experts can click to save the edited knowledge.

[0072] Through the aforementioned user interface, human experts can interact with the pre-trained model, input editing commands, view generated results, and provide feedback, thereby participating in the knowledge editing process.

[0073] In a non-limiting embodiment, the feedback module 203, based on the editing and feedback from human experts, returns the feedback results to the knowledge base 201 for knowledge updates, and resamples the feedback results and adds them to the pre-trained model for fine-tuning, forming a virtuous cycle of knowledge generation, editing, and updating.

[0074] In another non-limiting embodiment, the feedback module 203 can directly add the feedback results to the core samples to participate in the fine-tuning of the pre-trained model.

[0075] Specifically, during the training of the pre-trained model, feedback results are added to the core samples every preset iteration round. For example, if the pre-trained model has 3000 training iterations, feedback results can be added to the core samples in real time at the 1000th and 2000th iterations to participate in subsequent model training.

[0076] This invention combines the knowledge of a pre-trained model with that of human experts. Human experts can directly participate in the knowledge editing process and correct the knowledge generated by the large model in real time through editing instructions, making the knowledge generated by the pre-trained model more accurate and controllable.

[0077] The embodiments of this invention can be widely applied to knowledge editing and generation tasks in various fields. Whether it is scientific research or technical document writing, the embodiments of this invention can quickly generate and edit knowledge in a specific field.

[0078] For more specific implementations of the embodiments of this application, please refer to the foregoing embodiments, which will not be repeated here.

[0079] Please refer to Figure 3 , Figure 3 A pre-trained model adjustment device 30 is shown, which may include:

[0080] The acquisition module 301 is used to acquire initial samples, which include multiple question-answer pairs, and to calculate the sentence embedding corresponding to each question-answer pair. Each question-answer pair includes a question and an answer.

[0081] Clustering module 302 is used to cluster each question-answer pair using the sentence embeddings corresponding to each question-answer pair to obtain multiple clusters, each cluster including multiple points, and each point corresponding to a sentence embedding;

[0082] The selection module 303 is used to select multiple points in each cluster according to the maximum distance to other points in the same cluster to obtain core samples;

[0083] Adjustment module 304 is used to train and adjust the pre-trained model using core samples.

[0084] In specific implementation, the aforementioned pre-trained model adjustment device 30 may correspond to a chip in a computing device that has a pre-trained model adjustment function, such as a system-on-a-chip (SOC), a baseband chip, etc.; or to a chip module in a terminal device that includes a pre-trained model adjustment function; or to a chip module with a data processing function chip; or to a computing device.

[0085] Other relevant descriptions of the pre-trained model adjustment device 30 can be found in the descriptions in the foregoing embodiments, and will not be repeated here.

[0086] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on a processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented using a software program that runs on a processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0087] This application also discloses a storage medium, which is a computer-readable storage medium storing a computer program thereon. When the computer program is executed, it can perform the steps of the method shown in the foregoing embodiments. The storage medium may include read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0088] Please refer to Figure 4 This application also provides a schematic diagram of the hardware structure of a pre-trained model adjustment device. The device includes a processor 401, a memory 402, and a transceiver 403.

[0089] Processor 401 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program according to the present application. Processor 401 may also include multiple CPUs, and processor 401 can be a single-core processor or a multi-core processor. Here, processor can refer to one or more devices, circuits, or processing cores used to process data (e.g., computer program instructions).

[0090] The memory 402 can be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. This application embodiment does not impose any limitations on this. The memory 402 can exist independently (in this case, the memory 402 can be located outside or inside the device) or it can be integrated with the processor 401. The memory 402 may contain computer program code. The processor 401 is used to execute the computer program code stored in the memory 402 to implement the method provided in this application embodiment.

[0091] The processor 401, memory 402, and transceiver 403 are connected via a bus. The transceiver 403 is used to communicate with other devices or communication networks. Optionally, the transceiver 403 may include a transmitter and a receiver. The device in the transceiver 403 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 403 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0092] when Figure 4 The schematic diagram shown illustrates the structure of the terminal device involved in the above embodiments. The processor 401 is used to control and manage the actions of the terminal device; for example, the processor 401 is used to support the terminal device in performing... Figure 1 Steps 101 and 102 in the text, or Figure 2 The actions performed by the terminal device in steps 302, 303, and 304, and / or other processes described in the embodiments of this application. The processor 401 can communicate with other network entities via the transceiver 403, for example, with the aforementioned network device. The memory 402 is used to store the program code and data of the terminal device.

[0093] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0094] In the embodiments of this application, "multiple" refers to two or more.

[0095] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0096] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0097] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0098] It should be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0102] The integrated unit implemented as a software functional unit described above can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of this application.

[0103] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. A method for adjusting a pre-trained model, characterized in that, include: Obtain an initial sample, which includes multiple question-answer pairs, and calculate the sentence embedding corresponding to each question-answer pair, where each question-answer pair includes a question and an answer; Each question-answer pair is clustered using the sentence embeddings corresponding to each question-answer pair to obtain multiple clusters. Each cluster includes multiple points, and each point corresponds to a sentence embedding. In each cluster, multiple points are selected based on their maximum distance from other points in the same cluster to obtain core samples, which represent the initial samples of the whole. The pre-trained model is trained and adjusted using the core samples. The step of selecting multiple points in each cluster based on the maximum distance to other points in the same cluster includes: Calculate the Euclidean distance between every two points in each cluster and determine the maximum distance between each point; Within each cluster, the data are sorted in ascending order of maximum distance, and the top-ranked points are selected as the core samples.

2. The pre-trained model adjustment method according to claim 1, characterized in that, Each cluster has a centroid, and the process of clustering each question-answer pair using the sentence embeddings corresponding to each question-answer pair further includes: Calculate the cosine similarity between each point in each cluster and the center point of that cluster; Points with a cosine similarity greater than a threshold value are selected from each cluster for use in the selection of the core samples.

3. The pre-trained model adjustment method according to claim 1, characterized in that, The process of obtaining the initial sample includes: Obtain initial question-answer pairs from a knowledge base, which includes knowledge entries, and the knowledge entries include the initial question-answer pairs. The initial question-answer pair is updated using the pre-trained model to obtain the initial sample, the language style of which is consistent with the language style of the output of the pre-trained model.

4. The pre-trained model adjustment method according to claim 1, characterized in that, Also includes: Obtain user feedback on the output of the pre-trained model, the feedback including questions and updated answers; The feedback results are added to the core sample.

5. The pre-trained model adjustment method according to claim 4, characterized in that, The step of adding the feedback result to the core sample includes: During the training of the pre-trained model, the feedback results are added to the core samples every preset iteration round.

6. The pre-trained model adjustment method according to claim 4, characterized in that, Also includes: The feedback result is updated in a knowledge base, which includes knowledge entries, each of which includes multiple question-answer pairs.

7. The pre-trained model adjustment method according to claim 1, characterized in that, The pre-trained model includes an adapter, and the step of training and adjusting the pre-trained model using the core samples includes: The adapter is trained using the core samples to update the adapter's parameters, which are then used to calculate the output of the pre-trained model.

8. A pre-trained model adjustment device, characterized in that, include: The acquisition module is used to acquire initial samples, which include multiple question-answer pairs, and to calculate the sentence embedding corresponding to each question-answer pair, wherein each question-answer pair includes a question and an answer; The clustering module is used to cluster each question-answer pair using the sentence embeddings corresponding to each question-answer pair, so as to obtain multiple clusters. Each cluster includes multiple points, and each point corresponds to a sentence embedding. The selection module is used to select multiple points in each cluster according to the maximum distance to other points in the same cluster to obtain core samples, which represent the initial samples of the whole. The adjustment module is used to train and adjust the pre-trained model using the core samples; The selection module performs the following steps: Calculate the Euclidean distance between every two points in each cluster and determine the maximum distance between each point; Within each cluster, the data are sorted in ascending order of maximum distance, and the top-ranked points are selected as the core samples.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the pre-trained model adjustment method according to any one of claims 1 to 7.

10. A computing device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the pre-trained model adjustment method according to any one of claims 1 to 7.

Images