A fine-tuning fault-tolerant method, device, medium and product of an intelligent question and answer model

Abstract

The application discloses a fine-tuning fault-tolerant method, device, medium and product of an intelligent question and answer model, relates to the technical field of artificial intelligence, and comprises the following steps: dividing a preset low-rank matrix pre-injected in an intelligent question and answer model into at least two target sub-matrices according to the influence degree of the matrix parameters on the model performance, and matching corresponding target coding algorithms for different target sub-matrices, so that hierarchical and differentiated data protection is realized, and resource waste caused by uniform intensity protection on all parameters is avoided. Moreover, the target sub-matrices are respectively subjected to coding calculation at a first preset time point and a second preset time point of any two adjacent training steps, and the two coding results are compared, so that a data damage event occurring to the preset low-rank matrix in a training process can be effectively and quickly detected, thereby preventing error pollution from spreading to the whole model, and the reliability of the system is improved. The scheme disclosed by the application can significantly reduce fault-tolerant overhead and guarantee the stability in the fine-tuning process of the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, medium and product for fine-tuning and fault-tolerant intelligent question-answering models. Background Technology

[0002] Pre-trained models, such as Large Language Models (LLMs), are widely used in fields such as artificial intelligence and computer vision. To adapt general-purpose pre-trained models to specific downstream tasks, methods such as Low-Rank Adaptation (LoRA) can be used to fine-tune the pre-trained models.

[0003] However, when fine-tuning in a distributed environment, hardware failures and data corruption occur frequently, seriously affecting training stability and model accuracy. On the one hand, low-rank matrix parameters are susceptible to high-energy particle impacts, voltage fluctuations, etc., resulting in silent data corruption (SDC) without system error reports. Traditional mechanisms are difficult to detect this, and it will gradually pollute the parameters. On the other hand, traditional fault-tolerant methods mostly adopt unified protection strategies such as full parameter redundancy backup, without considering the differences in the impact of different parameters in the low-rank matrix on model performance. This results in wasted overhead for protecting non-critical parameters, which does not meet the lightweight requirement of efficient parameter fine-tuning.

[0004] In summary, how to quickly and timely detect data corruption events and implement differentiated data protection for low-rank matrices to reduce fault tolerance overhead and ensure stability during model fine-tuning are problems that need to be solved. Summary of the Invention

[0005] This application provides a method, device, medium, and product for fine-tuning and fault tolerance of an intelligent question-answering model, which can quickly and timely detect data corruption events and take differentiated data protection measures for low-rank matrices to reduce fault tolerance overhead and ensure stability during model fine-tuning.

[0006] This application provides a fine-tuning and fault-tolerant method for an intelligent question-answering model, applied to computing nodes in a distributed computing system, wherein a pre-trained intelligent question-answering model runs on the computing nodes. The method includes:

[0007] Obtain the pre-injected low-rank matrix used for model fine-tuning in the intelligent question answering model, and divide the pre-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the pre-rank matrix on the model performance.

[0008] For any target submatrix, a suitable target encoding algorithm is determined based on the degree of influence of any target submatrix on the model performance. The target encoding algorithm is then used to encode any target submatrix at the first and second preset time points corresponding to any two adjacent training steps, respectively, to obtain the first encoding result and the second encoding result.

[0009] Determine whether the first encoding result and the second encoding result are consistent. If the first encoding result and the second encoding result are inconsistent, it is determined that a data corruption event has occurred, and the data is recovered using the recovery mechanism corresponding to the data corruption event.

[0010] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the fine-tuning and fault-tolerant method of any of the above-described intelligent question-answering models when executing the computer program.

[0011] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the fine-tuning and fault-tolerant method of any of the above-described intelligent question-answering models.

[0012] This application also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, it implements the steps of fine-tuning and fault-tolerant methods for any of the above-described intelligent question-answering models.

[0013] Beneficial Effects: This application achieves layered and differentiated data protection by dividing the pre-injected low-rank matrix in the intelligent question-answering model into at least two target sub-matrices according to the degree of influence of matrix parameters on model performance, and matching corresponding target encoding algorithms to different target sub-matrices. This differentiated design avoids the resource waste caused by applying uniform strength protection to all parameters, allowing limited computing and storage resources to be prioritized for protecting the most critical data. This significantly reduces overall computing and storage overhead while ensuring the reliability of key parameters. Furthermore, by encoding the target sub-matrices at the first and second preset time points of any two adjacent training steps and comparing the two encoding results, this application can effectively and quickly detect data corruption events that occur in the pre-injected low-rank matrix during training, thereby preventing error contamination from spreading to the entire model and improving system reliability. Finally, when a data corruption event is detected, data recovery is performed using the recovery mechanism corresponding to the data corruption event, minimizing training interruption time caused by faults and improving overall training efficiency. The solution of this application can significantly reduce fault tolerance overhead and ensure stability during model fine-tuning. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a fine-tuning and fault-tolerant method for an intelligent question-answering model provided in this application embodiment;

[0016] Figure 2 A flowchart illustrating a specific fine-tuning and fault-tolerant method for an intelligent question-answering model provided in this application embodiment;

[0017] Figure 3 This is a schematic diagram of a matrix partitioning encoding and snapshot saving fault-tolerant process disclosed in this application;

[0018] Figure 4 This is a schematic diagram of the structure of a fine-tuning fault-tolerant device for an intelligent question-answering model disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] First, we will explain some of the terms and concepts involved in the fine-tuning and fault-tolerant method of an intelligent question-answering model provided in the embodiments of this application.

[0023] Pre-training is a strategy for training deep learning models. Its core lies in using large-scale datasets to initially train the model, enabling it to learn general feature representations. This process is similar to the basic learning stage humans undergo before learning new knowledge, accumulating experience through extensive reading and observation. Pre-trained language models generally refer to designing language model training tasks based on large-scale corpora (including language training materials such as sentences and paragraphs), training large-scale neural network algorithm structures to learn and implement these tasks. The resulting large-scale neural network algorithm structure and parameters constitute the pre-trained language model. Subsequent tasks can then be performed on this model, with feature extraction or task fine-tuning to achieve specific objectives. The idea behind pre-training is to first train a set of model parameters for one task, then use these parameters to initialize the network model parameters, and finally use the initialized network model to train other tasks, obtaining models adapted for those tasks. By pre-training on large-scale corpora, neural language representation models can learn powerful language representation capabilities, extracting rich syntactic and semantic information from text. Pre-trained language models can provide tokens with rich semantic information and sentence-level features for downstream tasks. They can also be fine-tuned directly on the pre-trained model for downstream tasks, making it convenient and quick to obtain downstream-specific models.

[0024] Fine-tuning refers to further training a pre-trained model on a specific task dataset to adjust its parameters and better suit the target task. During fine-tuning, most layers of the pre-trained model are typically frozen, and only newly added layers are trained or a few key layers are adjusted. This preserves the features learned by the pre-trained model while quickly adapting to the specific needs of the new task. Furthermore, choosing an appropriate learning rate and training epochs is crucial for successful fine-tuning. In other words, fine-tuning involves small-scale training on a pre-trained model for a specific task objective (downstream task) and task data (downstream data) to make minor adjustments to the pre-trained model's parameters, ultimately resulting in a model adapted to the specific task and data.

[0025] Low-rank adaptation is a parameter-efficient fine-tuning (PEFT) method used to adapt pre-trained models (especially large language models) to specific tasks without modifying all of the model's parameters. The core idea of ​​low-rank adaptation is to freeze the main weights of the pre-trained model and inject and train low-rank matrices only in specific layers of the model, thereby simulating the incremental updates of parameters in full fine-tuning.

[0026] See Figure 1As shown in the figure, this application discloses a fine-tuning and fault-tolerant method for an intelligent question-answering model, applied to computing nodes in a distributed computing system. The computing nodes run a pre-trained intelligent question-answering model. The method includes:

[0027] Step S11: Obtain the pre-injected low-rank matrix for model fine-tuning in the intelligent question answering model, and divide the pre-rank matrix into at least two target sub-matrices according to the influence of the matrix parameters in the pre-rank matrix on the model performance.

[0028] First, it should be noted that the intelligent question-answering model in this application can be a domain-specific model, such as those in law, medicine, or finance. Taking the medical field as an example, a knowledge base can be built by integrating multi-source medical data, and labeled data can be used to fine-tune the initial medical question-answering model, continuously updating medical findings. This allows the fine-tuned medical question-answering model to solve the problem of obtaining medical information, assist in diagnosis and health consultation, and provide services by combining medical knowledge and patient needs. Furthermore, each computing node in the distributed computing system is equipped with a Graphics Processing Unit (GPU).

[0029] This application employs a low-rank adaptation method during model fine-tuning. This method freezes the principal weights of the pre-trained model and injects and trains low-rank matrices only in specific layers of the model, thus simulating incremental parameter updates during full fine-tuning. Related methods typically perform redundant backups of the entire low-rank matrix to protect its computational process in terms of fault tolerance. However, this application, after obtaining the pre-injected low-rank matrix into the intelligent question-answering model, first divides the pre-injected low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters on model performance, so as to subsequently perform differentiated protection based on the importance of the matrix parameters.

[0030] Step S12: For any target submatrix, determine the appropriate target encoding algorithm based on the degree of influence of any target submatrix on the model performance, and use the target encoding algorithm to encode any target submatrix at the first preset time point and the second preset time point corresponding to any two adjacent training steps, so as to obtain the first encoding result and the second encoding result.

[0031] In this embodiment, as explained above, the impact of different target sub-matrices on model performance varies. Therefore, this application needs to determine a suitable target encoding algorithm based on the degree of impact of the target sub-matrices on model performance, thereby protecting the target sub-matrices using the target encoding algorithm. That is, this application matches corresponding target encoding algorithms to different target sub-matrices, achieving layered and differentiated data protection. This differentiated design avoids the resource waste caused by applying uniform strength protection to all parameters, allowing limited computing and storage resources to be prioritized for protecting the most critical data. This significantly reduces overall computing and storage overhead while ensuring the reliability of key parameters.

[0032] Furthermore, after determining the appropriate target encoding algorithm, the target submatrix is ​​encoded and calculated at the first and second preset time points of any two adjacent training steps using the target encoding algorithm, so as to obtain the corresponding first encoding result and second encoding result.

[0033] In a specific implementation, the target encoding algorithm is used to encode any target submatrix at a first preset time point and a second preset time point corresponding to any two adjacent training steps, respectively, to obtain a first encoding result and a second encoding result. This includes: after completing the reverse update operation of the previous training step, encoding the target submatrix to obtain a first encoding result; and before performing the forward propagation operation of the next training step, encoding the target submatrix to obtain a second encoding result; wherein, any two adjacent training steps include a previous training step and a next training step.

[0034] First, it should be noted that a complete training step mainly includes four core components: forward propagation, loss calculation, back propagation, and parameter update. Therefore, for any two adjacent training steps, specifically after completing the back propagation and parameter update operations of the previous training step, the target encoding algorithm is used to sequentially encode any target submatrix to obtain a first encoding result, which is then stored in a hash table in CPU memory. Then, before the forward propagation operation of the next training step, the target encoding algorithm is used again to re-encode any target submatrix to obtain a second encoding result, which is then compared with the first encoding result stored in the hash table in CPU memory. In other words, this application can verify whether data corruption has occurred in the target submatrix during the storage idle period between two adjacent training steps. Ideally, from the end of the previous training step to the start of the next, the weight data is frozen in GPU memory and should not be modified by any computational operations. Therefore, if the encoding result changes during this period, it indicates that silent data corruption (SDC) has occurred. For example, if a bit in the GPU's video memory is flipped due to a high-energy particle impact, causing data corruption, this error will not cause the program to crash, but it will contaminate the data and lead to incorrect calculation results.

[0035] Step S13: Determine whether the first encoding result and the second encoding result are consistent. If the first encoding result and the second encoding result are inconsistent, it is determined that a data corruption event has occurred, and the data is recovered using the recovery mechanism corresponding to the data corruption event.

[0036] In this embodiment, by comparing the consistency of the first encoding result and the second encoding result, data corruption events occurring in the preset low-rank matrix during training can be effectively and quickly detected with extremely low computational overhead. By intercepting these data corruption events, the spread of error contamination to the entire model is prevented, thus improving system reliability. Finally, when a data corruption event is detected, the corresponding recovery mechanism is used to recover the data, minimizing training interruption time caused by faults and improving overall training efficiency. The solution proposed in this application can significantly reduce fault tolerance overhead and ensure stability during model fine-tuning.

[0037] For example, assuming the current training step is step=a, after the backward update of step=a-1, it means the weight parameters of the target submatrix have been updated according to the gradient. Therefore, the first encoding result corresponding to the target submatrix is ​​calculated using the target encoding algorithm and stored in a hash table in CPU memory. Then, before the forward propagation of step=a begins, that is, before using the weight parameters of the target submatrix for a new forward calculation, the encoding operation is performed again to obtain the second encoding result. This second encoding result is then compared with the first encoding result stored in the hash table in CPU memory to check whether soft errors such as SDC have contaminated the data. If the first encoding result is consistent with the second encoding result, it means that the data of the target submatrix has not changed at all during the time from the end of step a-1 to the start of step a, and the data is intact and can be used for the forward propagation of step a. If the first encoding result and the second encoding result are inconsistent, it means that the target submatrix has changed during storage, i.e., data contamination has occurred. In this case, the corresponding recovery mechanism is used to recover the data.

[0038] As can be seen, this application achieves layered and differentiated data protection by dividing the pre-injected low-rank matrix in the intelligent question-answering model into at least two target sub-matrices according to the degree of influence of the matrix parameters on model performance, and matching corresponding target encoding algorithms to different target sub-matrices. This differentiated design avoids the resource waste caused by applying uniform strength protection to all parameters, allowing limited computing and storage resources to be prioritized for protecting the most critical data, significantly reducing overall computing and storage overhead while ensuring the reliability of key parameters. Furthermore, by encoding the target sub-matrices at the first and second preset time points of any two adjacent training steps and comparing the two encoding results, this application can effectively and quickly detect data corruption events that occur in the pre-injected low-rank matrix during training, thereby preventing error contamination from spreading to the entire model and improving system reliability. Finally, when a data corruption event is detected, the data is recovered using the recovery mechanism corresponding to the data corruption event, minimizing training interruption time caused by faults and improving overall training efficiency. The solution proposed in this application can significantly reduce fault tolerance overhead and ensure stability during model fine-tuning.

[0039] See Figure 2 and Figure 3 As shown in the illustration, this application discloses a specific method for fine-tuning and fault-tolerant intelligent question-answering models. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0040] Step S21: Obtain the preset low-rank matrix pre-injected into the intelligent question answering model for model fine-tuning; the preset low-rank matrix includes a first preset low-rank matrix and a second preset low-rank matrix, and the rank parameters of the first preset low-rank matrix and the second preset low-rank matrix are the same.

[0041] In this embodiment, the preset low-rank matrices used include a first preset low-rank matrix and a second preset low-rank matrix, denoted as W respectively. A and W B The first and second preset low-rank matrices have the same rank parameter, denoted as r.

[0042] Step S22: Determine the parameter update matrix based on the product of the first preset low-rank matrix and the second preset low-rank matrix; perform singular value decomposition on the parameter update matrix to obtain the singular value decomposition result.

[0043] In this embodiment, the parameter update matrix of LoRA, namely W, is determined based on the product of the first preset low-rank matrix and the second preset low-rank matrix. update =W A ×W B Then, singular value decomposition is performed on the parameter update matrix to obtain the corresponding singular value decomposition results.

[0044] Step S23: Determine the degree of influence of matrix parameters on model performance in the first and second preset low-rank matrices based on the rank parameter and singular value decomposition results, so as to divide the target low-rank matrix into at least two target sub-matrices; wherein the target low-rank matrix is ​​the first preset low-rank matrix or the second preset low-rank matrix.

[0045] It is understandable that the magnitude of singular values ​​represents the importance of the corresponding feature dimension (i.e., matrix rows or matrix columns). Therefore, this embodiment specifically determines the degree of influence of matrix parameters in the first and second preset low-rank matrices on model performance based on the rank parameter and the singular value decomposition results, thereby performing matrix partitioning operations.

[0046] Specifically, the singular value decomposition result includes a target number of singular values ​​arranged in descending order, with the target number being numerically the same as the rank parameter. The target sub-matrix includes a core sub-matrix, a secondary core sub-matrix, and a non-core sub-matrix, where the influence of the core sub-matrix, secondary core sub-matrix, and non-core sub-matrix on model performance decreases sequentially. Correspondingly, based on the rank parameter and the singular value decomposition result, the influence of matrix parameters in the first and second preset low-rank matrices on model performance is determined to divide the target low-rank matrix into at least two target sub-matrices, including: calculating the cumulative energy contribution rate of the singular values ​​sequentially in descending order; comparing the cumulative energy contribution rate with a preset threshold, and dividing the target low-rank matrix into a core sub-matrix, a secondary core sub-matrix, and a non-core sub-matrix based on the comparison result.

[0047] That is, the singular value decomposition result includes r singular values ​​arranged in descending order, denoted as r. The target submatrix includes core submatrix, secondary core submatrix, and non-core submatrix, whose impact on model performance decreases sequentially. In other words, this application will... A and W B The matrix is ​​divided into core sub-matrix, secondary core sub-matrix, and non-core sub-matrix. In a specific implementation, the cumulative energy contribution rate of singular values ​​is calculated sequentially from largest to smallest, and the cumulative energy contribution rate is compared with a preset threshold. Based on the comparison result, the target low-rank matrix is ​​divided into core sub-matrix, secondary core sub-matrix, and non-core sub-matrix.

[0048] The process of calculating the cumulative energy contribution rate of singular values ​​in descending order includes: calculating the sum of squares of all singular values ​​as the total energy; sequentially accumulating the sum of squares of each singular value in descending order to obtain the cumulative energy; and determining the cumulative energy contribution rate based on the ratio of the cumulative energy to the total energy. In other words, this embodiment of the application needs to determine how many singular values ​​can represent most of the matrix's energy. First, the sum of squares of all singular values ​​is calculated as the total energy, i.e., total energy = Then calculate the sum of squares of the first i singular values ​​to obtain the cumulative energy, i.e., cumulative energy = Where 1≤i≤r; the cumulative energy contribution rate can be determined by calculating the ratio of cumulative energy to total energy, and the cumulative energy contribution rate is denoted as . Then we have:

[0049] .

[0050] In a specific implementation, the cumulative energy contribution rate is compared with a preset threshold, and the target low-rank matrix is ​​divided into core sub-matrices, secondary core sub-matrices, and non-core sub-matrices based on the comparison result. This includes: when the cumulative energy contribution rate is greater than a first preset threshold, determining the current first singular value information used to calculate the cumulative energy contribution rate, and determining the matrix row or column in the target low-rank matrix corresponding to the current first singular value information as the core sub-matrix; when the cumulative energy contribution rate is greater than a second preset threshold, determining the current second singular value information used to calculate the cumulative energy contribution rate, and removing the current first singular value information from the current second singular value information to obtain processed singular value information, so as to determine the matrix row or column in the target low-rank matrix corresponding to the processed singular value information as the secondary core sub-matrix; wherein, the first preset threshold is less than the second preset threshold; and determining the matrix rows or columns in the target low-rank matrix other than the core sub-matrix and secondary core sub-matrix as non-core sub-matrices.

[0051] In this embodiment, when When determining the minimum number of singular values ​​that make the relation hold, we identify the first k (k < r) singular values, and then use the matrix columns corresponding to these k singular values ​​in the first preset low-rank matrix, and the matrix rows corresponding to these k singular values ​​in the second preset low-rank matrix, as the core submatrix. Wherein:

[0052] ;

[0053] The calculation method for the secondary core submatrix is ​​similar to that of the core submatrix, except that it takes the matrix rows or columns corresponding to the t singular values ​​that account for 40%-70%, where t+k<r. The matrix rows or columns in the target low-rank matrix other than the core and secondary core submatrix are considered non-core submatrixes.

[0054] Furthermore, the above method also includes: after the current total number of training iterations reaches the target number of iterations, performing the step of dividing the preset low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the preset low-rank matrix on the model performance at preset intervals; and when the current total number of training iterations has not reached the target number of iterations, determining the preset low-rank matrix as a non-core sub-matrix.

[0055] It is understandable that when dividing the matrix into core submatrices, secondary core submatrices, and non-core submatrices, the initial matrix W... A ×W BSince the model hasn't been updated since initialization, a warm-up-like approach is used for partitioning. In deep learning, warm-up typically refers to using a small learning rate initially during training, then gradually increasing it to a preset value. This helps the model begin training stably and avoids oscillations caused by initial gradient instability. In other words, during the initial training phase (warm-up stage), all parameters are updated equally for a period of time to collect sufficient statistical data (i.e., singular values), and then the initial partitioning is performed based on these preliminary importance metrics.

[0056] Specifically, in this embodiment, this means that during the first few iterations of training (e.g., the first 100 steps), the submatrix is ​​not divided temporarily, and the preset low-rank matrix is ​​regarded as a non-core submatrix. During this period, the low-rank matrix W A and W B During a normal update, the system calculates the parameter update matrix and performs singular value decomposition, collecting singular value information. However, this information is not immediately used for partitioning; instead, it is used to calculate statistics such as the cumulative energy contribution rate. After a certain number of steps (i.e., the end of the Warm-up period), the system has collected enough singular value data to preliminarily determine the importance of different directions. Therefore, the initial partitioning is performed at this point, following the aforementioned rules, i.e., based on the cumulative energy contribution rate of the singular values ​​calculated so far, to obtain the core submatrix, secondary core submatrix, and non-core submatrix. Specifically:

[0057] ;

[0058] in, Representation matrix W A and W B The number of columns or rows in the core submatrix, where step is the number of training iterations. Representation matrix W update After singular value decomposition, the cumulative energy of its singular values ​​occupies the top 40% of the matrix rows or columns.

[0059] Furthermore, it should be noted that the initial partition is not fixed. As training progresses, the importance of the parameters changes. Therefore, the matrix partition is performed every preset interval. For example, every 500 training steps, the singular value decomposition is re-executed and the core submatrix, secondary core submatrix, and non-core submatrix are recalculated to ensure that the protection strategy always matches the importance of the current parameters.

[0060] Step S24: For any target submatrix, determine the appropriate target encoding algorithm based on the degree of influence of any target submatrix on the model performance, and use the target encoding algorithm to encode any target submatrix at the first preset time point and the second preset time point corresponding to any two adjacent training steps, so as to obtain the first encoding result and the second encoding result.

[0061] In this embodiment, the above-mentioned determination of the appropriate target encoding algorithm based on the degree of influence of any target sub-matrix on model performance includes: if any target sub-matrix is ​​a core sub-matrix, then the appropriate target encoding algorithm is determined to be an erasure coding algorithm and a hash check algorithm; if any target sub-matrix is ​​a secondary core sub-matrix, then the appropriate target encoding algorithm is determined to be a hash check algorithm; if any target sub-matrix is ​​a non-core sub-matrix, then the appropriate target encoding algorithm is determined to be empty. That is, this application uses different encoding methods for core sub-matrix and secondary core sub-matrix for fault tolerance, while no processing is done for non-core sub-matrix. Specifically, as shown... Figure 3 As shown, the core submatrix requires the simultaneous use of erasure coding and hash verification algorithms. Specifically, the erasure coding algorithm can be the RS coding algorithm, and the hash verification algorithm can be the xxHash algorithm. For the secondary core submatrix, only the xxHash algorithm is used for encoding. That is, this embodiment employs a layered redundancy coding mechanism, providing differentiated protection based on the importance of parameters. It uses computationally expensive erasure coding only for the core submatrix, while using ultra-lightweight hash verification for other parts, achieving a balance between protection strength and performance overhead.

[0062] Specifically, the process of encoding any target submatrix using the target encoding algorithm includes: if any target submatrix is ​​a core submatrix, then redundant blocks are generated for the core submatrix based on the erasure coding algorithm, and the redundant blocks are stored in the processor memory corresponding to the local computing node and the adjacent computing node respectively. Then, the hash value corresponding to the redundant block is calculated based on the hash check algorithm to obtain the corresponding encoding result; if any target submatrix is ​​a secondary core submatrix, then the hash value corresponding to the secondary core submatrix is ​​calculated based on the hash check algorithm to obtain the corresponding encoding result. That is, for the core submatrix, RS code redundancy backup is first used, configured as (k, m), where k is the number of rows or columns of the core submatrix, and m = ceil(k × 0.1), meaning that the RS code supports a maximum of 10% data loss or corruption. In addition, this redundant block is not only stored inside the GPU of the local computing node, but also stored in the CPU memory of the local computing node, and then asynchronously transmitted to the CPU memory of the adjacent computing node for storage through a high-speed interconnect network to form a redundant backup. Then, for redundant blocks of the core submatrix and the secondary core submatrix, a lightweight hash verification algorithm, xxHash, is used to calculate the hash value of each column or row in the secondary core submatrix and store it in a hash table in CPU memory. This method can quickly verify the data with low computational overhead.

[0063] Meanwhile, in order to alleviate computational congestion, the encoding computation of the core submatrix and the secondary core submatrix is ​​performed by starting computation through independent background computation streams such as CUDA_Stream after the backpropagation of this layer is completed, and then the computation is transmitted to the corresponding location for storage after completion.

[0064] Step S25: Determine whether the first encoding result and the second encoding result are consistent. If the first encoding result and the second encoding result are inconsistent, it is determined that a data corruption event has occurred, and the data is recovered using the recovery mechanism corresponding to the data corruption event.

[0065] In this embodiment, the data recovery using the recovery mechanism corresponding to the data corruption event includes: if the data corruption event indicates that the secondary core sub-matrix has data corruption, then read the target snapshot information to perform data recovery on the secondary core sub-matrix based on the target snapshot information; the target snapshot information is the snapshot information saved at the time closest to the time of the corruption; if the data corruption event indicates that the core sub-matrix has data corruption, then determine whether the corruption range of the core sub-matrix exceeds the preset recovery range of the redundant blocks; if the corruption range does not exceed the preset recovery range, then read the redundant blocks from the processor memory corresponding to the local computing node or the adjacent computing node, and perform data recovery on the core sub-matrix based on the erasure coding decoding algorithm; if the corruption range exceeds the preset recovery range, then perform data recovery on the core sub-matrix based on the target snapshot information.

[0066] In other words, when the secondary core submatrix is ​​corrupted, a backtracking mechanism is activated, which reads the most recently saved target snapshot information to recover the secondary core submatrix based on the target snapshot information. When the core submatrix is ​​corrupted, it needs to be determined whether the corruption range of the core submatrix exceeds the preset recovery range of the redundant blocks. As mentioned above, RS codes support a maximum of 10% data loss. If the corruption range of the core submatrix does not exceed the preset recovery range, the redundant blocks can be read from the processor memory of the local computing node or adjacent computing nodes, and the core submatrix can be recovered based on the erasure coding decoding algorithm. However, when the corruption range exceeds the preset recovery range, such as when the corruption range of the core submatrix exceeds 10%, reconstruction based on the redundant blocks is not possible. Therefore, the core submatrix is ​​recovered based on the target snapshot information. In addition, for hard failures such as GPU hardware crashes, the core submatrix and redundant blocks may become unreadable along with the GPU memory. In this case, the target snapshot information can also be used for recovery.

[0067] Furthermore, the above method also includes: after each training step, calculating the L2 norm change rate of a preset low-rank matrix; comparing the L2 norm change rate with a preset change rate threshold; if the L2 norm change rate exceeds the preset change rate threshold, initiating a preset snapshot saving process to save the current snapshot information; otherwise, prohibiting the execution of the step of initiating the preset snapshot saving process; wherein, the current snapshot information includes the current low-rank matrix information, optimizer state, and gradient information. It is understood that, regarding whether to save snapshot information, this embodiment of the application determines whether to save snapshot information by calculating whether the L2 norm change rate of the preset low-rank matrix is ​​greater than the preset change rate threshold. If the L2 norm change rate exceeds the preset change rate threshold, it indicates that the model has undergone significant learning, and the preset snapshot saving process is initiated to trigger a new round of snapshot information saving; otherwise, it is not necessary to save the current snapshot information. Taking matrix W as an example... A For example, suppose the current step = a, then if the matrix W of step = a A (a) Compared to matrix W with step=a-1 A The rate of change of the L2 norm of (a-1) is greater than the preset rate of change threshold. Then, the snapshot saving process is initiated. The formula for calculating the L2 norm rate of change is as follows:

[0068] .

[0069] In other words, the snapshots described above are not saved based on time, but rather are a snapshot mechanism that is dynamically triggered based on the magnitude of changes in the model's state. This enables intelligent, on-demand state saving, greatly reducing unnecessary I / O and storage overhead. After a snapshot is triggered, the current low-rank matrix information (i.e., W) is saved. A and WB Optimizer state The gradient information *g* used to calculate this update is asynchronously saved as an incremental snapshot to local CPU memory and then asynchronously transmitted to a remote persistent storage device in the background. The local CPU memory only stores the most recent snapshot. That is, the snapshot content is minimized, containing only the current low-rank matrix, the corresponding optimizer state, and the gradient information that led to this update. Correspondingly, when reading the target snapshot information, it is first read from local CPU memory; if the target snapshot information cannot be read, it is then read from the remote persistent storage device.

[0070] The calculation process for the preset rate of change threshold specifically includes: processing the rank parameter based on preset parameter processing rules to obtain the processed parameter; wherein the preset parameter processing rules are constructed based on the square root operation rule and the reciprocal operation rule; determining the preset default rate of change, and determining the maximum value between the default rate of change and the processed parameter as the preset rate of change threshold. That is, the preset rate of change threshold needs to be determined based on the rank parameter r, and its expression is as follows:

[0071] ;

[0072] The default rate of change is 0.05.

[0073] Furthermore, the data recovery process based on the target snapshot information includes: overwriting the data on the local computing node based on the current low-rank matrix information and optimizer state in the target snapshot information; and performing gradient replay operations based on the gradient information in the target snapshot information to restore the data to the state before the data corruption event. That is, when an inconsistency is detected between the hash value on a GPU and the one stored in CPU memory, the corresponding target snapshot information is first loaded from the local CPU memory to the GPU, and the LoRA matrix W in the target snapshot information is used... A and W B and optimizer status The data on the original GPU is overwritten, and the gradient information g from the target snapshot is read to perform gradient replay, restoring the GPU to its state before the data corruption event. In other words, when an error is detected, only the low-rank matrix and optimizer state need to be restored from the most recent snapshot, and the updates within the snapshot interval are compensated by gradient replay—that is, performing an optimizer step once using the gradients saved in the snapshot—thus avoiding time-consuming data loading and backpropagation recalculation.

[0074] The fault-tolerance method commonly used in related technologies is the checkpoint / restart mechanism. However, the traditional checkpoint / restart mechanism has huge recovery overhead and low efficiency. This mechanism requires all healthy nodes to stop working, wait for the failed node to recover, and roll back to the previous checkpoint. This leads to wasted computational overhead, as all computational results from the previous checkpoint to the time of failure are invalidated and need to be re-executed. In contrast, the incremental snapshot and backtracking mechanism disclosed in this application avoids time-consuming data loading and backpropagation recomputation through gradient replay. At the same time, it is superimposed with a dynamic triggering mechanism based on the model's L2 norm change rate, which only performs backups when the model undergoes significant learning, achieving intelligent and on-demand state saving, greatly reducing unnecessary I / O and storage overhead, and reducing computational waste. Furthermore, the incremental synchronous recovery mechanism of this application, combined with the LoRA fine-tuning effect, only synchronizes small LoRA incremental parameters when recovering a failed node, rather than the entire state, thereby reducing resource overhead. In addition, the hierarchical redundancy coding and hash verification mechanism of this application can detect data corruption in memory in real time, avoiding the threat of SDC.

[0075] As can be seen, this application employs hierarchical redundancy coding for the low-rank matrix, applying different levels of protection to parts of varying importance within the low-rank matrix. This is achieved by dividing the low-rank matrix into core sub-matrices with a greater impact on model performance, secondary core sub-matrices with relatively smaller impact, and non-core sub-matrices with less significant impact. An erasure coding technique based on RS codes is applied to the core sub-matrices to generate redundant blocks. When some data in the core sub-matrices is corrupted, these redundant blocks can be used to accurately reconstruct the data through mathematical operations, achieving data recovery. Lightweight hash verification is used for the non-secondary core sub-matrices. Furthermore, real-time comparisons are performed before and after training steps at certain intervals to quickly detect data corruption with extremely low computational overhead. Moreover, this application calculates the L2 norm change rate of the low-rank matrix after each optimizer step, triggering a snapshot only when the change rate exceeds a preset threshold. The snapshot content is minimized, containing only the current low-rank matrix, the corresponding optimizer state, and the gradient information that led to the update, significantly reducing storage overhead.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0077] See Figure 4 As shown in the figure, this application discloses a fine-tuning and fault-tolerant device for an intelligent question-answering model, applied to a computing node in a distributed computing system. The computing node runs a pre-trained intelligent question-answering model. The device includes:

[0078] The low-rank matrix partitioning module 11 is used to obtain the preset low-rank matrix that is pre-injected into the intelligent question answering model for model fine-tuning, and to partition the preset low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the preset low-rank matrix on the model performance.

[0079] The encoding calculation module 12 is used to determine a suitable target encoding algorithm for any target submatrix based on the degree of influence of any target submatrix on the model performance, and to use the target encoding algorithm to perform encoding calculation on any target submatrix at the first preset time point and the second preset time point corresponding to any two adjacent training steps, so as to obtain the first encoding result and the second encoding result.

[0080] The data recovery module 13 is used to determine whether the first encoding result and the second encoding result are consistent. If the first encoding result and the second encoding result are inconsistent, a data corruption event is determined to have occurred, and the data is recovered using the recovery mechanism corresponding to the data corruption event.

[0081] As can be seen, this application achieves layered and differentiated data protection by dividing the pre-injected low-rank matrix in the intelligent question-answering model into at least two target sub-matrices according to the degree of influence of the matrix parameters on model performance, and matching corresponding target encoding algorithms to different target sub-matrices. This differentiated design avoids the resource waste caused by applying uniform strength protection to all parameters, allowing limited computing and storage resources to be prioritized for protecting the most critical data, significantly reducing overall computing and storage overhead while ensuring the reliability of key parameters. Furthermore, by encoding the target sub-matrices at the first and second preset time points of any two adjacent training steps and comparing the two encoding results, this application can effectively and quickly detect data corruption events that occur in the pre-injected low-rank matrix during training, thereby preventing error contamination from spreading to the entire model and improving system reliability. Finally, when a data corruption event is detected, the data is recovered using the recovery mechanism corresponding to the data corruption event, minimizing training interruption time caused by faults and improving overall training efficiency. The solution proposed in this application can significantly reduce fault tolerance overhead and ensure stability during model fine-tuning.

[0082] For a description of the features in the embodiment corresponding to the fine-tuning fault-tolerant device of the intelligent question-answering model, please refer to the relevant description of the embodiment corresponding to the fine-tuning fault-tolerant method of the intelligent question-answering model, which will not be repeated here.

[0083] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in the embodiments of the fine-tuning fault-tolerant method for any of the above-described intelligent question-answering models.

[0084] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in the embodiments of the fine-tuning and fault-tolerant method of any of the above-described intelligent question-answering models when it is run.

[0085] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0086] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the embodiments of the fine-tuning and fault-tolerant method for any of the above-described intelligent question-answering models.

[0087] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the embodiments of the fine-tuning fault-tolerant method for any of the above-described intelligent question-answering models.

[0088] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0089] The foregoing has provided a detailed description of the fine-tuning and fault-tolerant method, device, medium, and product for an intelligent question-answering model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A fine-tuning and fault-tolerant method for an intelligent question-answering model, characterized in that, The method, applied to computing nodes in a distributed computing system, wherein a pre-trained intelligent question-answering model runs on the computing nodes, includes: Obtain the preset low-rank matrix pre-injected into the intelligent question answering model for model fine-tuning, and divide the preset low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the preset low-rank matrix on the model performance; For any target submatrix, a suitable target encoding algorithm is determined based on the degree of influence of the target submatrix on the model performance. The target encoding algorithm is then used to encode the target submatrix at the first preset time point and the second preset time point corresponding to any two adjacent training steps, respectively, to obtain the first encoding result and the second encoding result. Determine whether the first encoding result and the second encoding result are consistent. If the first encoding result and the second encoding result are inconsistent, determine that a data corruption event has occurred, and use the recovery mechanism corresponding to the data corruption event to recover the data.

2. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 1, characterized in that, The preset low-rank matrix includes a first preset low-rank matrix and a second preset low-rank matrix, wherein the first preset low-rank matrix and the second preset low-rank matrix have the same rank parameter. Accordingly, the step of dividing the preset low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the preset low-rank matrix on the model performance includes: The parameter update matrix is ​​determined based on the product of the first preset low-rank matrix and the second preset low-rank matrix. Perform singular value decomposition on the parameter update matrix to obtain the singular value decomposition result; Based on the rank parameter and the singular value decomposition result, determine the degree of influence of matrix parameters in the first preset low-rank matrix and the second preset low-rank matrix on model performance, so as to divide the target low-rank matrix into at least two target sub-matrices; The target low-rank matrix is ​​either the first preset low-rank matrix or the second preset low-rank matrix.

3. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 2, characterized in that, The singular value decomposition result includes a target number of singular values ​​arranged in descending order, and the target number is numerically the same as the rank parameter; the target submatrix includes a core submatrix, a secondary core submatrix, and a non-core submatrix, wherein the influence of the core submatrix, the secondary core submatrix, and the non-core submatrix on the model performance decreases in that order. Accordingly, determining the degree of influence of matrix parameters in the first and second preset low-rank matrices on model performance based on the rank parameter and the singular value decomposition result, so as to divide the target low-rank matrix into at least two target sub-matrices, includes: Calculate the cumulative energy contribution rate of singular values ​​in descending order; The cumulative energy contribution rate is compared with a preset threshold, and the target low-rank matrix is ​​divided into the core sub-matrix, the secondary core sub-matrix, and the non-core sub-matrix based on the comparison result.

4. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 3, characterized in that, The calculation of the cumulative energy contribution rate of singular values ​​in descending order includes: Calculate the sum of squares of all singular values ​​and use it as the total energy; The cumulative energy is obtained by summing the squares of the singular values ​​in descending order. The cumulative energy contribution rate is determined based on the ratio of the cumulative energy to the total energy.

5. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 3, characterized in that, The step of comparing the cumulative energy contribution rate with a preset threshold and dividing the target low-rank matrix into the core sub-matrix, the secondary core sub-matrix, and the non-core sub-matrix based on the comparison result includes: When the cumulative energy contribution rate is greater than the first preset threshold, the current first singular value information used to calculate the cumulative energy contribution rate is determined, and the matrix row or matrix column in the target low-rank matrix corresponding to the current first singular value information is determined as the core submatrix; When the cumulative energy contribution rate is greater than the second preset threshold, the current second singular value information used to calculate the cumulative energy contribution rate is determined, and the current first singular value information is removed from the current second singular value information to obtain the processed singular value information, so as to determine the matrix row or matrix column in the target low-rank matrix corresponding to the processed singular value information as the secondary core submatrix; wherein, the first preset threshold is less than the second preset threshold; The matrix rows or columns in the target low-rank matrix other than the core submatrix and the secondary core submatrix are determined as the non-core submatrix.

6. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 3, characterized in that, Also includes: After the current total number of training iterations reaches the target number of iterations, the step of dividing the preset low-rank matrix into at least two target sub-matrices according to the degree of influence of the matrix parameters in the preset low-rank matrix on the model performance is performed at preset intervals. When the current total number of training iterations has not reached the target number of iterations, the preset low-rank matrix is ​​determined as the non-core submatrix.

7. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 3, characterized in that, The method for determining the appropriate target encoding algorithm based on the degree of influence of any target sub-matrix on model performance includes: If any of the target submatrixes is the core submatrix, then the appropriate target encoding algorithms are determined to be erasure coding algorithm and hash verification algorithm; If any of the target submatrixes is the secondary core submatrix, then the appropriate target encoding algorithm is determined to be the hash verification algorithm; If any of the target submatrices is a non-core submatrice, then the corresponding target encoding algorithm is determined to be empty.

8. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 7, characterized in that, The process of encoding and calculating any target submatrix using the target encoding algorithm includes: If any of the target submatrixes is the core submatrix, then redundant blocks are generated for the core submatrix based on the erasure coding algorithm, and the redundant blocks are stored in the processor memory corresponding to the local computing node and the adjacent computing node respectively. Then, the hash value corresponding to the redundant blocks is calculated based on the hash verification algorithm to obtain the corresponding encoding result. If any of the target submatrixes is the secondary core submatrix, then the hash value corresponding to the secondary core submatrix is ​​calculated based on the hash verification algorithm to obtain the corresponding encoding result.

9. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 8, characterized in that, The data recovery using the recovery mechanism corresponding to the data corruption event includes: If the data corruption event indicates that the secondary core submatrix has suffered data corruption, then the target snapshot information is read to perform data recovery on the secondary core submatrix based on the target snapshot information; the target snapshot information is the snapshot information saved at the time closest to the time of the corruption. If the data corruption event indicates that the core submatrix has been corrupted, then it is determined whether the corruption range of the core submatrix exceeds the preset recovery range of the redundant block; If the damage range does not exceed the preset recovery range, the redundant block is read from the processor memory corresponding to the local computing node or the adjacent computing node, and the core submatrix is ​​recovered based on the erasure coding decoding algorithm. If the damage range exceeds the preset recovery range, then data recovery is performed on the core submatrix based on the target snapshot information.

10. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 9, characterized in that, Also includes: After each training step, the L2 norm change rate of the preset low-rank matrix is ​​calculated; The L2 norm change rate is compared with a preset change rate threshold. If the L2 norm change rate exceeds the preset change rate threshold, a preset snapshot saving process is initiated to save the current snapshot information; otherwise, the step of initiating the preset snapshot saving process is prohibited. The current snapshot information includes the current low-rank matrix information, optimizer state, and gradient information.

11. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to claim 10, characterized in that, The process of data recovery based on target snapshot information includes: Data on the local computing node is overwritten based on the current low-rank matrix information and optimizer state in the target snapshot information; A gradient replay operation is performed based on the gradient information in the target snapshot information to restore the data to its state before the data corruption event occurred.

12. The fine-tuning and fault-tolerant method for the intelligent question-answering model according to any one of claims 1 to 11, characterized in that, The step of using the target encoding algorithm to encode any target submatrix at a first preset time point and a second preset time point corresponding to any two adjacent training steps, respectively, to obtain a first encoding result and a second encoding result, includes: After completing the reverse update operation of the previous training step, the encoding calculation is performed on any of the target submatrices to obtain the first encoding result; Before performing the forward propagation operation in the next training step, the encoding calculation is performed on any of the target submatrices to obtain the second encoding result; Wherein, any two adjacent training steps include the previous training step and the next training step.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the fine-tuning fault-tolerant method for the intelligent question-answering model as described in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the fine-tuning and fault-tolerant method for the intelligent question-answering model as described in any one of claims 1 to 12.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the fine-tuning and fault-tolerant method for the intelligent question-answering model as described in any one of claims 1 to 12.

Images