Model training method and device, equipment and storage medium

Abstract

The invention discloses a model training method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence, and the model training method comprises the steps: obtaining a training sample set; performing multi-stage pre-training on an initial model according to the training sample set to obtain a pre-trained model; wherein the initial model is obtained by carrying out structure transformation on the original model by utilizing a pre-constructed multi-head potential attention module. According to the method, the multi-head potential attention module is used for carrying out structure transformation on the original model, so that the structure of the initial model is completely benchmarked with the structure of the MLA multi-head potential attention module, then multi-stage pre-training is carried out on the initial model according to the training sample set, the utilization rate of training data is effectively improved, and the model training effect is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device and storage medium. Background Technology

[0002] The MLA (Multi-head Latent Attention) architecture excels in balancing model efficiency and performance: during training, MLA employs MHA (Multi-Head Attention) to ensure feature representation capabilities. During inference, it switches to MQA (Multi-Query Attention) to optimize computational efficiency. Compared to GQA (Group-Query Attention), MLA offers stronger expressive power without significantly increasing computational cost.

[0003] Current mainstream models (e.g., the Qwen series of large models) are built based on the GQA mechanism. While methods like TransMLA and MHA2MLA can be used to transform other forms of attention modules in a model into an MLA-like structure, a universal transformation from any attention mechanism to MLA is still not possible. Furthermore, the availability of high-quality training data for the modified model is very limited. If the modified model is trained end-to-end, it is prone to converging to poor local extrema, ultimately resulting in poor training performance and failing to fully leverage the performance advantages of the MLA structure. Summary of the Invention

[0004] The main purpose of this application is to provide a model training method, apparatus, device and storage medium, which aims to improve the training effect of the model.

[0005] To achieve the above objectives, this application proposes a model training method, comprising: Obtain the training sample set; Based on the training sample set, the initial model is pre-trained in multiple stages to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

[0006] In one embodiment, the training sample set includes a first-stage training sample set and a second-stage training sample set; The step of performing multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model includes: Each training sample in the first stage training sample set is input into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model. Based on the intermediate features corresponding to each training sample, the model parameters associated with the feature processing module in the initial model are optimized to obtain an intermediate model; Each training sample in the second-stage training sample set is input into the intermediate model to obtain the first prediction result output by the output layer of the intermediate model; Based on each of the first prediction results, the model parameters of the intermediate model are optimized to obtain the pre-trained model.

[0007] In one embodiment, the feature processing module includes multiple cascaded Transformer modules; The step of inputting each training sample from the first-stage training sample set into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model includes: For each training sample in the first phase training sample set: The training samples are input into the initial model to obtain the intermediate features output by each Transformer module; The step of optimizing the model parameters associated with the feature processing module in the initial model based on the intermediate features corresponding to each training sample to obtain an intermediate model includes: For each Transformer module, the feature loss value of the Transformer module is determined based on the intermediate features output by the Transformer module. Based on the feature loss value of the Transformer module, the model parameters associated with the Transformer module are optimized to obtain the intermediate model.

[0008] In one embodiment, the second-stage training sample set is obtained based on the following steps: Acquire several multi-dimensional data, including general knowledge data, mathematical code data, and thought chain data; According to different division ratios, the general knowledge data, the mathematical code data, and the thought chain data are divided into different categories. Based on the segmented general knowledge data, mathematical code data, and thought chain data, the second-stage training sample set is formed.

[0009] In one embodiment, the initial model is obtained by structural modification according to the following steps: Obtain the original model and the multi-head potential attention module; The attention layer of the Transformer module in the original model is replaced with the multi-head latent attention module, and the parameters of the multi-head latent attention module are randomly initialized.

[0010] In one embodiment, after performing multi-stage pre-training on the initial model based on the training sample set to obtain the pre-trained model, the method further includes: Obtain the training dataset for the target domain; Based on the training dataset of the target domain, the pre-trained model is post-trained to obtain the target model.

[0011] In one embodiment, the step of post-training the pre-trained model based on the training dataset of the target domain to obtain the target model includes: Based on each piece of training data in the training dataset, a second prediction result corresponding to each piece of training data is obtained by using the pre-trained model. The model loss value is calculated based on the second prediction result corresponding to any of the training data. Based on the model loss value, the model parameters of the pre-trained model are optimized to obtain the target model.

[0012] Furthermore, to achieve the above objectives, this application also proposes a model training apparatus, which includes: The acquisition module is used to acquire the training sample set; The pre-training module is used to perform multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

[0013] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model training method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the model training method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the model training method described above.

[0016] This application provides a model training method, apparatus, device, and storage medium. The model training method includes: acquiring a training sample set; and performing multi-stage pre-training on an initial model based on the training sample set to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-constructed multi-head latent attention module (MLA). By using the MLA MLA module to modify the structure of the original model, the structure of the initial model is made to perfectly match the structure of the MLA MLA module. Furthermore, by performing multi-stage pre-training on the initial model based on the training sample set, the utilization rate of training data is effectively improved, thereby enhancing the model training effect. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the model training method of this application in Embodiment 1. Figure 2 This is a flowchart illustrating the second embodiment of the model training method in this application. Figure 3 This is a flowchart illustrating the model training method of embodiment three of this application; Figure 4 This is a model structure diagram provided in one embodiment of this application; Figure 5 This is a flowchart illustrating the model training method of embodiment four in this application; Figure 6 This is a schematic diagram of the module structure of the model training device according to an embodiment of this application; Figure 7 This is a schematic diagram of the hardware operating environment involved in the model training method in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, big data service platform, or model training system capable of performing the above functions. The following description uses a model training system as an example to illustrate this embodiment and the subsequent embodiments.

[0024] The MLA (Multi-head Latent Attention) architecture excels in balancing model efficiency and performance: during training, MLA employs MHA (Multi-Head Attention) to ensure feature representation capabilities. During inference, it switches to MQA (Multi-Query Attention) to optimize computational efficiency. Compared to GQA (Group-Query Attention), MLA offers stronger expressive power without significantly increasing computational cost.

[0025] However, current mainstream models (such as the Qwen series of large models) are built based on the GQA mechanism. Currently, methods like TransMLA and MHA2MLA can be used to transform other forms of attention modules in the model into structures approximating MLA. MHA2MLA performs joint SVD on the key and value projection matrices to obtain a shared dimensionality reduction matrix for the key and value, and groups the feature dimensions of the query and key, splitting them into parts with and without positional encoding, thus resembling MLA in form. Compared to MHA2MLA, TransMLA transforms the part of the key without rotational positional encoding from GQA to MQA, further approaching MLA in form. However, it is still impossible to achieve a general conversion from arbitrary attention mechanisms to MLA. Taking the Qwen2.5 and Qwen3 series models as examples: the Qwen2.5 model adds a bias term to the Query, Key, and Value mapping matrix. If the MLA model still has a bias term, it will be unable to reuse the matrix absorption code provided by vllm during inference. The Qwen3 model adds RMS Norm (Root Mean Square Layer Normalization) to the Query and Key mapping matrix. This operation is a non-linear operation, which makes it impossible for the model to be converted through the linear transformation of matrix factorization.

[0026] Furthermore, the amount of high-quality training data that can be collected for the modified model is very limited. If the modified model is trained directly in an end-to-end manner during the training process, the model is prone to converge to poor local extrema, which ultimately leads to poor training results and makes it difficult to give full play to the performance advantages of the MLA structure.

[0027] To address the aforementioned problems, this application provides a model training method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the model training method of this application. In this embodiment, the model training method includes the following steps: Step S11: Obtain the training sample set; It should be noted that the training sample set includes general knowledge data, mathematical code data, and thought chain data. General knowledge data refers to general-domain text data, including basic language, everyday common sense, and general knowledge; its core objective is to enable the model to acquire basic common sense abilities. Mathematical code data focuses on structured or semi-structured data related to numerical calculations, logical deduction, and programming implementations; its core objective is to improve the model's precise logical thinking ability. Thought chain data refers to labeled data with intermediate reasoning steps. For example, the format of thought chain data is: Question, Reasoning Step 1, Reasoning Step 2, Reasoning Step N, Answer; its core objective is to upgrade the model from directly outputting answers to learning to deduce logic step by step and solve complex problems.

[0028] Step S12: Based on the training sample set, perform multi-stage pre-training on the initial model to obtain a pre-trained model; wherein, the initial model is obtained by structurally modifying the original model using a pre-constructed multi-head latent attention module.

[0029] It should be noted that the original model is the Qwen model. The Attention structure of the original model and the modified initial model are different. For the MLA weights in the Attention part, matrix factorization can be used to assign initial values ​​to ensure that the model has certain initial performance and reduce the difficulty of model training. However, because the Qwen3 series introduces the non-linear operation of QK-Norm root mean square layer normalization in the Attention part, matrix factorization cannot be used. Therefore, in this embodiment, the initial model strictly matches the MLA structure of Deepseek-V3 / R1 in terms of structure to adapt to the MLA ecosystem.

[0030] To improve the utilization of training data and enhance the model training effect, in this embodiment, the initial model is pre-trained in multiple stages based on the training sample set. Optionally, the training sample set includes a first-stage training sample set and a second-stage training sample set. The multi-stage pre-training includes feature-level training and model-level training processes.

[0031] Regarding the feature-level training process: Due to the accumulation of error propagation between Transformer modules, the model is prone to convergence to poor extreme points, ultimately leading to poor training results. Therefore, a coarse-grained training method at the Transformer level is used to train the initial model, focusing on features rather than the model's output during training. Optionally, each training sample from the first-stage training sample set is input into the initial model to optimize the model parameters based on the intermediate features corresponding to each training sample output by the feature processing module.

[0032] Regarding the model-level training process: As feature-level training progresses, end-to-end training of the MLA-modified model no longer results in severe error accumulation. Therefore, in the later stages of training, a gradual transition is made to using model-level fine-grained training methods to train the model and supervise its output. Optionally, each training sample from the second-stage training sample set can be input into the feature-level trained model to optimize the model parameters based on the final output, thereby improving the training effect and fully leveraging the performance advantages of the MLA structure.

[0033] This embodiment employs the above-described scheme: obtaining a training sample set; performing multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model; wherein the initial model is obtained by structurally modifying the original model using a pre-constructed multi-head latent attention module. By using the multi-head latent attention module to structurally modify the original model, the structure of the initial model is made to completely align with the structure of the MLA multi-head latent attention module. Furthermore, by performing multi-stage pre-training on the initial model based on the training sample set, the utilization rate of training data is effectively improved, significantly enhancing the model training effect.

[0034] In one feasible implementation, refer to Figure 2 , Figure 2 This is a flowchart illustrating Embodiment 2 of the model training method of this application. The initial model is obtained by structural modification according to the following steps: Step S21: Obtain the original model and the multi-head potential attention module; Step S22: Replace the attention layer of the Transformer module in the original model with the multi-head latent attention module, and randomly initialize the parameters of the multi-head latent attention module.

[0035] Understandably, the Deepseek open-source MLA (Multi-Head Attention) module is obtained, the attention layer in the original model is determined, and then the attention layer of the Transformer module in the original model is replaced with the MLA module. This ensures that the initial model structurally closely matches the MLA structure. During training, MLA uses MHA (Multi-Head Attention) to ensure feature representation capabilities, while during inference, it switches to MQA (Multi-Query Attention) to optimize computational efficiency. It should be noted that during the MLA structure replacement process, the internal dimensionality of MLA differs from that of the original attention module. Strict alignment of the hidden layer dimensions, number of heads, and low-rank projection dimensions is necessary; otherwise, tensor shape mismatch errors may occur.

[0036] Furthermore, to avoid interference from the original model's parameters on MLA inference optimization during subsequent model training, this embodiment randomly initializes the parameters of the multi-head latent attention module; that is, it is not necessary to assign initial values ​​to the parameters of the multi-head latent attention module. During training, the intermediate features output by the randomly initialized Transformer module are used for training. The Transformer module includes the MLA module. At this time, the MLA parameters will quickly learn how to generate effective features through low-rank projection under the guidance of the supervision signal, without relying on the initial parameters. After the distribution of intermediate features stabilizes, the parameters are fine-tuned through end-to-end output supervision, allowing the randomly initialized parameters to further adapt to the final task while maintaining the inference optimization characteristics of MLA.

[0037] This embodiment employs the above-described scheme: obtaining the original model and the multi-head latent attention module; replacing the attention layer of the Transformer module in the original model with the multi-head latent attention module; and randomly initializing the parameters of the multi-head latent attention module. This achieves the goal of directly replacing the attention layer of the Transformer module in the original model with the multi-head latent attention module, thereby ensuring that the attention layer structure of the original model strictly aligns with the MLA structure, thus improving the model training effect.

[0038] In one feasible implementation, refer to Figure 3 , Figure 3 This is a flowchart illustrating the model training method of Embodiment 3 of this application. Based on the training sample set, the initial model undergoes multi-stage pre-training to obtain a pre-trained model, including: Step S31: Input each training sample in the first stage training sample set into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model. It should be noted that the training sample set in the first phase includes general knowledge data, which refers to general domain text data, including basic language, common sense, and general knowledge.

[0039] It should be noted that, referring to Figure 4 , Figure 4 This is a model structure diagram provided in one embodiment of the present application. The initial model includes an input layer, an embedding layer, a feature processing module, a normalization layer, a head layer, and an output layer. The feature processing module includes multiple cascaded Transformer modules. In one embodiment, the parameters of the input layer, the embedding layer, the normalization layer, the head layer, and the output layer are frozen, and each Transformer module is set to a learnable state. That is, during training, the model parameters of each Transformer module need to be iteratively optimized.

[0040] For each training sample in the first stage training sample set, the following steps are performed: the training sample is input into the initial model to extract the initial features of the training sample through the embedding layer, and then the initial features of the training sample are used to extract intermediate features using the first Transformer module. The output of the previous Transformer module is used as the input of the next Transformer module. The intermediate features output by the first Transformer module are input into the second Transformer module, and this operation is repeated to obtain the intermediate features input by each Transformer module.

[0041] Step S32: Based on the intermediate features corresponding to each training sample, optimize the model parameters associated with the feature processing module in the initial model to obtain an intermediate model; It should be noted that, due to the accumulation of error propagation between Transformer modules, the model is prone to convergence to poor extreme points. Therefore, in this embodiment, a coarse-grained training method at the Transformer level is used to train the model. During training, the focus is on the features output by the Transformer modules rather than the model's output results. Understandably, for each training sample in the first-stage training sample set: after obtaining the intermediate features corresponding to the training sample output by each Transformer module, the feature loss value of each Transformer module is calculated based on the intermediate features output by each Transformer module; furthermore, based on the feature loss value of each Transformer module, the model parameters associated with each Transformer module are optimized to obtain the intermediate model. By optimizing the Transformer modules based on the features output by the Transformer modules, the accumulation of error propagation between Transformer modules is effectively reduced, thereby effectively improving the final training effect of the model.

[0042] Step S33: Input each training sample in the second stage training sample set into the intermediate model to obtain the first prediction result output by the output layer of the intermediate model; Step S34: Based on each of the first prediction results, optimize the model parameters of the intermediate model to obtain the pre-trained model.

[0043] It should be noted that the training sample set for the second phase includes general knowledge data, mathematical code data, and thought chain data.

[0044] It should be noted that as feature-level training progresses, end-to-end training of the intermediate model no longer results in severe error accumulation. In this embodiment, each training sample from the second-stage training sample set is input into the intermediate model to obtain the first prediction result output by the output layer of the intermediate model. Further, based on the first prediction result of each training sample and its associated sample label, a loss value is calculated. Then, the model parameters of the intermediate model are optimized in conjunction with the loss value until a preset termination condition is reached to obtain the pre-trained model. The preset termination condition includes the model reaching convergence or the number of iterations reaching a preset number of iterations. By gradually transitioning to a model-level fine-grained training process in the later stages of training, that is, supervising the model's output during training, the basic capabilities of the model are improved.

[0045] This embodiment employs the above-described scheme: Each training sample from the first-stage training sample set is input into the initial model to obtain intermediate features corresponding to each training sample output by the feature processing module in the initial model; based on the intermediate features corresponding to each training sample, the model parameters associated with the feature processing module in the initial model are optimized to obtain an intermediate model; each training sample from the second-stage training sample set is input into the intermediate model to obtain a first prediction result output by the output layer in the intermediate model; based on each first prediction result, the model parameters of the intermediate model are optimized to obtain the pre-trained model. This achieves feature-level training of the model based on the intermediate features corresponding to each training sample output by the feature processing module in the initial model, reducing the accumulation of error propagation between Transformer modules, and then performing end-to-end iterative training of the intermediate model. That is, coarse-grained feature-level training is performed on the model in the early stage of training, and fine-grained model-level training is performed in the later stage of training, effectively improving the utilization rate of training data and enhancing the training effect of the model.

[0046] In one feasible implementation, each training sample in the first-stage training sample set is input into the initial model to obtain intermediate features corresponding to each training sample output by the feature processing module in the initial model, including: For each training sample in the first phase training sample set: Step S41: Input the training samples into the initial model to obtain the intermediate features output by each Transformer module; The step of optimizing the model parameters associated with the feature processing module in the initial model based on the intermediate features corresponding to each training sample to obtain an intermediate model includes: Step S42: For each Transformer module, determine the feature loss value of the Transformer module based on the intermediate features output by the Transformer module. Step S43: Optimize the model parameters associated with the Transformer module based on the feature loss value of the Transformer module to obtain the intermediate model.

[0047] In one embodiment, the Transformer module may sequentially include a multi-head latent attention module, a first residual connection layer, a first normalization layer, a feedforward network, a second residual connection layer, and a second normalization layer.

[0048] In this embodiment, for each training sample in the first-stage training sample set, the following operations are performed: the training sample is input into the initial model, and the initial features of the training sample extracted by the embedding layer are then input into the multi-head latent attention module in the Transformer module to obtain the output of the multi-head latent attention module. Further, the initial features input to the multi-head latent attention module and the output of the multi-head latent attention module are concatenated using the first residual connection layer to obtain a first concatenated feature. Then, the first concatenated feature is normalized using the first normalization layer to obtain a normalized feature. Further, the normalized feature is nonlinearly transformed using the feedforward network to obtain a nonlinearly transformed feature. Then, the normalized feature input to the feedforward network and the nonlinearly transformed feature output by the feedforward network are concatenated using the second residual connection layer to obtain a second concatenated feature. Further, the second concatenated feature is normalized using the second normalization layer to obtain an intermediate feature corresponding to the Transformer module, wherein the intermediate feature corresponding to the Transformer module is used as the input of the next Transformer module. The subsequent processing steps of the Transformer module are similar to those described above, and will not be repeated here. This yields the intermediate features output by each Transformer module.

[0049] It should be noted that the training samples are input into the original model to obtain the target features output by each Transformer module. Further, for each Transformer module: based on the target features associated with the Transformer module and the intermediate features output by the Transformer module, the feature loss value of the Transformer module is calculated. That is, the feature loss value of the Transformer module is calculated by using the target features and intermediate features of Transformer modules at the same level through a pre-defined loss function, such as mean squared error (MSE) or cosine similarity. Further, based on the feature loss value of the Transformer module, the model parameters associated with the Transformer module are optimized to obtain the intermediate model.

[0050] This embodiment employs the above-described scheme: inputting the training samples into the initial model to obtain intermediate features output by each Transformer module; for each Transformer module, determining its feature loss value based on the intermediate features; and optimizing the model parameters associated with the Transformer module based on its feature loss value to obtain the intermediate model. This achieves the optimization of model parameters associated with the Transformer module based on the intermediate features output by the Transformer module, reducing the accumulation of error propagation between Transformer modules and effectively improving the model training effect.

[0051] In one feasible implementation, the second-stage training sample set is obtained based on the following steps: Step S51: Obtain several multi-dimensional data, including general knowledge data, mathematical code data, and thought chain data; It's important to clarify that general knowledge data refers to textual data in common domains, including basic language, everyday common sense, and general knowledge. The core objective is to enable models to acquire basic common sense abilities. Mathematical code data focuses on structured or semi-structured data related to numerical calculations, logical deduction, and programming implementations. The core objective is to improve the model's precise logical thinking ability. Thinking chain data refers to labeled data with intermediate reasoning steps. For example, the format of thinking chain data is "question, reasoning step 1, reasoning step 2, reasoning step N, answer." The core objective is to upgrade the model from directly outputting answers to learning to deduce logic step by step and solve complex problems.

[0052] Step S52: Divide the general knowledge data, mathematical code data, and thought chain data into different division ratios respectively; Step S53: Based on the divided general knowledge data, mathematical code data, and thought chain data, form the second-stage training sample set.

[0053] It should be noted that the training sample set for the second stage includes training sample sets from the early, middle, and late stages of training.

[0054] In this implementation, the general knowledge data, mathematical code data, and thought chain data are divided according to the division ratio associated with different training cycles. For example, in the early training stage: general knowledge data accounts for 70%, mathematical code data accounts for 20%, and thought chain data accounts for 10%. In the middle training stage: general knowledge data accounts for 30%, mathematical code data accounts for 40%, and thought chain data accounts for 30%. In the later training stage: general knowledge data accounts for 10%, mathematical code data accounts for 20%, and thought chain data accounts for 70%. The general knowledge data, mathematical code data, and thought chain data divided in the early training stage are concatenated to obtain the training sample set for the early training stage. The general knowledge data, mathematical code data, and thought chain data divided in the middle training stage are concatenated to obtain the training sample set for the middle training stage. The general knowledge data, mathematical code data, and thought chain data divided in the later training stage are concatenated to obtain the training sample set for the later training stage, finally forming the second-stage training sample set.

[0055] It's important to note that during training, the proportion of thought chain data gradually increases, while the proportion of general knowledge data and mathematical code data relatively decreases. This simulates the learning path from basic knowledge to logic to complex reasoning, thereby improving the model's understanding and reasoning abilities. For example, in the early stages of training: General knowledge data is the primary focus, with a small amount of mathematical code and thought chain data mixed in, allowing the model to solidify basic language understanding and common sense cognition, establishing underlying semantic capabilities. In the middle stages of training: The proportion of mathematical code data and thought chain data gradually increases. At this point, the model has a foundation and begins to learn logical reasoning and symbolic manipulation in mathematical code data, while also being exposed to simple multi-step reasoning logic, laying the foundation for complex tasks. In the later stages of training: The proportion of thought chain data significantly increases. By this time, the model has acquired basic and intermediate-level abilities, and the focus shifts to strengthening multi-step reasoning and complex logical derivation (e.g., financial statement analysis, mathematical proofs), allowing the model to master deep reasoning skills.

[0056] This embodiment employs the above-described scheme: acquiring several multi-dimensional data sets, including general knowledge data, mathematical code data, and thought chain data; dividing each set of general knowledge data, mathematical code data, and thought chain data according to different division ratios; and forming the second-stage training sample set based on the divided general knowledge data, mathematical code data, and thought chain data. This achieves the goal of training the model by combining general knowledge data, mathematical code data, and thought chain data with different division ratios, allowing the model to gradually master deep reasoning skills and effectively improving model training.

[0057] In one feasible implementation, refer to Figure 5 , Figure 5 This is a flowchart illustrating the model training method of Embodiment 4 of this application. After performing multi-stage pre-training on the initial model based on the training sample set to obtain the pre-trained model, the method further includes: Step S61: Obtain the training dataset for the target domain; Step S62: Based on the training dataset of the target domain, post-train the pre-trained model to obtain the target model.

[0058] It should be noted that the target domain can be set according to the actual situation, taking the financial field as an example for specific explanation. In one embodiment, in order to enable the modified pre-trained model to possess general knowledge and basic reasoning capabilities in the financial field, such as understanding financial terminology, mastering basic financial logic, and answering general financial questions, a training dataset for the target domain is obtained, and the pre-trained model is trained using training methods such as on-policy or on-policy distillation, thereby improving the pre-trained model's general financial capabilities. Optionally, the training dataset includes data of financial terminology and common sense, basic financial reasoning, and financial text understanding, to strengthen the financial capabilities of the modified model.

[0059] In one embodiment, the pre-trained model is equipped with the ability to understand and reason in a single financial scenario, such as credit risk control Q&A, financial statement data extraction, and investment research report generation. The pre-trained model is fine-tuned under supervision using a pre-collected training dataset. For example, credit data is collected, with each data point in the format of customer information and approval results. The customer information and approval results are input into the pre-trained model, and the model parameters are fine-tuned based on the model's output.

[0060] Step S61 includes: Step S621: Based on each piece of training data in the training dataset, use the pre-trained model to predict the second prediction result corresponding to each piece of training data. Step S622: Calculate the model loss value based on the second prediction result corresponding to any of the training data. Step S623: Optimize the model parameters of the pre-trained model based on the model loss value to obtain the target model.

[0061] Specifically, for each piece of training data in the training dataset, the following operations are performed: based on the training data, a second prediction result corresponding to the training data is predicted using the pre-trained model; further, based on the second prediction result corresponding to the training data and the label associated with the training data, the model loss value is calculated. It should be noted that most past modifications used cross-entropy loss for incremental pre-training, which could not obtain the capabilities gained from post-training. In order to restore the post-training capabilities of the original model, this embodiment uses on-policy or on-policy distillation training methods to train the pre-trained model. For example, for the on-policy algorithm, the model loss value can be estimated using Monte Carlo sampling, Actor-Critic, or other methods; further, based on the model loss value, the model parameters of the pre-trained model are fine-tuned until a preset termination condition is reached to obtain the target model. The preset termination condition includes the model reaching convergence or the number of iterations reaching a preset number of iterations.

[0062] This embodiment achieves the following: after obtaining the training dataset for the target domain, the pre-trained model is post-trained based on the training dataset for the target domain to obtain the target model. This strengthens the large model's understanding and reasoning capabilities for the target domain by allowing post-training of the pre-trained model based on the training dataset for the target domain after pre-training.

[0063] It should be noted that the examples in the figure are only for understanding this application and do not constitute a limitation on the model training method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0064] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0065] This application also provides a model training device; please refer to [reference needed]. Figure 6 , Figure 6 This is a schematic diagram of the module structure of the model training device according to an embodiment of this application; the model training device includes: Module 71 is used to acquire the training sample set; The pre-training module 72 is used to perform multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

[0066] The pre-training module 72 is also used for: Each training sample in the first stage training sample set is input into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model. Based on the intermediate features corresponding to each training sample, the model parameters associated with the feature processing module in the initial model are optimized to obtain an intermediate model; Each training sample in the second-stage training sample set is input into the intermediate model to obtain the first prediction result output by the output layer of the intermediate model; Based on each of the first prediction results, the model parameters of the intermediate model are optimized to obtain the pre-trained model.

[0067] The pre-training module 72 is also used for: The feature processing module includes multiple cascaded Transformer modules; for each training sample in the first stage training sample set: The training samples are input into the initial model to obtain the intermediate features output by each Transformer module; For each Transformer module, the feature loss value of the Transformer module is determined based on the intermediate features output by the Transformer module. Based on the feature loss value of the Transformer module, the model parameters associated with the Transformer module are optimized to obtain the intermediate model.

[0068] The pre-training module 72 is also used for: Acquire several multi-dimensional data, including general knowledge data, mathematical code data, and thought chain data; According to different division ratios, the general knowledge data, the mathematical code data, and the thought chain data are divided into different categories. Based on the segmented general knowledge data, mathematical code data, and thought chain data, the second-stage training sample set is formed.

[0069] The model training device is also used for: Obtain the original model and the multi-head potential attention module; The attention layer of the Transformer module in the original model is replaced with the multi-head latent attention module, and the parameters of the multi-head latent attention module are randomly initialized.

[0070] The model training device is also used for: Obtain the training dataset for the target domain; Based on the training dataset of the target domain, the pre-trained model is post-trained to obtain the target model.

[0071] The model training device is also used for: Based on each piece of training data in the training dataset, a second prediction result corresponding to each piece of training data is obtained by using the pre-trained model. The model loss value is calculated based on the second prediction result corresponding to any of the training data. Based on the model loss value, the model parameters of the pre-trained model are optimized to obtain the target model.

[0072] The model training apparatus provided in this application, employing the model training method described in the above embodiments, can solve the technical problems mentioned in the background section. Compared with the prior art, the beneficial effects of the model training apparatus provided in this application are the same as those of the model training method provided in the above embodiments, and other technical features in the model training apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0073] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the model training method in Embodiment 1 above.

[0074] The following is for reference. Figure 7 , Figure 7 This is a schematic diagram of the hardware operating environment involved in the model training method in this application embodiment. The electronic devices in this application embodiment may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0075] like Figure 7 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0076] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0077] The electronic device provided in this application, employing the model training method described in the above embodiments, can solve the technical problems mentioned in the background section. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the model training method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0078] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0080] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the model training method in the above embodiments.

[0081] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0082] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0083] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to: Obtain the training sample set; Based on the training sample set, the initial model is pre-trained in multiple stages to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

[0084] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0086] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0087] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described model training method, and is capable of solving the technical problems described in the background section. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the model training method provided in the above embodiments, and will not be repeated here.

[0088] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the model training method described above.

[0089] The computer program product provided in this application can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the model training method provided in the above embodiments, and will not be repeated here.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0092] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A model training method, characterized in that, include: Obtain the training sample set; Based on the training sample set, the initial model is pre-trained in multiple stages to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

2. The model training method as described in claim 1, characterized in that, The training sample set includes a first-stage training sample set and a second-stage training sample set; The step of performing multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model includes: Each training sample in the first stage training sample set is input into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model. Based on the intermediate features corresponding to each training sample, the model parameters associated with the feature processing module in the initial model are optimized to obtain an intermediate model; Each training sample in the second-stage training sample set is input into the intermediate model to obtain the first prediction result output by the output layer of the intermediate model; Based on each of the first prediction results, the model parameters of the intermediate model are optimized to obtain the pre-trained model.

3. The model training method as described in claim 2, characterized in that, The feature processing module includes multiple cascaded Transformer modules; The step of inputting each training sample from the first-stage training sample set into the initial model to obtain the intermediate features corresponding to each training sample output by the feature processing module in the initial model includes: For each training sample in the first phase training sample set: The training samples are input into the initial model to obtain the intermediate features output by each Transformer module; The step of optimizing the model parameters associated with the feature processing module in the initial model based on the intermediate features corresponding to each training sample to obtain an intermediate model includes: For each Transformer module, the feature loss value of the Transformer module is determined based on the intermediate features output by the Transformer module. Based on the feature loss value of the Transformer module, the model parameters associated with the Transformer module are optimized to obtain the intermediate model.

4. The model training method as described in claim 2, characterized in that, The second-stage training sample set was obtained based on the following steps: Acquire several multi-dimensional data, including general knowledge data, mathematical code data, and thought chain data; According to different division ratios, the general knowledge data, the mathematical code data, and the thought chain data are divided into different categories. Based on the segmented general knowledge data, mathematical code data, and thought chain data, the second-stage training sample set is formed.

5. The model training method as described in claim 1, characterized in that, The initial model was obtained by structural modification according to the following steps: Obtain the original model and the multi-head potential attention module; The attention layer of the Transformer module in the original model is replaced with the multi-head latent attention module, and the parameters of the multi-head latent attention module are randomly initialized.

6. The model training method as described in claim 1, characterized in that, After performing multi-stage pre-training on the initial model based on the training sample set to obtain the pre-trained model, the process further includes: Obtain the training dataset for the target domain; Based on the training dataset of the target domain, the pre-trained model is post-trained to obtain the target model.

7. The model training method as described in claim 6, characterized in that, The step of post-training the pre-trained model based on the training dataset of the target domain to obtain the target model includes: Based on each piece of training data in the training dataset, a second prediction result corresponding to each piece of training data is obtained by using the pre-trained model. The model loss value is calculated based on the second prediction result corresponding to any of the training data. Based on the model loss value, the model parameters of the pre-trained model are optimized to obtain the target model.

8. A model training device, characterized in that, include: The acquisition module is used to acquire the training sample set; The pre-training module is used to perform multi-stage pre-training on the initial model based on the training sample set to obtain a pre-trained model. The initial model is obtained by structurally modifying the original model using a pre-built multi-head latent attention module.

9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the model training method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the model training method as described in any one of claims 1 to 7.

Images