Low-resource multilingual large model training method and system for personalized course learning

By combining an adaptive sampling scheduler and a dynamic loss scheduler, the problems of data imbalance and learning progress matching in low-resource multilingual training are solved, achieving efficient training and cross-language alignment of low-resource language models.

CN121835748BActive Publication Date: 2026-07-10MINZU UNIVERSITY OF CHINA

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
MINZU UNIVERSITY OF CHINA
Filing Date
2025-12-23
Publication Date
2026-07-10

AI Technical Summary

Technical Problem

In low-resource, multilingual scenarios, the data imbalance problem severely limits the performance of large language models (LLMs), and existing methods cannot effectively reduce the dependence on high-quality data or adaptively match the difficulty of multilingual tasks with the model's learning progress.

Method used

An adaptive sampling scheduler and a dynamic loss scheduler are employed to dynamically adjust the sample sampling probability and loss weight, combined with a pre-trained large language model, to achieve personalized course learning, gradually balance sampling and adapt to changes in the difficulty of multiple languages.

Benefits of technology

It effectively reduces the reliance on high-quality data in low-resource languages, dynamically matches the difficulty of multilingual tasks, and improves the model's learning performance in low-resource languages ​​and its cross-language alignment ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121835748B_ABST
    Figure CN121835748B_ABST
Patent Text Reader

Abstract

The application provides a low-resource multilingual large model training method and system for personalized course learning, and belongs to the technical field of large language models, and comprises the following steps: S1: collecting training samples in multiple languages; each sample is subjected to deduplication and noise removal processing, then the data format is unified, and a language attribute is added as a language label; S2: initializing parameters of an adaptive sampling scheduler and a dynamic loss scheduler; S3: using a pre-trained large language model as a base model, and adding the adaptive sampling scheduler and the dynamic loss scheduler in the training process. The method can reduce the dependence on low-resource language labeled data, can adaptively balance the training weights of different language samples, and dynamically matches the difficulty of multilingual tasks and the learning progress of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of large language model technology, and specifically relates to a low-resource multilingual large model training method and system for personalized course learning. Background Technology

[0002] Large Language Models (LLMs) have demonstrated exceptional capabilities in fields such as Natural Language Processing, but data imbalance severely limits their effectiveness in low-resource, multilingual scenarios. Currently, the development of LLMs is mainly focused on "mainstream languages" such as English and Chinese, while services for many low-resource or underrepresented languages ​​are insufficient. This not only hinders the equitable global deployment of LLMs but also highlights critical gaps in model capabilities and evaluation frameworks in multilingual scenarios.

[0003] In training large multilingual models with low resources, enhancing the multilingual capabilities of LLMs is challenging and costly due to the scarcity of high-quality multilingual data. Traditional multilingual training and cross-language alignment methods are often limited by the bottleneck of data scarcity or over-reliance on the performance of mainstream languages.

[0004] The core logic of existing multilingual training techniques is to improve the multilingual capabilities of models by expanding the language coverage and scale of training data. Representative methods include large-scale multilingual pre-training and post-training. For example, the PolyLM model employs large-scale multilingual pre-training, constructing a training corpus containing over 100 languages, covering major global language families. It uses a unified Transformer architecture to jointly pre-train multilingual texts, learning universal semantic representations for different languages. During pre-training, byte-pair encoding (BPE) technology is used to construct a multilingual vocabulary, achieving unified token-level processing for different languages. Simultaneously, language identifier embeddings are introduced to distinguish different language inputs. DeepSeek-V3 proposes a reinforcement learning post-training technique. Based on the pre-trained multilingual model, a task-specific reward model is constructed, and the model parameters are optimized using the policy gradient method of reinforcement learning. First, the reward model is trained using high-resource language task data. Then, the model's generated results are scored based on the reward model, and gradients are fed back, achieving iterative performance optimization of the model on specific tasks.

[0005] The core of cross-language alignment-based training techniques is to establish semantic mapping relationships between different languages ​​(usually based on parallel corpora) to achieve cross-language transfer of semantic representations. Mainstream methods include knowledge distillation. The xCoT framework proposes a cross-language thought chain instruction fine-tuning method. By constructing a multilingual thought chain sample library, it combines the thought chain reasoning process with task instructions to fine-tune the pre-trained model. First, multilingual parallel corpora are collected, and the thought chain reasoning steps (including problem decomposition, intermediate reasoning, and conclusion generation) corresponding to each corpus are manually labeled to construct a multilingual thought chain dataset. Then, the instruction fine-tuning method is used to transfer the thought chain reasoning ability to the model, improving the model's cross-language contextual understanding and reasoning ability. The knowledge distillation method proposes a multilingual extension scheme for monolingual models. It uses a high-performance monolingual pre-trained model (such as English BERT) as the teacher model and a model that has not been trained in many languages ​​as the student model. The monolingual training data is translated into multiple target languages ​​through machine translation tools to construct a multilingual training corpus. Then, the teacher model is used to guide the semantic representation of multilingual texts. The distillation loss function enables the student model to learn the cross-lingual semantic mapping ability of the teacher model, thereby realizing the extension from monolingual model to multilingual model.

[0006] Imbalanced training techniques include resampling and cost-sensitive learning. Resampling adjusts the data distribution by copying low-resource samples or removing high-resource samples. By sampling data from multiple languages ​​according to the language distribution ratio in the model training dataset, the performance of the compressed model on low-resource languages ​​is improved. Cost-sensitive learning highlights low-resource samples by adjusting loss weights. Adaptive gradient scaling falls under the category of cost-sensitive learning. Its core is to assign differentiated loss weight coefficients to different languages, and adjust the training focus on different languages ​​through gradient scaling. This method first initializes the loss weights based on prior information such as the number of language samples and the quality of the corpus. Then, during training, it dynamically adjusts the weight coefficients according to the classification accuracy of each language, increasing the weights for languages ​​with lower accuracy, and accelerating their performance improvement through the gradient amplification effect.

[0007] Course-based learning involves progressively learning by arranging training samples sequentially or setting training objectives based on prior knowledge. The model is trained by gradually using a curriculum consisting of token-level code switching, sentence-level code switching, and monolingual corpora, mimicking the stages of human language learning. A multi-stage pre-training strategy incorporating course-based learning is employed, with each stage progressively increasing the focus on low-resource languages ​​to improve their performance.

[0008] The aforementioned existing technology has the following problems:

[0009] 1. Multilingual Training and Cross-Language Alignment Methods: The scarcity of high-quality data for low-resource languages ​​is the core bottleneck in multilingual training. Firstly, the absolute quantity is small; most low-resource languages ​​have fewer than 100,000 labeled samples, while mainstream languages ​​like English typically have over 100 million labeled samples. The low proportion of low-resource language samples makes it difficult for the model to fully learn the features of low-resource languages. Secondly, data diversity is insufficient; low-resource language samples are mostly concentrated in everyday conversational scenarios, lacking text data from professional fields such as law, medicine, and technology. Furthermore, some samples have quality issues such as grammatical errors and semantic ambiguity, resulting in extremely poor multilingual transfer capabilities for the model in professional scenarios and low accuracy in translating professional terminology. In addition, data acquisition costs are extremely high. For example, when constructing thought chain samples using the xCoT framework, the manual annotation cost for low-resource languages ​​is 8-10 times that for high-resource languages. Moreover, in machine translation data relied upon by knowledge distillation methods, the translation accuracy for low-resource languages ​​is generally below 60%.

[0010] 2. Data Imbalance Handling Methods: Existing resampling methods generally suffer from rigid strategies, adjusting sampling probabilities only based on language distribution ratios and basic convergence, without considering the differences in task difficulty among different languages. For low-resource languages, even if the sampling ratio is increased, the model struggles to quickly learn core features. Simply copying low-resource samples leads to excessive data redundancy, causing overfitting and increased generalization error during model training. On the other hand, removing high-resource samples results in the loss of core semantic features of high-resource languages, leading to performance degradation.

[0011] 3. Course Learning Methods: Existing course learning technologies generally lack dynamic adaptability and cannot match the complex dynamic changes in multilingual scenarios. First, they struggle to adapt to the varying learning difficulties of different languages. For example, code-switching course learning uses a fixed three-stage process: token-sentence-monolingual corpus, failing to consider the different learning rates caused by the grammatical differences between isolating languages ​​(e.g., Chinese) and inflectional languages ​​(e.g., German). The model may converge prematurely at a certain stage but still needs to proceed according to the fixed process. Second, they cannot match the personalized learning progress of the model. For example, the focus on low-resource languages ​​uses only a linearly increasing strategy, failing to differentiate the learning difficulty of different low-resource languages. For endangered languages ​​with extremely small vocabularys, the linearly increasing focus is insufficient to meet learning needs, resulting in some languages ​​reaching the target performance while others remain underfitting, leading to uneven multilingual performance.

[0012] Therefore, there is an urgent need for a method that can: 1) reduce the dependence on low-resource language annotation data; 2) adaptively balance the training weights of samples from different languages; and 3) dynamically match the difficulty of multilingual tasks with the model learning progress. Summary of the Invention

[0013] To address the aforementioned technical problems, this invention provides a low-resource, multilingual, large-model training method for personalized course learning, comprising the following steps:

[0014] Step S1: Collect training samples in multiple languages; perform deduplication and noise removal on each sample, then standardize the data format and add language attributes as language labels;

[0015] Step S2: Initialize the parameters of the adaptive sampling scheduler and the dynamic loss scheduler;

[0016] Step S3: Use the pre-trained large language model as the base model, and add the adaptive sampling scheduler and dynamic loss scheduler during the training process.

[0017] Beneficial effects:

[0018] 1. This invention provides an adaptive sampling scheduler for a low-resource, multilingual, large-scale model training method for personalized course learning. It does not rely on additional high-quality labeled data. By dynamically adjusting the sampling probability of each language sample, it gradually achieves balanced sampling based on the original data distribution, effectively reducing the dependence of low-resource languages ​​on scarce high-quality data.

[0019] 2. The adaptive sampling scheduler avoids the drawback of traditional resampling methods that require manual determination of sample ratios by adaptive adjustment driven by the training process. At the same time, the dynamic loss scheduler does not rely on multilingual prior knowledge, but adjusts the loss weights only by adjusting the model's cross-language alignment progress, thus solving the rigidity problem of imbalanced data processing techniques.

[0020] 3. The adaptive sampling scheduler and dynamic loss scheduler work together to achieve dynamic matching of "sampling strategy - loss weight - model progress", which allows course learning to break free from the single static linear limitation and can specifically focus on the "learning difficulty" of low-resource languages, adapting in real time to changes in the difficulty of multiple languages ​​and the model learning status. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a low-resource, multilingual, large-model training method for personalized course learning according to the present invention.

[0022] Figure 2 This is a structural block diagram of a low-resource, multilingual, large-model training system for personalized course learning according to the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] Example 1:

[0025] like Figure 1 As shown in the figure, the low-resource multilingual large model training method for personalized course learning provided by this embodiment of the invention includes the following steps:

[0026] Step S1: Collect training samples in multiple languages; perform deduplication and noise removal on each sample, then standardize the data format and add language attributes as language labels;

[0027] Step S2: Initialize the parameters of the adaptive sampling scheduler and the dynamic loss scheduler;

[0028] Step S3: Use the pre-trained large language model as the base model, and add an adaptive sampling scheduler and a dynamic loss scheduler during the training process.

[0029] In one embodiment, step S1 above involves collecting training samples in multiple languages; deduplicating and removing noise from each sample; then standardizing the data format and adding language attributes as language labels, specifically including:

[0030] Step S11: For low-resource languages ​​with too few samples, use a large model translation method to translate the English text S0 to the corresponding low-resource language S1, and then translate S1 back to the English text S2. Then compare the word matching degree between S0 and S2. If the matching degree score is lower than the threshold, discard this sample.

[0031] This invention collects multilingual training data, covering 10 languages. Typically, there are more than 10 languages ​​covered, with a focus on including low-resource languages. Low-resource language data is defined as samples representing less than 5% of the total samples. For languages ​​with fewer than 1% of the total samples, a large-scale model translation approach is first used to translate English text S0 into its corresponding language S1, and then S1 is translated back into English text S2. The word matching scores of S0 and S2 are then compared; if the matching score is low, the sample is discarded. The matching score is calculated based on the BLEU algorithm, a method for measuring the quality of model-generated text compared to reference text, using n-gram matching.

[0032] Step S12: Filter and remove duplicate data from the remaining samples based on rules.

[0033] This invention implements a set of heuristic preprocessing techniques for all data, including rule-based filtering and data deduplication:

[0034] (1) Rule-based filtering: including keyword filtering, abnormal character filtering, and advertising filtering.

[0035] (2) Deduplication: The xorbits tool was used, which utilizes the MinHash-LSH algorithm to quickly and efficiently remove duplicate data from the corpus. Subsequently, the data format was unified to apacal format, which includes key attributes such as instruction, input, and output, and a language attribute was added as a language tag in multilingual scenarios.

[0036] In one embodiment, step S2 above, which initializes the parameters of the adaptive sampling scheduler and the dynamic loss scheduler, specifically includes:

[0037] Step S21: Set the number of training rounds for the adaptive sampling scheduler and the dynamic loss scheduler to complete the transition, respectively. and The two remain consistent.

[0038] Step S22: Before each training round begins, the adaptive sampling scheduler initializes the transition factor according to the current round t, where t takes the value of t. , For the total number of training rounds, satisfying .

[0039] When the first round of training began Transition factor At this point, the sampling probability of the adaptive sampling scheduler is:

[0040] ;

[0041] in, For language The total number of samples, The total number of samples across all languages. Lang represents the number of languages. Indicate language The sampling probability of the 0th round of training.

[0042] The dynamic loss scheduler initializes the loss control weights: .

[0043] In one embodiment, step S3 above, using the pre-trained large language model as the base model, incorporates an adaptive sampling scheduler and a dynamic loss scheduler during training, specifically including:

[0044] Step S31: In the t-th round of training, the sampling probability of the adaptive sampling scheduler... for:

[0045] ;

[0046] in, , Indicates the total number of languages, and For uniform distribution, transition factor ;

[0047] The learning of parameters in the model is achieved by minimizing the value of the objective function. The optimization objective function of the dynamic loss scheduler is:

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] in, This refers to the batch sample size. For the sample eigenvectors, For the sample The feature vectors of positive samples (samples from different languages), For the sample The feature vector of negative sample (samples of the same language but different semantics); The cosine similarity function is used. Temperature coefficient; For loss control weights;

[0053] The classification loss function is calculated using the cross-entropy loss function; For the sample Real language tags, The probability distribution of language labels predicted by the model.

[0054] The dynamic loss scheduler balances the dual objectives of representation learning and classification performance by dynamically adjusting the weights of representation loss and classification loss. The adjustment process is guided by the model's progress in cross-linguistic representation alignment. Specifically, the representation loss optimizes the structure of language samples in the representation space, while the classification loss optimizes the accuracy of language label prediction. This invention achieves this through… Controlling the dynamic weights of the two losses during the early stages of training. The model focuses primarily on representation learning; in the later stages of training, as alignment improves... The entire training process shifts towards optimizing task performance. This dynamic transition avoids the rigidity of fixed scheduling in traditional methods.

[0055] Step S32: The parameters in the model are learned using the stochastic gradient descent method. First, some parameters of the base model are frozen, and the remaining parameters are the trainable parameters, denoted as . The adaptive sampling scheduler adjusts the frequency according to the current training round. Calculated sampling probability A specified number of samples are randomly selected from the training samples, and the parameters are updated accordingly. The AdamW optimizer is then used to update the parameters, and it automatically adjusts the learning rate in each iteration.

[0056] ;

[0057] in, For the first Parameters after training round Choosing an appropriate learning rate is crucial for model convergence: an excessively large learning rate... This will cause oscillations in the objective function, while an excessively small value will... This will result in slow convergence; It is the weight decay coefficient, used for regularization; These are estimates of the first-order and second-order matrices of the AdamW optimizer:

[0058] ;

[0059] ;

[0060] in, , which are the attenuation coefficients of the first-order matrix and the second-order matrix, respectively.

[0061] Step S33: Repeat steps S2 to S3 until the maximum number of training rounds is reached.

[0062] For example, the training process is as follows:

[0063] Step 1: Perform data preprocessing according to step S1;

[0064] Step 2: Set the transition rounds for the adaptive sampling scheduler Dynamic loss scheduler transition rounds Total training rounds Learning rate Initialize the adaptive sampling scheduler for batch sample sampling and the dynamic loss scheduler for initial loss control weights;

[0065] Step 3: Train the model to learn the parameters. Repeat steps 2-3 for each training round, for a total of 3 training rounds.

[0066] In summary, the method of the present invention solves the following technical problems:

[0067] First, the adaptive sampling scheduler proposed in this invention does not rely on additional high-quality labeled data. By dynamically adjusting the sampling probability of each language sample, it gradually achieves balanced sampling based on the original data distribution, effectively reducing the dependence of low-resource languages ​​on scarce high-quality data.

[0068] Secondly, the adaptive sampling scheduler avoids the drawback of traditional resampling methods requiring manual determination of sample ratios through adaptive adjustment driven by the training process. At the same time, the dynamic loss scheduler does not rely on multilingual prior knowledge, but adjusts the loss weights only through the model's cross-language alignment progress, thus solving the rigidity problem of imbalanced data processing techniques.

[0069] Finally, the two schedulers work together to achieve dynamic matching of "sampling strategy - loss weight - model progress", which allows course learning to break free from the single static linear limitation and can specifically focus on the "learning difficulty" of low-resource languages, adapting in real time to changes in the difficulty of multiple languages ​​and the model learning status.

[0070] Example 2:

[0071] like Figure 2 As shown, this embodiment of the invention provides a low-resource, multilingual, large-model training system for personalized course learning, comprising the following modules:

[0072] The preprocessing module 41 is used to collect training samples in multiple languages; it performs deduplication and noise removal on each sample, then unifies the data format and adds language attributes as language labels.

[0073] Initialization module 42 is used to initialize the parameters of the adaptive sampling scheduler and the dynamic loss scheduler;

[0074] The model training module 43 is used to take the pre-trained large language model as the base model and add an adaptive sampling scheduler and a dynamic loss scheduler during the training process.

[0075] A low-resource multilingual large model training device for personalized course learning includes one or more electronic devices, wherein the one or more electronic devices are used to implement a low-resource multilingual large model training method for personalized course learning.

[0076] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors enable a low-resource, multilingual, large-model training method for personalized curriculum learning.

[0077] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement a low-resource, multilingual, large-model training method for personalized curriculum learning.

[0078] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, enables a low-resource, multilingual, large-model training method for personalized curriculum learning.

[0079] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A low-resource, multilingual, large-scale model training method for personalized course learning, characterized in that, include: Step S1: Collect training samples in multiple languages; Each sample is deduplicated and noise is removed. Then, the data format is standardized and language attributes are added as language labels. Step S2: Initialize the parameters of the adaptive sampling scheduler and the dynamic loss scheduler, specifically including: Step S21: Set the number of training rounds for the adaptive sampling scheduler and the dynamic loss scheduler to complete the transition, respectively. and The two remain consistent; Step S22: Before each training round begins, the adaptive sampling scheduler initializes the transition factor according to the current round t, where t takes the value of t. , For the total number of training rounds, satisfying ; When the first round of training began Transition factor At this point, the sampling probability of the adaptive sampling scheduler is: ; in, For language The total number of samples, The total number of samples across all languages. Indicate language The sampling probability of the 0th round of training; The dynamic loss scheduler initializes the loss control weights: ; Step S3: Using the pre-trained large language model as the base model, the adaptive sampling scheduler and dynamic loss scheduler are added during training. Specifically, this includes: using the pre-trained large language model as the base model, the adaptive sampling scheduler and dynamic loss scheduler are added during training. Step S31: In the t-th round of training, the sampling probability of the adaptive sampling scheduler... for: ; in, , Indicates the total number of languages, and For uniform distribution, transition factor ; The learning of parameters in the model is achieved by minimizing the value of the objective function. The optimization objective function of the dynamic loss scheduler is: ; ; ; ; in, This refers to the batch sample size. For the sample eigenvectors, For the sample The feature vector of positive samples, For the sample The negative sample feature vector; The cosine similarity function is used. Temperature coefficient; For loss control weights; The classification loss function is calculated using the cross-entropy loss function; For the sample Real language tags, The probability distribution of language labels predicted by the model; Step S32: The parameters in the model are learned using the stochastic gradient descent method. First, some parameters of the base model are frozen, and the remaining parameters are the trainable parameters, denoted as . The adaptive sampling scheduler adjusts the frequency according to the current training round. Calculated sampling probability A specified number of samples are randomly selected from the training samples, and the parameters are updated accordingly. The AdamW optimizer is then used to update the parameters. ; in, For the first Parameters after training round The learning rate; It is the weight decay coefficient, used for regularization; These are estimates of the first-order and second-order matrices of the AdamW optimizer: ; ; in, , which are the attenuation coefficients of the first-order matrix and the second-order matrix, respectively; Step S33: Repeat steps S2 to S3 until the maximum number of training rounds is reached.

2. The low-resource, multilingual, large-scale model training method for personalized course learning according to claim 1, characterized in that, Step S1: Collect training samples in multiple languages; Each sample is deduplicated and noise-removed, then the data format is standardized, and a language attribute is added as a language label, specifically including: Step S11: For low-resource languages ​​with too few samples, use a large model translation method to translate the English text S0 to the corresponding low-resource language S1, and then translate S1 back to the English text S2. Then compare the word matching degree between S0 and S2. If the matching degree score is lower than the threshold, discard this sample. Step S12: Filter and remove duplicate data from the remaining samples based on rules.

3. A low-resource, multilingual, large-scale model training system for personalized course learning, characterized in that: Includes the following modules: The preprocessing module is used to collect training samples in multiple languages; each sample is deduplicated and noise is removed, then the data format is standardized and language attributes are added as language labels; The initialization module is used to initialize the parameters of the adaptive sampling scheduler and the dynamic loss scheduler, specifically including: Step S21: Set the number of training rounds for the adaptive sampling scheduler and the dynamic loss scheduler to complete the transition, respectively. and The two remain consistent; Step S22: Before each training round begins, the adaptive sampling scheduler initializes the transition factor according to the current round t, where t takes the value of t. , For the total number of training rounds, satisfying ; When the first round of training began Transition factor At this point, the sampling probability of the adaptive sampling scheduler is: ; in, For language The total number of samples, The total number of samples across all languages. Indicate language The sampling probability of the 0th round of training; The dynamic loss scheduler initializes the loss control weights: ; The model training module is used to take the pre-trained large language model as the base model and add the adaptive sampling scheduler and dynamic loss scheduler during the training process. Specifically, it includes: taking the pre-trained large language model as the base model and adding the adaptive sampling scheduler and dynamic loss scheduler during the training process. Step S31: In the t-th round of training, the sampling probability of the adaptive sampling scheduler... for: ; in, , Indicates the total number of languages, and For uniform distribution, transition factor ; The learning of parameters in the model is achieved by minimizing the value of the objective function. The optimization objective function of the dynamic loss scheduler is: ; ; ; ; in, This refers to the batch sample size. For the sample eigenvectors, For the sample The feature vector of positive samples, For the sample The negative sample feature vector; The cosine similarity function is used. Temperature coefficient; For loss control weights; The classification loss function is calculated using the cross-entropy loss function; For the sample Real language tags, The probability distribution of language labels predicted by the model; Step S32: The parameters in the model are learned using the stochastic gradient descent method. First, some parameters of the base model are frozen, and the remaining parameters are the trainable parameters, denoted as . The adaptive sampling scheduler adjusts the frequency according to the current training round. Calculated sampling probability A specified number of samples are randomly selected from the training samples, and the parameters are updated accordingly. The AdamW optimizer is then used to update the parameters. ; in, For the first Parameters after training round The learning rate; It is the weight decay coefficient, used for regularization; These are estimates of the first-order and second-order matrices of the AdamW optimizer: ; ; in, , which are the attenuation coefficients of the first-order matrix and the second-order matrix, respectively; Step S33: Repeat steps S2 to S3 until the maximum number of training rounds is reached.

4. A low-resource, multilingual, large-scale model training device for personalized course learning, characterized in that: It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 2.

5. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to implement the method described in any one of claims 1 to 2.

7. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 2.