Staged training making method and system based on artificial intelligence

Abstract

The invention provides a staged training making method and system based on artificial intelligence. The method comprises the steps of obtaining a training target and data set characteristics; according to the training target and the data set characteristics, a training plan draft is generated, and the training plan draft comprises stage division, a learning rate strategy of each stage, a data enhancement strategy and a model evaluation standard; based on the training plan draft, initializing one or more neural network models, and allocating corresponding training resources for each stage; before each stage starts, training parameters of the current stage are adjusted according to the training result of the previous stage, and the adjustment at least comprises learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment; at the end of each stage, the performance of the model is evaluated, and whether to enter the next stage or repeat the training of the current stage is determined according to the evaluation result, so that the efficiency and quality of model training can be effectively improved, and effective utilization of resources is ensured. Due to the flexibility and the self-adaptability of the method, the method can well adapt to various complex training tasks and data set characteristics.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for formulating phased training based on artificial intelligence. Background Art

[0002] In the existing AI model training process, most methods rely on a single learning rate strategy and a fixed training plan, which often shows limitations when faced with complex data sets and diverse application requirements. Especially for tasks that require multi-stage training, such as pre-training-fine-tuning, feature extraction-end-to-end training, traditional training methods lack sufficient flexibility to adapt to changes in requirements at different stages, which may lead to poor model performance or inefficient training. Summary of the invention

[0003] The embodiments of the present invention provide a method and system for formulating phased training based on artificial intelligence, which can at least solve the technical problem that traditional training methods in the prior art lack sufficient flexibility to adapt to changes in requirements at different stages.

[0004] A first aspect of an embodiment of the present invention provides a method for formulating phased training based on artificial intelligence, comprising:

[0005] Get training objectives and dataset characteristics;

[0006] Generate a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes phase division, learning rate strategy for each phase, data enhancement strategy, and model evaluation criteria;

[0007] Initialize one or more neural network models based on the draft training plan, and allocate corresponding training resources to each stage;

[0008] Before each stage starts, the training parameters of the current stage are adjusted according to the training results of the previous stage, and the adjustment includes at least learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment;

[0009] At the end of each stage, the model performance is evaluated and based on the evaluation results, a decision is made whether to proceed to the next stage or repeat the current stage of training.

[0010] Furthermore, the obtaining of training objectives and data set characteristics includes:

[0011] Determine the training objectives, which at least include the expected use of the model, performance indicators, training time limits, etc.;

[0012] Analyze the characteristics of the data set, which at least include the size of the data set, data distribution, degree of class imbalance, noise level, etc.

[0013] Furthermore, generating a draft training plan according to the training objectives and data set characteristics includes:

[0014] Determine a phase division strategy according to the training objectives and data set characteristics, wherein the phase division strategy includes at least a model pre-training phase, a feature extraction phase, an end-to-end training phase, etc.;

[0015] For each stage, set the corresponding learning rate strategy, data enhancement strategy, model optimization strategy, etc.

[0016] Furthermore, before each stage starts, adjusting the training parameters of the current stage according to the training results of the previous stage includes:

[0017] According to the performance evaluation results of the model in the previous stage, the learning rate strategy of the current stage is adjusted, and the learning rate strategy adjustment at least includes learning rate decay, learning rate rebound, etc.;

[0018] According to the performance evaluation results of the model in the previous stage, the regularization parameters of the current stage are adjusted, and the regularization parameter adjustment includes at least the weight attenuation coefficient and the Dropout ratio;

[0019] According to the performance evaluation results of the model in the previous stage, the data enhancement strategy of the current stage is adjusted, and the data enhancement strategy adjustment includes at least rotation angle, scaling ratio, and shear ratio.

[0020] Furthermore, at the end of each stage, evaluating the model performance includes:

[0021] Evaluate the model using the validation set, where the evaluation includes at least accuracy, recall, and F1 score;

[0022] Whether the criteria for entering the next stage are met is determined based on the evaluation results, and the criteria at least include a performance indicator threshold and a convergence speed.

[0023] Furthermore, it also includes:

[0024] During the entire training process, model snapshots are saved regularly to facilitate resuming training or model fusion;

[0025] After the training is completed, the final model is comprehensively evaluated and necessary hyperparameters are fine-tuned based on the evaluation results.

[0026] Furthermore, generating a draft training plan according to the training objectives and data set characteristics also includes:

[0027] A dynamic learning rate adjustment strategy is used, which calculates the learning rate using the following formula:

[0028]

[0029] Among them, η(t) represents the learning rate at time t; η max is the maximum learning rate; L v (t) is the loss function value on the validation set at the tth training iteration; L threshold is the loss function threshold; α and β control the degree and speed of the loss function's influence on the learning rate; γ and δ control the degree and speed of the training progress's influence on the learning rate; epoch is the current training round; E is the total training round.

[0030] In a second aspect, an embodiment of the present invention further provides a staged training formulation device based on artificial intelligence, comprising:

[0031] The acquisition module is used to obtain the training target and data set characteristics;

[0032] A generation module, used to generate a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes stage division, learning rate strategy for each stage, data enhancement strategy, and model evaluation criteria;

[0033] An initialization module, used to initialize one or more neural network models based on the draft training plan and allocate corresponding training resources to each stage;

[0034] In the learning module, before each stage starts, the training parameters of the current stage are adjusted according to the training results of the previous stage, and the adjustment includes at least learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment;

[0035] The evaluation module is used to evaluate the model performance at the end of each stage and decide whether to enter the next stage or repeat the training of the current stage based on the evaluation results.

[0036] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the artificial intelligence-based phased training formulations described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, wherein a computer program is stored in the readable storage medium, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes any of the artificial intelligence-based phased training formulations described in the first aspect.

[0038] The embodiment of the present invention obtains the training objectives and data set characteristics; generates a draft training plan according to the training objectives and data set characteristics, the draft training plan includes stage division, learning rate strategy of each stage, data enhancement strategy and model evaluation criteria; initializes one or more neural network models based on the draft training plan, and allocates corresponding training resources for each stage; before each stage starts, adjusts the training parameters of the current stage according to the training results of the previous stage, and the adjustment includes at least learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment; at the end of each stage, evaluates the model performance, and decides whether to enter the next stage or repeat the training of the current stage according to the evaluation results, which can effectively improve the efficiency and quality of model training, while ensuring the effective use of resources. The flexibility and adaptability of this method enable it to adapt well to various complex training tasks and data set characteristics.

[0039] The beneficial effects of the embodiments of the present invention can refer to the technical effects corresponding to the technical features in the specific implementation methods, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a phased training formulation process based on artificial intelligence provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a system module for formulating phased training based on artificial intelligence provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0044] Existing training methods have obvious defects in the following aspects:

[0045] Static learning rate strategy: Most training methods use a fixed learning rate or a simple decay strategy, which may be effective in the early stages of training, but can easily lead to slower model convergence or fall into a local optimal solution in the later stages of training. Especially in multi-stage training, the goals and requirements of each stage are different, and a single learning rate strategy cannot fully adapt to these changes.

[0046] Lack of adaptability: Existing methods generally lack the ability to dynamically adjust training parameters based on model performance, such as adjusting learning rates, regularization parameters, or data augmentation strategies, etc. This means that during training, even if the model performs poorly, it cannot be automatically optimized, which limits the generalization ability and final performance of the model.

[0047] Improper allocation of training resources: In multi-stage training, how to reasonably allocate computing resources is a challenge. Existing training methods often adopt the same resource allocation strategy for all stages, which may lead to excess resources in some stages and insufficient resources in other stages, thus affecting training efficiency and model performance.

[0048] Imperfect evaluation and decision-making mechanisms: During the training process, the evaluation of model performance is usually based on fixed standards, and there is a lack of a mechanism to dynamically adjust the evaluation criteria according to the model status and training objectives. In addition, existing methods are often too simple in transition decisions between stages and lack effective judgment criteria and strategies, which may lead to unnecessary repetition or premature termination of the training process.

[0049] Lack of comprehensive consideration of the training process: Existing training methods often only focus on model training itself, while ignoring other important factors in the training process, such as model fusion, hyperparameter fine-tuning, etc. These factors are equally important for improving model performance and stability, but have not been given enough attention in existing technologies.

[0050] In order to solve the above problems, it is necessary to develop a more flexible and adaptive AI-based phased training formulation method to improve the efficiency and quality of model training. Such a method should be able to dynamically adjust training parameters according to the needs of different stages, while taking into account the effective allocation of training resources and the comprehensive evaluation of model performance, so as to better adapt to the requirements of complex tasks.

[0051] First, the embodiment of the present invention provides a phased training formulation method based on artificial intelligence, such as Figure 1 As shown, including:

[0052] Step S1, obtaining training objectives and data set characteristics;

[0053] In this step, assume that we are working on a computer vision task where our goal is to train a deep convolutional neural network (CNN) for object recognition on a large image dataset. The dataset contains 1 million images covering 1,000 different object categories. Our training goal is to maximize the recognition accuracy of the model while minimizing the training time and computing resource consumption.

[0054] Step S2, generating a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes stage division, learning rate strategy for each stage, data enhancement strategy, and model evaluation criteria;

[0055] In this step, based on the above training objectives and dataset characteristics, we developed a draft training plan, which includes the following:

[0056] Stage division: The entire training process is divided into three stages: pre-training stage, feature extraction stage and end-to-end fine-tuning stage.

[0057] Learning rate strategy for each stage:

[0058] Pre-training stage: A step decay strategy is used with an initial learning rate of 0.001, which decays every 10 epochs with a decay factor of 0.1 each time.

[0059] Feature extraction stage: the learning rate is fixed to 0.0001.

[0060] End-to-end fine-tuning stage: an exponential decay strategy is used, with an initial learning rate of 0.0001, which decays every 1000 steps and a decay factor of 0.95 each time.

[0061] Data augmentation strategy:

[0062] Pre-training stage: random cropping, flipping and color jittering are used.

[0063] Feature extraction stage: center cropping and random scaling are used.

[0064] End-to-end fine-tuning stage: random crops and random rotations are adopted.

[0065] Model evaluation criteria: The average accuracy on the cross-validation set is used as the evaluation indicator.

[0066] Step S3, based on the draft training plan, initializing one or more neural network models, and allocating corresponding training resources to each stage;

[0067] In this step, we initialize a ResNet-50 model according to the draft training plan. In the pre-training phase, 8 GPUs in the GPU cluster are used for distributed training; in the feature extraction phase, 4 GPUs are used; and in the end-to-end fine-tuning phase, 2 GPUs are used.

[0068] Step S4, before each stage starts, adjusting the training parameters of the current stage according to the training results of the previous stage, wherein the adjustment at least includes learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment;

[0069] In this step, after the pre-training phase, we adjust the learning rate based on the model’s performance on the validation set. If the accuracy on the validation set slows down, we reduce the learning rate by an order of magnitude.

[0070] When entering the feature extraction stage, we observed that the model performed poorly on some categories, so we increased the complexity of data augmentation by introducing additional random rotations and brightness changes.

[0071] During the end-to-end fine-tuning stage, we adjusted the weight decay (regularization parameter) based on the model’s performance to prevent overfitting.

[0072] Step S5: At the end of each stage, the model performance is evaluated, and based on the evaluation results, a decision is made as to whether to proceed to the next stage or repeat the training of the current stage.

[0073] In this step, at the end of each epoch, we evaluate the performance of the model using the validation set.

[0074] If the accuracy of the model on the validation set reaches a predetermined threshold, it enters the next stage; otherwise, we will repeat the training of the current stage until the predetermined accuracy is reached or the maximum number of training times is reached.

[0075] Through the above method, the efficiency and quality of model training can be effectively improved while ensuring the effective use of resources. The flexibility and adaptability of this method make it well adapted to various complex training tasks and data set characteristics.

[0076] This embodiment provides a phased training formulation method based on artificial intelligence, which effectively improves the efficiency and quality of model training by dynamically adjusting training parameters, reasonably allocating training resources, and comprehensively evaluating model performance. This method is particularly suitable for complex tasks that require multi-stage training, such as pre-training-fine-tuning, feature extraction-end-to-end training, etc., and can significantly improve the performance of the model.

[0077] In another embodiment of the present invention, obtaining the training target and the data set characteristics includes:

[0078] Determine the training objectives, which at least include the expected use of the model, performance indicators, training time limits, etc.;

[0079] Analyze the characteristics of the data set, which at least include the size of the data set, data distribution, degree of class imbalance, noise level, etc.

[0080] In addition, generating a draft training plan according to the training objectives and data set characteristics includes:

[0081] Determine a phase division strategy according to the training objectives and data set characteristics, wherein the phase division strategy includes at least a model pre-training phase, a feature extraction phase, an end-to-end training phase, etc.;

[0082] For each stage, set the corresponding learning rate strategy, data enhancement strategy, model optimization strategy, etc.

[0083] Based on this, the present invention provides two specific embodiments as follows:

[0084] 1. Obtain training objectives and dataset characteristics

[0085] Determine your training goals:

[0086] Our training goal is to develop a deep learning model for image classification, specifically a convolutional neural network (CNN) that can recognize images in the CIFAR-10 dataset. The model needs to meet the following requirements:

[0087] Intended Use: The model will be deployed in a mobile application to recognize 10 categories in images in real-time.

[0088] Performance metric: The model must have an accuracy of at least 85% on the test set.

[0089] Training time limit: The entire training process should be completed within one week.

[0090] Analyze the characteristics of the dataset:

[0091] Dataset size: The CIFAR-10 dataset contains 50,000 training images and 10,000 test images.

[0092] Data Distribution: The 10 categories in the dataset are evenly distributed, with 5,000 training images and 1,000 testing images for each category.

[0093] Class Imbalance: Since the number of images in each class is the same, there is no class imbalance problem.

[0094] Noise level: There is a certain degree of noise in the dataset, including image blur, lighting changes, etc.

[0095] 2. Generate a draft training plan:

[0096] Phase division strategy

[0097] According to the training objectives and dataset characteristics, we plan to divide the entire training process into three stages:

[0098] Model pre-training stage: The model is pre-trained using the ImageNet dataset to learn common image features.

[0099] Feature extraction stage: Freeze the convolutional layers of the pre-trained model and only train the fully connected layers to adapt to the CIFAR-10 dataset.

[0100] End-to-end training phase: fine-tuning the entire model to further improve performance.

[0101] Setting training strategy

[0102] Model pre-training stage:

[0103] Learning rate strategy: Use exponential decay strategy, the initial learning rate is 0.001, decays every 5 epochs, and each decay factor is 0.9.

[0104] Data augmentation strategy: Use random crops, random horizontal flips, and random color jittering.

[0105] Model optimization strategy: Use Adam optimizer with a batch size of 64.

[0106] Feature extraction stage:

[0107] Learning rate strategy: The fixed learning rate is 0.0001.

[0108] Data augmentation strategy: Use center cropping and random rotation.

[0109] Model optimization strategy: Continue to use the Adam optimizer with a batch size of 32.

[0110] End-to-end training phase:

[0111] Learning rate strategy: Use a step decay strategy with an initial learning rate of 0.0001, which decays every 10 epochs with a decay factor of 0.1 each time.

[0112] Data augmentation strategy: Use random cropping and random rotation.

[0113] Model optimization strategy: Use SGD optimizer with momentum of 0.9 and batch size of 32.

[0114] Implementation steps

[0115] Initialize the model: Use the pre-trained ResNet-50 model as the base model.

[0116] Allocate training resources: use 4 GPUs in the pre-training stage, 2 GPUs in the feature extraction stage, and 2 GPUs in the end-to-end training stage.

[0117] Adjust training parameters: Before each stage starts, adjust the training parameters of the current stage based on the results of the previous stage.

[0118] Evaluate model performance: At the end of each stage, use the validation set to evaluate model performance and decide whether to enter the next stage or repeat the current stage of training based on the evaluation results.

[0119] Through the above specific embodiments, it is shown in detail how to formulate a reasonable training plan draft according to the training objectives and data set characteristics, including the stage division strategy and the training strategy of each stage. This method can effectively improve the efficiency and performance of model training, while ensuring that the model can achieve the expected effect in practical applications.

[0120] In the embodiment of the present invention, before each stage starts, adjusting the training parameters of the current stage according to the training results of the previous stage includes:

[0121] According to the performance evaluation results of the model in the previous stage, the learning rate strategy of the current stage is adjusted, and the learning rate strategy adjustment at least includes learning rate decay, learning rate rebound, etc.;

[0122] According to the performance evaluation results of the model in the previous stage, the regularization parameters of the current stage are adjusted, and the regularization parameter adjustment includes at least the weight attenuation coefficient and the Dropout ratio;

[0123] According to the performance evaluation results of the model in the previous stage, the data enhancement strategy of the current stage is adjusted, and the data enhancement strategy adjustment includes at least rotation angle, scaling ratio, and shear ratio.

[0124] In the embodiment of the present invention, 1. Obtaining training objectives and data set characteristics

[0125] Determine your training goals

[0126] Our training goal is to develop a deep learning model for image classification, specifically a convolutional neural network (CNN) that can recognize images in the CIFAR-10 dataset. The model needs to meet the following requirements:

[0127] Intended Use: The model will be deployed in a mobile application to recognize 10 categories in images in real-time.

[0128] Performance metric: The model must have an accuracy of at least 85% on the test set.

[0129] Training time limit: The entire training process should be completed within one week.

[0130] Analyze the characteristics of the dataset

[0131] Dataset size: The CIFAR-10 dataset contains 50,000 training images and 10,000 test images.

[0132] Data Distribution: The 10 categories in the dataset are evenly distributed, with 5,000 training images and 1,000 testing images for each category.

[0133] Class Imbalance: Since the number of images in each class is the same, there is no class imbalance problem.

[0134] Noise level: There is a certain degree of noise in the dataset, including image blur, lighting changes, etc.

[0135] 2. Generate a draft training plan

[0136] Phase division strategy

[0137] According to the training objectives and dataset characteristics, we plan to divide the entire training process into three stages:

[0138] Model pre-training stage: The model is pre-trained using the ImageNet dataset to learn common image features.

[0139] Feature extraction stage: Freeze the convolutional layers of the pre-trained model and only train the fully connected layers to adapt to the CIFAR-10 dataset.

[0140] End-to-end training phase: fine-tuning the entire model to further improve performance.

[0141] Setting training strategy

[0142] Model pre-training stage:

[0143] Learning rate strategy: Use exponential decay strategy, the initial learning rate is 0.001, decays every 5 epochs, and each decay factor is 0.9.

[0144] Data augmentation strategy: Use random crops, random horizontal flips, and random color jittering.

[0145] Model optimization strategy: Use Adam optimizer with a batch size of 64.

[0146] Feature extraction stage:

[0147] Learning rate strategy: The fixed learning rate is 0.0001.

[0148] Data augmentation strategy: Use center cropping and random rotation.

[0149] Model optimization strategy: Continue to use the Adam optimizer with a batch size of 32.

[0150] End-to-end training phase:

[0151] Learning rate strategy: Use a step decay strategy with an initial learning rate of 0.0001, which decays every 10 epochs with a decay factor of 0.1 each time.

[0152] Data augmentation strategy: Use random cropping and random rotation.

[0153] Model optimization strategy: Use SGD optimizer with momentum of 0.9 and batch size of 32.

[0154] 3. Adjust training parameters

[0155] Before the start of each stage, we need to adjust the training parameters of the current stage based on the training results of the previous stage.

[0156] From model pre-training stage to feature extraction stage

[0157] Learning rate strategy adjustment: After the model pre-training phase, we evaluate the model performance and increase the initial learning rate to 0.0005 during the feature extraction phase if the model converges slowly.

[0158] Regularization parameter adjustment: If the model shows signs of overfitting during the pre-training phase, we increase the weight decay coefficient (e.g., from 0.0001 to 0.0005) during the feature extraction phase.

[0159] Data augmentation strategy adjustment: If the model is less sensitive to certain types of image changes, we can increase the angle range of random rotation (such as from ±15° to ±30°).

[0160] From feature extraction to end-to-end training

[0161] Learning rate strategy adjustment: If the performance of the model in the feature extraction stage is close to saturation, a learning rate rebound strategy can be adopted in the end-to-end training stage (i.e., alternating between higher and lower learning rates within a certain period).

[0162] Regularization parameter adjustment: If the generalization ability of the model in the feature extraction stage needs to be improved, the Dropout ratio can be increased in the end-to-end training stage (for example, from 0.2 to 0.5).

[0163] Data augmentation strategy adjustment: If the model is less sensitive to changes in image size during the feature extraction phase, the scaling ratio can be increased (e.g. from 0.8-1.2 to 0.7-1.3).

[0164] Through the above specific embodiments, we have shown in detail how to formulate a reasonable training plan draft based on the training objectives and data set characteristics, and adjust the training parameters of the current stage according to the training results of the previous stage. This method can effectively improve the efficiency and performance of model training, while ensuring that the model can achieve the expected results in practical applications.

[0165] In another embodiment provided by the present invention, at the end of each stage, evaluating the model performance includes:

[0166] Evaluate the model using the validation set, where the evaluation includes at least accuracy, recall, and F1 score;

[0167] Whether the criteria for entering the next stage are met is determined based on the evaluation results, and the criteria at least include a performance indicator threshold and a convergence speed.

[0168] Also includes:

[0169] During the entire training process, model snapshots are saved regularly to facilitate resuming training or model fusion;

[0170] After training, the final model is comprehensively evaluated and necessary hyperparameters are fine-tuned based on the evaluation results.

[0171] 1. Obtain training objectives and dataset characteristics

[0172] Determine your training goals

[0173] Our training goal is to develop a deep learning model for image classification, specifically a convolutional neural network (CNN) that can recognize images in the CIFAR-10 dataset. The model needs to meet the following requirements:

[0174] Intended Use: The model will be deployed in a mobile application to recognize 10 categories in images in real-time.

[0175] Performance metric: The model must have an accuracy of at least 85% on the test set.

[0176] Training time limit: The entire training process should be completed within one week.

[0177] Analyze the characteristics of the dataset

[0178] Dataset size: The CIFAR-10 dataset contains 50,000 training images and 10,000 test images.

[0179] Data Distribution: The 10 categories in the dataset are evenly distributed, with 5,000 training images and 1,000 testing images for each category.

[0180] Class Imbalance: Since the number of images in each class is the same, there is no class imbalance problem.

[0181] Noise level: There is a certain degree of noise in the dataset, including image blur, lighting changes, etc.

[0182] 2. Generate a draft training plan

[0183] Phase division strategy

[0184] According to the training objectives and dataset characteristics, we plan to divide the entire training process into three stages:

[0185] Model pre-training stage: The model is pre-trained using the ImageNet dataset to learn common image features.

[0186] Feature extraction stage: Freeze the convolutional layers of the pre-trained model and only train the fully connected layers to adapt to the CIFAR-10 dataset.

[0187] End-to-end training phase: fine-tuning the entire model to further improve performance.

[0188] Setting training strategy

[0189] Model pre-training stage:

[0190] Learning rate strategy: Use exponential decay strategy, the initial learning rate is 0.001, decays every 5 epochs, and each decay factor is 0.9.

[0191] Data augmentation strategy: Use random crops, random horizontal flips, and random color jittering.

[0192] Model optimization strategy: Use Adam optimizer with a batch size of 64.

[0193] Feature extraction stage:

[0194] Learning rate strategy: The fixed learning rate is 0.0001.

[0195] Data augmentation strategy: Use center cropping and random rotation.

[0196] Model optimization strategy: Continue to use the Adam optimizer with a batch size of 32.

[0197] End-to-end training phase:

[0198] Learning rate strategy: Use a step decay strategy with an initial learning rate of 0.0001, which decays every 10 epochs with a decay factor of 0.1 each time.

[0199] Data augmentation strategy: Use random cropping and random rotation.

[0200] Model optimization strategy: Use SGD optimizer with momentum of 0.9 and batch size of 32.

[0201] 3. Evaluate model performance at the end of each stage

[0202] At the end of each stage, we need to use the validation set to evaluate the model performance and determine whether it meets the criteria for entering the next stage based on the evaluation results.

[0203] Model Evaluation Methods

[0204] Accuracy: The proportion of samples that the model correctly predicts.

[0205] Recall rate: For a certain type of sample, the proportion of samples that the model correctly predicts to be of that class.

[0206] F1 score: The harmonic mean of precision and recall, used to measure the overall performance of the model.

[0207] Criteria for entering the next stage

[0208] Performance indicator threshold: For example, if the accuracy of the feature extraction stage reaches 80%, you can enter the end-to-end training stage.

[0209] Convergence speed: If the loss function of the model does not change much within several consecutive epochs, the model is considered to have converged and can be considered to enter the next stage.

[0210] 4. Save model snapshots regularly

[0211] Throughout the training process, we will periodically save model snapshots to facilitate resuming training or performing model fusion.

[0212] Save frequency: save once after each training phase or after a certain number of epochs (such as 5 epochs).

[0213] What is saved: model weights, training status (such as optimizer status, learning rate, etc.), and training history.

[0214] 5. Comprehensive evaluation and fine-tuning after training

[0215] After training, we will conduct a comprehensive evaluation of the final model and make necessary hyperparameter fine-tuning based on the evaluation results.

[0216] Comprehensive evaluation: Use the test set to comprehensively evaluate the model to ensure that the model performance meets expectations.

[0217] Hyperparameter fine-tuning: If the model performance does not meet expectations, you can further optimize the hyperparameters through grid search or random search.

[0218] Through the above specific examples, we have shown in detail how to formulate a reasonable draft training plan based on the training objectives and data set characteristics, and evaluate the model performance at the end of each stage to decide whether to enter the next stage. In addition, we also introduced the importance of regularly saving model snapshots and performing comprehensive evaluations after training. This method can effectively improve the efficiency and performance of model training and ensure that the model can achieve the expected results in practical applications.

[0219] In the embodiment of the present invention, generating a draft training plan according to the training objectives and the characteristics of the data set further includes:

[0220] A dynamic learning rate adjustment strategy is used, which calculates the learning rate using the following formula:

[0221]

[0222] Among them, η(t) represents the learning rate at time t; η max is the maximum learning rate; L v (t) is the loss function value on the validation set at the tth training iteration; L threshold is the loss function threshold; α and β control the degree and speed of the loss function's influence on the learning rate; γ and δ control the degree and speed of the training progress's influence on the learning rate; epoch is the current training round; E is the total training round.

[0223] The dynamic learning rate adjustment strategy mentioned in the embodiment of the present invention is an adaptive learning rate strategy, which aims to dynamically adjust the learning rate according to the performance of the model during the training process. This method can help the model converge better, avoiding the problem of model oscillation caused by too high learning rate in the early stage of training or slow convergence caused by too low learning rate in the later stage of training. The design reasons of the dynamic learning rate adjustment strategy and the functions of its components are described in detail below.

[0224] Design rationale for dynamic learning rate adjustment strategy

[0225] 1. Strong adaptability: Dynamically adjusting the learning rate can change the learning step size according to the actual situation of the model during training, which helps the model find the optimal solution or a solution space close to the optimal solution more quickly.

[0226] 2. Prevent overfitting: By observing the loss function value on the validation set, when the loss function value no longer decreases significantly, appropriately reducing the learning rate can prevent the model from overfitting the training data.

[0227] 3. Accelerated convergence: As the number of training rounds increases, the learning rate gradually decreases, which helps the model adjust the weights more finely and accelerates the convergence process.

[0228] Components and functions of dynamic learning rate adjustment strategy

[0229] η max Maximum learning rate: This is the maximum value of the learning rate. The model will use a learning rate close to this value at the beginning of training.

[0230] L v (t) Loss function value on the validation set: As training progresses, the loss function value on the validation set reflects the performance of the model on unseen data. If the loss value is high, it means that the model has not generalized well, so a higher learning rate is needed to accelerate learning. Conversely, if the loss value is low, it means that the model has learned well, and the learning rate should be reduced to avoid overfitting.

[0231] L threshold Loss function threshold: This is a set threshold. When the loss function value on the validation set is lower than this threshold, it means that the model's performance is good enough, and the learning rate should be reduced to prevent overfitting.

[0232] α and β control the degree and speed of the loss function's influence on the learning rate: α controls how much the learning rate decreases, and β controls how sensitive the learning rate is to changes in validation loss. When the validation loss approaches the threshold, the learning rate gradually decreases.

[0233] γ and δ control the degree and speed of the impact of training progress on the learning rate: as the number of training rounds increases, γ controls the degree to which the learning rate decreases, while δ controls the sensitivity of the learning rate to changes in training progress. Generally speaking, the learning rate is high at the beginning of training, and gradually decreases as training progresses.

[0234] epoch current training round and E total training round: These two parameters are used to control the adjustment rhythm of the learning rate during the entire training process, ensuring that the learning rate is higher in the early stages of training and gradually decreases in the later stages.

[0235] Suppose we are training a deep learning model with the goal of achieving good classification performance on the CIFAR-10 dataset. We set the following parameters:

[0236] Maximum learning rate η max =0.01

[0237] Loss function threshold Lthreshold =0.2

[0238] α=0.1,β=5

[0239] γ=0.1,δ=5

[0240] Total training rounds E = 100

[0241] During the training process, suppose that at the 50th iteration, the loss function value L on the validation set v (t) = 0.25, the learning rate at this time will be calculated according to the above formula. As the training progresses, the loss function value on the validation set gradually decreases. When it approaches L threshold , the learning rate will gradually decrease according to the logic in the formula to help the model converge better.

[0242] In another embodiment of the present invention, a staged training formulation device based on artificial intelligence is provided, such as Figure 2 As shown, including:

[0243] Acquisition module 01, used to obtain training targets and data set characteristics;

[0244] A generation module 02 is used to generate a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes phase division, learning rate strategy for each phase, data enhancement strategy, and model evaluation criteria;

[0245] Initialization module 03, used to initialize one or more neural network models based on the draft training plan, and allocate corresponding training resources to each stage;

[0246] In the learning module 04, before each stage starts, the training parameters of the current stage are adjusted according to the training results of the previous stage, and the adjustment at least includes the adjustment of the learning rate, the adjustment of the regularization parameters or the adjustment of the data enhancement strategy;

[0247] Evaluation module 05 is used to evaluate the model performance at the end of each stage and decide whether to enter the next stage or repeat the training of the current stage based on the evaluation results.

[0248] An embodiment of the present invention further provides an electronic device, including:

[0249] processor;

[0250] a memory for storing processor-executable instructions;

[0251] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0252] A fourth aspect of the embodiments of the present invention is:

[0253] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0254] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0255] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A phased training formulation method based on artificial intelligence, characterized in that: include: Get training objectives and dataset characteristics; Generate a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes phase division, learning rate strategy for each phase, data enhancement strategy, and model evaluation criteria; Initialize one or more neural network models based on the draft training plan, and allocate corresponding training resources to each stage; Before each stage starts, the training parameters of the current stage are adjusted according to the training results of the previous stage, and the adjustment includes at least learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment; At the end of each stage, the model performance is evaluated and based on the evaluation results, a decision is made whether to proceed to the next stage or repeat the current stage of training.

2. The method according to claim 1, characterized in that The acquisition of training objectives and data set characteristics includes: Determine the training objectives, which at least include the expected use of the model, performance indicators, training time limits, etc.; Analyze the characteristics of the data set, which at least include the size of the data set, data distribution, degree of class imbalance, noise level, etc.

3. The method according to claim 1, characterized in that Generating a draft training plan according to the training objectives and data set characteristics includes: Determine a stage division strategy according to the training objectives and data set characteristics, wherein the stage division strategy includes at least a model pre-training stage, a feature extraction stage, an end-to-end training stage, etc.; For each stage, set the corresponding learning rate strategy, data enhancement strategy, model optimization strategy, etc.

4. The method according to claim 3, characterized in that Before each stage starts, adjusting the training parameters of the current stage according to the training results of the previous stage includes: According to the performance evaluation results of the model in the previous stage, the learning rate strategy of the current stage is adjusted, and the learning rate strategy adjustment at least includes learning rate decay, learning rate rebound, etc.; According to the performance evaluation results of the model in the previous stage, the regularization parameters of the current stage are adjusted, and the regularization parameter adjustment includes at least the weight attenuation coefficient and the Dropout ratio; According to the performance evaluation results of the model in the previous stage, the data enhancement strategy of the current stage is adjusted, and the data enhancement strategy adjustment includes at least rotation angle, scaling ratio, and shear ratio.

5. The method according to claim 1, characterized in that At the end of each stage, the model performance is evaluated including: Evaluate the model using the validation set, where the evaluation includes at least accuracy, recall, and F1 score; Whether the criteria for entering the next stage are met is determined based on the evaluation results, and the criteria at least include a performance indicator threshold and a convergence speed.

6. The method according to claim 1, characterized in that Also includes: During the entire training process, model snapshots are saved regularly to facilitate resuming training or model fusion; After training, the final model is comprehensively evaluated and necessary hyperparameters are fine-tuned based on the evaluation results.

7. The method according to claim 1, characterized in that The generating a draft training plan according to the training objectives and the characteristics of the data set further comprises: A dynamic learning rate adjustment strategy is used, which calculates the learning rate using the following formula: Among them, η(t) represents the learning rate at time t; η max is the maximum learning rate; L v (t) is the loss function value on the validation set at the tth training iteration; L threshold is the loss function threshold; α and β control the degree and speed of the impact of the loss function on the learning rate; γ and δ control the degree and speed of the impact of the training progress on the learning rate; epoch is the current training round; E is the total training round.

8. A phased training formulation device based on artificial intelligence, characterized in that: include: The acquisition module is used to obtain the training target and data set characteristics; A generation module, used to generate a draft training plan according to the training objectives and data set characteristics, wherein the draft training plan includes stage division, learning rate strategy for each stage, data enhancement strategy, and model evaluation criteria; An initialization module, used to initialize one or more neural network models based on the draft training plan and allocate corresponding training resources to each stage; In the learning module, before each stage starts, the training parameters of the current stage are adjusted according to the training results of the previous stage, and the adjustment includes at least learning rate adjustment, regularization parameter adjustment or data enhancement strategy adjustment; The evaluation module is used to evaluate the model performance at the end of each stage and decide whether to enter the next stage or repeat the training of the current stage based on the evaluation results.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the artificial intelligence-based phased training formulation as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the artificial intelligence-based phased training formulation according to any one of claims 1 to 7.

Images