Classification model training method, device and computer-readable storage medium

Abstract

The present disclosure relates to a training method, device, and computer-readable storage medium for a classification model, and relates to the field of artificial intelligence technology. The method includes: dividing samples for training into multiple sub-batches, wherein each sub-batch includes one or more samples; inputting the samples of each sub-batch into a classification model, determining a gradient vector of the classification error corresponding to each sub-batch relative to an initialization parameter of the classification model; determining a value of an objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the classification loss of the classification model; if the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the classification model according to the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the classification model as optimized initialization parameters, and training the classification model with the optimized initialization parameters.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a classification model training method, device, and computer-readable storage medium. Background Art

[0002] Deep learning-based classification algorithms have developed rapidly in recent years and have been gradually applied to various industries in our daily lives, work, and education, such as facial recognition, posture classification, medical image analysis, obstacle detection for autonomous vehicles, web page classification, and audio classification. However, due to the large number of parameters in deep classification models, they often require appropriate initialization before training to optimize them using gradient descent algorithms and obtain a convergent and stable classification model.

[0003] However, due to the diversity of real-world application scenarios and classification datasets, initialization parameters that are appropriate for one dataset may no longer be appropriate for a transformed dataset. Therefore, the field of deep classification networks raises the question of learning-based initialization, specifically how to quickly and efficiently find the optimal initialization parameters through learning optimization.

[0004] Related methods optimize initialization parameters based on the curvature of the initial optimization region. The quality of these initialization parameters is evaluated by calculating the curvature of the classification model function at these initial parameters. Curvature reflects the degree of change in the parameter gradient as the optimization progresses. Generally, the smaller the change in the initialization parameter gradient during the optimization process, the smaller the curvature, which corresponds to a more stable and easier-to-train optimization environment, and thus a better initialization. Summary of the Invention

[0005] The inventors discovered that related methods, which optimize initialization parameters based on the curvature of the initial optimization region, involve calculating the second-order gradient of the parameters, and the third-order gradient must also be calculated during the optimization process. This process consumes a lot of computing resources and fails to efficiently capture information applicable to a given dataset for optimizing initialization parameters. Furthermore, this method fails to consider the model's classification error and cannot guarantee the model's classification accuracy.

[0006] A technical problem to be solved by the present disclosure is: how to improve the efficiency of initialization parameter optimization of the classification model, save computing resources and improve classification accuracy.

[0007] According to some embodiments of the present disclosure, a training method for a classification model is provided, comprising: dividing samples for training into multiple sub-batches, wherein each sub-batch includes one or more samples; inputting the samples of each sub-batch into the classification model, and determining the gradient vector of the classification error corresponding to each sub-batch relative to the initialization parameters of the classification model; determining the value of an objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the classification loss of the classification model; when the value of the objective function has not reached a maximum value, adjusting the initialization parameters of the classification model according to the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the classification model as optimized initialization parameters, and training the classification model with the optimized initialization parameters.

[0008] In some embodiments, the objective function includes at least one of a first sub-function and a second sub-function, and determining the value of the objective function according to the gradient vector corresponding to each sub-batch includes at least one of the following: determining the modulus of the gradient vector corresponding to each sub-batch according to the gradient vector corresponding to each sub-batch, and determining the value of the first sub-function according to the modulus of the gradient vector corresponding to each sub-batch; determining the similarity of the gradient vectors corresponding to every two sub-batches according to the gradient vector corresponding to each sub-batch, and determining the value of the second sub-function according to the similarity of the gradient vectors corresponding to every two sub-batches.

[0009] In some embodiments, determining the value of the first sub-function according to the modulus of the gradient vector corresponding to each sub-batch includes: determining an average modulus according to the modulus of the gradient vector corresponding to each sub-batch, as the value of the first sub-function.

[0010] In some embodiments, determining the similarity of the gradient vectors corresponding to every two subbatches based on the gradient vector corresponding to each subbatch, and determining the value of the second subfunction based on the similarity of the gradient vectors corresponding to every two subbatches includes: determining the cosine similarity of the gradient vectors corresponding to every two subbatches based on the gradient vector corresponding to each subbatch; and determining the average cosine similarity based on the cosine similarity of the gradient vectors corresponding to every two subbatches as the value of the second subfunction.

[0011] In some embodiments, the objective function further includes a condition that the maximum modulus of the moduli of the gradient vectors corresponding to each sub-batch does not exceed a preset value.

[0012] In some embodiments, the objective function is determined by the following method: performing an optimization surface analysis on the classification loss function of the classification model to determine an evaluation index function, wherein the evaluation index function is a function for determining the upper limit of the classification loss of the classification model; and determining the objective function based on the evaluation index function.

[0013] In some embodiments, the evaluation index function includes a third sub-function for determining the maximum modulus of the modulus of the gradient vector corresponding to each sub-batch, a fourth sub-function for determining the similarity between the gradient vectors corresponding to each two sub-batches, and a fifth sub-function for determining the ratio of the maximum modulus to the minimum modulus of the optimized initialization parameters corresponding to each sub-batch. Determining the objective function according to the evaluation index function includes: determining the condition that the maximum modulus of the modulus of the gradient vector corresponding to each sub-batch in the objective function does not exceed a preset value according to the third sub-function; determining the second sub-function according to the fourth sub-function; and converting the fifth sub-function into the first sub-function by approximating the optimized initialization parameters corresponding to each sub-batch by utilizing the first-order gradient of the initialization parameter and the gradient vector corresponding to each sub-batch.

[0014] In some embodiments, the cosine similarity of the gradient vectors corresponding to every two sub-batches is the ratio of the dot product of the gradient vectors corresponding to every two sub-batches to the modulus of the gradient vectors corresponding to every two sub-batches.

[0015] In some embodiments, adjusting the initialization parameters of the classification model according to the objective function includes: determining the gradient value of the objective function relative to the adjustment vector of the previous cycle; determining the adjustment vector of the current cycle based on the adjustment vector, gradient value and learning rate of the previous cycle; determining the initialization parameters of the classification model of the current cycle based on the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle.

[0016] In some embodiments, the product of the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle is used as the initialization parameters of the classification model of the current cycle.

[0017] In some embodiments, the average gradient vector of the classification error corresponding to each sample in each subbatch relative to the gradient vector of the initialization parameters of the classification model is used as the gradient vector of the classification error corresponding to each subbatch relative to the initialization parameters of the classification model.

[0018] In some embodiments, dividing the samples for training into multiple sub-batches includes: determining the number of samples in each sub-batch according to the number of samples, the number of sub-batches, and the overlap rate between sub-batches.

[0019] In some embodiments, training a classification model with optimized initialization parameters includes: inputting samples into the classification model with optimized initialization parameters to obtain classification results; determining the classification loss based on the classification results and the labeling information of the training samples; and adjusting the parameters of the classification model based on the classification loss and the classification loss function until the training is completed.

[0020] In some embodiments, the classification model is an image classification model, and the sample images used for training are divided into multiple sub-batches, wherein each sub-batch includes one or more sample images; the sample images of each sub-batch are input into the image classification model, and the gradient vector of the image classification error corresponding to each sub-batch relative to the initialization parameters of the image classification model is determined; based on the gradient vector corresponding to each sub-batch, the value of the objective function is determined, wherein the objective function is negatively correlated with the image classification loss of the image classification model; if the value of the objective function does not reach a maximum value, the initialization parameters of the image classification model are adjusted according to the objective function; until the value of the objective function reaches a maximum value, the initialization parameters of the image classification model are used as the optimized initialization parameters, and the image classification model with the optimized initialization parameters is trained.

[0021] In some embodiments, the classification model is a text classification model, and the sample text for training is divided into multiple sub-batches, wherein each sub-batch includes one or more sample texts; the sample text of each sub-batch is input into the text classification model, and the gradient vector of the text classification error corresponding to each sub-batch relative to the initialization parameters of the text classification model is determined; based on the gradient vector corresponding to each sub-batch, the value of the objective function is determined, wherein the objective function is negatively correlated with the text classification loss of the text classification model; if the value of the objective function does not reach a maximum value, the initialization parameters of the text classification model are adjusted according to the objective function; until the value of the objective function reaches a maximum value, the initialization parameters of the text classification model are used as optimized initialization parameters, and the text classification model with the optimized initialization parameters is trained.

[0022] In some embodiments, the classification model is an audio classification model, and the sample audio for training is divided into multiple sub-batches, wherein each sub-batch includes one or more sample audios; the sample audio of each sub-batch is input into the audio classification model, and the gradient vector of the audio classification error corresponding to each sub-batch relative to the initialization parameters of the audio classification model is determined; based on the gradient vector corresponding to each sub-batch, the value of the objective function is determined, wherein the objective function is negatively correlated with the audio classification loss of the audio classification model; if the value of the objective function does not reach the maximum value, the initialization parameters of the audio classification model are adjusted according to the objective function; until the value of the objective function reaches the maximum value, the initialization parameters of the audio classification model are used as the optimized initialization parameters, and the audio classification model with the optimized initialization parameters is trained.

[0023] In some embodiments, the classification model is a video classification model, and the sample videos for training are divided into multiple sub-batches, wherein each sub-batch includes one or more sample videos; the sample videos of each sub-batch are input into the video classification model, and the gradient vector of the video classification error corresponding to each sub-batch relative to the initialization parameters of the video classification model is determined; based on the gradient vector corresponding to each sub-batch, the value of the objective function is determined, wherein the objective function is negatively correlated with the video classification loss of the video classification model; when the value of the objective function does not reach a maximum value, the initialization parameters of the video classification model are adjusted according to the objective function; until the value of the objective function reaches a maximum value, the initialization parameters of the video classification model are used as optimized initialization parameters, and the video classification model with the optimized initialization parameters is trained.

[0024] In some embodiments, the method further includes: inputting the object to be classified into a trained classification model to obtain a category of the object to be classified; wherein the object to be classified includes: an image, text, video, or audio.

[0025] According to some other embodiments of the present disclosure, a training device for a classification model is provided, including: a division module for dividing samples for training into multiple sub-batches, wherein each sub-batch includes one or more samples; a first determination module for inputting the samples of each sub-batch into the classification model, and determining the gradient vector of the classification error corresponding to each sub-batch relative to the initialization parameters of the classification model; a second determination module for determining the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the classification loss of the classification model; an adjustment module for adjusting the initialization parameters of the classification model according to the objective function when the value of the objective function has not reached a maximum value; and a training module for using the initialization parameters of the classification model as optimized initialization parameters until the value of the objective function reaches a maximum value, and training the classification model with the optimized initialization parameters.

[0026] According to some further embodiments of the present disclosure, a classification model training device is provided, comprising: a processor; and a memory coupled to the processor, for storing instructions, which, when executed by the processor, causes the processor to execute a classification model training method as described in any of the aforementioned embodiments.

[0027] According to some further embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, the training method of the classification model of any of the aforementioned embodiments is implemented.

[0028] In the training method of the classification model disclosed in the present invention, the samples are divided into sub-batches, and for each sub-batch, the gradient vector of the classification error corresponding to the sub-batch relative to the initialization parameters of the classification model is determined. According to the gradient vector corresponding to each sub-batch, the value of the objective function is determined, and then the initialization parameters of the classification model are adjusted according to the value of the objective function, thereby obtaining the optimized initialization parameters, and then the classification model is trained. Since the value of the objective function is determined according to the gradient vector corresponding to each sub-batch, only the first-order derivative calculation is involved, which can improve the efficiency of the optimization of the initialization parameters of the classification model and save computing resources. In addition, the objective function is negatively correlated with the classification loss of the classification model. By adjusting the value of the objective function to the maximum value, the classification loss can be minimized. Therefore, after training the classification model with optimized initialization parameters, the accuracy of the classification model can be improved.

[0029] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 A flowchart illustrating a method for training a classification model according to some embodiments of the present disclosure is shown.

[0032] Figure 2 A flowchart illustrating a method for training a classification model according to other embodiments of the present disclosure is shown.

[0033] Figure 3 A schematic structural diagram of a training device for a classification model according to some embodiments of the present disclosure is shown.

[0034] Figure 4 A schematic structural diagram of a training device for a classification model according to other embodiments of the present disclosure is shown.

[0035] Figure 5 A schematic structural diagram of a training device for a classification model according to some further embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0037] This disclosure proposes a training method for a classification model. Figures 1-2 Provide a description.

[0038] Figure 1 Flowchart of some embodiments of the training method of the classification model disclosed in the present invention. Figure 1 As shown, the method of this embodiment includes steps S102 to S110.

[0039] In step S102 , the samples for training are divided into multiple sub-batches.

[0040] Each subbatch includes one or more samples. When the samples used for training are divided into multiple batches, each batch of samples can be divided into multiple subbatches (mini-batches). When each subbatch includes multiple samples, it can save resources in the subsequent objective function calculation and improve efficiency. The subsequent process will be described in detail.

[0041] In some embodiments, the number of samples in each sub-batch is determined based on the number of samples, the number of sub-batches, and the overlap ratio between sub-batches. For example, the number of samples is divided by the difference between the number of sub-batches and the overlap ratio between sub-batches, and the resulting ratio is rounded down to obtain the number of samples in each sub-batch. By setting sample overlap between consecutive sub-batches, the optimization process can be stabilized.

[0042] For example, each batch of samples S = (x i ,y i ) i∈[B] Divide into D sub-batches, set the sub-batch overlap rate to r, so that for a batch of input with B samples, the number of samples in each sub-batch can be And the sample number corresponding to each sub-batch can be written as follows:

[0043] S d =(x i ,y i ) i＝N(d-1)(1-r)，…，N(d-1)(1-r)+N (1)

[0044] Where d = 1, ..., D. When D = B and r = 0, the number of samples in each subbatch N = 1. By reducing the number of subbatches D and correspondingly increasing the overlap ratio r, the efficiency of the model's gradient calculation can be effectively improved without affecting the quality of the final initialization parameters.

[0045] In step S104, the samples of each sub-batch are input into the classification model, and the gradient vector of the classification error corresponding to each sub-batch relative to the initialization parameters of the classification model is determined.

[0046] The initialization parameters can be randomly generated. For each subbatch, the samples of the subbatch are input into the classification model to obtain the gradient vector of the classification error corresponding to each sample with respect to the initialization parameters of the classification model. The average gradient vector of the gradient vector of the classification error corresponding to each sample in each subbatch with respect to the initialization parameters of the classification model is used as the gradient vector of the classification error corresponding to each subbatch with respect to the initialization parameters of the classification model, that is, the gradient vector corresponding to each subbatch.

[0047] For example, for the gradient vector g corresponding to the i-th sub-batch i , can be calculated using the following formula.

[0048]

[0049] Among them, x j represents the data of sample j, y j represents the labeled data of sample j, f θ (·) is the representation function of the classification model, l(·) represents the loss function, θ0 represents the initialization parameter, Represents the gradient vector of the classification error corresponding to sample j relative to the initialization parameters of the classification model.

[0050] In step S106, the value of the objective function is determined according to the gradient vector corresponding to each sub-batch.

[0051] The objective function is negatively correlated with the classification loss of the classification model. Those skilled in the art can understand, based on the gradient calculation method or formula (2), that the gradient vector corresponding to each subbatch is calculated based on the initialization parameters of the classification model and the data of the samples of each subbatch. Based on the gradient vector corresponding to each subbatch, the value of the objective function can be determined, that is, the objective function is also determined based on the initialization parameters of the classification model and the data of the samples of each subbatch. The objective function uses the data of the samples of each subbatch and the initialization parameters as variables. When the samples are determined, the value of the objective function can be adjusted by adjusting the initialization parameters.

[0052] In some embodiments, the objective function includes at least one of a first sub-function and a second sub-function. Furthermore, the objective function may include a condition that the maximum modulus of the moduli of the gradient vectors corresponding to each sub-batch does not exceed a preset value. When the objective function includes the first sub-function and the second sub-function, the objective function may be expressed as the sum of the first sub-function and the second sub-function.

[0053] In some embodiments, based on the gradient vector corresponding to each subbatch, the modulus of the gradient vector corresponding to each subbatch is determined, and the value of the first sub-function is determined based on the modulus of the gradient vector corresponding to each subbatch; based on the gradient vector corresponding to each subbatch, the similarity between the gradient vectors corresponding to every two subbatches is determined, and the value of the second sub-function is determined based on the similarity between the gradient vectors corresponding to every two subbatches.

[0054] Furthermore, in some embodiments, an average modulus is determined based on the modulus of the gradient vectors corresponding to each subbatch, and used as the value of the first subfunction. Based on the gradient vectors corresponding to each subbatch, the cosine similarity of the gradient vectors corresponding to each two subbatches is determined; and based on the cosine similarity of the gradient vectors corresponding to each two subbatches, the average cosine similarity is determined as the value of the second subfunction. The similarity of the gradient vectors corresponding to each two subbatches can be calculated using cosine similarity or other calculation methods, such as Euclidean distance, without being limited to the examples given. However, using cosine similarity is more effective in improving classification accuracy.

[0055] For example, the first sub-function is a function that determines the average norm of the norms of the gradient vectors corresponding to each sub-batch. For example, for each batch S, based on the initial parameter θ0, the first sub-function GradNorm can be determined using the following formula.

[0056]

[0057] For example, the second sub-function is a function that determines the average similarity of the gradient vectors corresponding to each two sub-batches. The similarity is, for example, cosine similarity. The cosine similarity of the gradient vectors corresponding to each two sub-batches is the ratio of the dot product of the gradient vectors corresponding to each two sub-batches to the modulus of the gradient vectors corresponding to each two sub-batches. For example, the cosine similarity between the gradient vector corresponding to the i-th sub-batch and the gradient vector corresponding to the j-th sub-batch can be determined using the following formula.

[0058]

[0059] Furthermore, for each batch S, based on the initial parameter θ0, the second sub-function GradCosine can be determined using the following formula.

[0060]

[0061] The objective function may also include a condition that the maximum modulus of the modulus of the gradient vector corresponding to each subbatch does not exceed a preset value, which can be expressed by the following formula.

[0062]

[0063] Among them, γ is a fixed upper bound of the gradient, i.e., a preset value.

[0064] For the initialization parameters θ0=[W1, ..., W m ], the corresponding adjustment vector is recorded as M = [ω1, ..., ω m ], the adjusted initialization parameter is recorded as θ M =[ω1W1, ..., ω m W m ]. By adjusting the initialization parameters in this way, the overall variance of the initialization parameters can be optimized. The optimization of the initialization parameters can be achieved by adjusting M. Therefore, in the process of optimizing the initialization parameters, M is used as a variable. M Based on this, for the gradient vector g corresponding to the i-th sub-batch i , can be calculated using the following formula.

[0065]

[0066] In the variable initialization parameter θ M Based on this, the first sub-function GradNorm can be determined using the following formula.

[0067]

[0068] In the variable initialization parameter θ M Based on this, the second sub-function GradCosine can be determined using the following formula.

[0069]

[0070] The objective function may include the first sub-function or the second sub-function. For example, when the objective function includes the first sub-function, the objective function may be expressed using the following formula.

[0071]

[0072] Among them, GC(S,θ M )=GradCosine(S,θ M ).

[0073] When the objective function includes a first sub-function and a second sub-function, the objective function can be expressed using the following formula.

[0074]

[0075] Among them, GC(S,θ M )=GradCosine(S,θ M ), GN(S, θ M ) = GradNorm(S, θ M ), please refer to formulas (8) and (9).

[0076] It can be seen from formulas (10) and (11) that the objective function takes the sample data and the adjustment vector as variables. When the sample is fixed, the value of the objective function can be changed by changing the adjustment vector so that the objective function reaches its maximum value.

[0077] For large datasets, if the similarity between the gradient vectors corresponding to each two samples is calculated, the time and resources consumed by the algorithm are unbearable. In order to make the calculation more efficient, each batch of samples is divided into several sub-batches, and different sub-batches are treated as single samples, thereby reducing the gradient g i The number of calculations is reduced to improve the efficiency of initialization parameter optimization.

[0078] In step S108 , when the value of the objective function has not reached the maximum value, the initialization parameters of the classification model are adjusted according to the objective function.

[0079] The initialization parameters can be optimized using a gradient descent method based on the objective function over one or more cycles. In some embodiments, the gradient value of the objective function relative to the adjustment vector of the previous cycle is determined; the adjustment vector of the current cycle is determined based on the adjustment vector of the previous cycle, the gradient value, and the learning rate; and the initialization parameters of the classification model of the current cycle are determined based on the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle. As in the aforementioned embodiment, the product of the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle is used as the initialization parameters of the classification model of the current cycle.

[0080] In step S110 , until the value of the objective function reaches a maximum value, the initialization parameters of the classification model are used as the optimized initialization parameters, and the classification model with the optimized initialization parameters is trained.

[0081] In some embodiments, samples are input into a classification model with optimized initialization parameters to obtain classification results; a classification loss is determined based on the classification results and the labeled information of the training samples; and parameters of the classification model are adjusted based on the classification loss and the classification loss function until training is complete. The training process can use existing methods and will not be further described here.

[0082] The optimization process of initialization parameters is as follows:

[0083] (1) Divide the samples used for training in the current cycle into multiple sub-batches, where each sub-batch includes one or more samples;

[0084] (2) Input the samples of each sub-batch of the current cycle into the classification model and determine the gradient vector of the classification error corresponding to each sub-batch of the current cycle relative to the initialization parameters of the current cycle;

[0085] (3) Determine the value of the objective function of the current cycle based on the gradient vector corresponding to each sub-batch of the current cycle, where the objective function is negatively correlated with the classification loss of the classification model;

[0086] (4) When the value of the objective function of the current cycle does not reach the maximum value, the initialization parameters of the classification model are adjusted according to the objective function of the current cycle to obtain the initialization parameters of the classification model for the next cycle, and the next cycle is entered and the process returns to step (1) to start again;

[0087] (5) When the value of the objective function in the current period reaches the maximum value, the initialization parameters of the classification model are used as the optimized initialization parameters, and the classification model with the optimized initialization parameters is trained.

[0088] The following describes the theoretical basis of the objective function in the present disclosure and how to obtain the objective function.

[0089] In some embodiments, an optimization surface analysis is performed on the loss function of the classification model to determine an evaluation index function, wherein the evaluation index function is a function for determining the upper limit of the classification loss of the classification model; and the objective function is determined based on the evaluation index function.

[0090] An optimization surface analysis is performed on the loss function of the deep learning classification model to obtain the following evaluation index function, which is used to represent the upper limit of the training or test (generalization) classification error. That is, the smaller the value of the evaluation index function, the smaller the training or test (generalization) classification error.

[0091]

[0092] The left side of the equation shows that using As the initialization parameter of the deep learning classification model f, in the training data set S = (x i ,y j ) i∈[n] The initialization index Θ is calculated under the condition of and loss function l. The right side of the equation shows how to calculate the index, which mainly includes:

[0093] a. Fixed constants, including smooth upper bounds Model learning rate η; input sample size n and fixed parameters and C2. These constants are not affected by the initialization parameter θ0, so they can be ignored during optimization and do not need to be calculated separately.

[0094] b. The gradient to be optimized, including the maximum modulus of the gradient vector corresponding to each sub-batch in each batch of input samples And the cosine similarity cos(g i , g j ). where g i Represents the gradient vector corresponding to the i-th sub-batch, which will be affected by the initialization parameter θ0, and g i The calculation method of is also affected by the division of different sub-batches in each batch.

[0095] c. Modulus of parameters to be optimized Where α and β represent the optimized initialization parameters corresponding to each sub-batch. The maximum and minimum modulus lengths of because It needs to be obtained through full training of the classification model and cannot be represented by the initialization parameter θ0. Therefore, it is approximated by the first-order gradient approximation of θ0.

[0096] It can be theoretically proved that the indicator It is closely related to the model's training dynamics (convergence speed and stability) and final classification performance. Specifically, smaller Θ corresponds to smaller classification error and better training dynamics.

[0097] The objective function can be constructed based on the items to be optimized in the above evaluation index function. For example, the objective function includes a third sub-function for determining the maximum modulus of the modulus of the gradient vectors corresponding to each sub-batch, a fourth sub-function for determining the cosine similarity between the gradient vectors corresponding to each two sub-batches, and a fourth sub-function for determining the ratio of the maximum and minimum modulus of the optimized initialization parameters corresponding to each sub-batch. The fifth subfunction of .

[0098] Furthermore, in some embodiments, a condition that the maximum modulus of the modulus of the gradient vector corresponding to each subbatch in the objective function does not exceed a preset value is determined according to the third sub-function; the second sub-function is determined according to the fourth sub-function; and the fifth sub-function is converted into the first sub-function by approximating the optimized initialization parameters corresponding to each subbatch by utilizing the first-order gradient of the initialization parameters and the gradient vector corresponding to each subbatch.

[0099] The third sub-function determines the maximum modulus of the modulus of the gradient vectors corresponding to each sub-batch. By constraining the maximum modulus to be less than a given upper bound γ, the maximum modulus can be effectively reduced, and the condition that the maximum modulus of the modulus of the gradient vectors corresponding to each sub-batch in the objective function does not exceed a preset value is obtained. The fourth sub-function determines the cosine similarity between the gradient vectors corresponding to each two sub-batches, which can be calculated according to formula (9).

[0100] The ratio of the maximum and minimum modulus lengths of the optimized initialization parameters corresponding to each sub-batch is determined The fifth function of It needs to be obtained through full training of the classification model and cannot be represented by the initialization parameter θ0. Therefore, it is approximated by the first-order gradient approximation of θ0, as shown in the following formula.

[0101]

[0102] Among them, η represents the parameter learning rate, g i is the gradient vector corresponding to the i-th sub-batch. Based on the first-order approximation, The modulus length can be determined to some extent by g i Furthermore, because we note that By constraining all g i The modulus lengths of are close to γ, so that α≈β, and then As small as possible, ideally reaching a minimum value of 1. Since the maximum gradient modulus has been constrained to be less than γ, in order to make all g i The modulus of the gradient is close to γ, and only one more condition needs to be added, that is, under the constraint that the maximum gradient modulus is less than γ, all g are maximized at the same time. i The modulus of , that is, the function expressed by formula (8) is maximized.

[0103] In the training method of the classification model of the above embodiment, the samples are divided into sub-batches. For each sub-batch, the gradient vector of the classification error corresponding to the sub-batch relative to the initialization parameters of the classification model is determined. The value of the objective function is determined based on the gradient vector corresponding to each sub-batch, and then the initialization parameters of the classification model are adjusted according to the value of the objective function, thereby obtaining the optimized initialization parameters, and then the classification model is trained. Since the value of the objective function is determined based on the gradient vector corresponding to each sub-batch, only the first-order derivative calculation is involved, which can improve the efficiency of the optimization of the initialization parameters of the classification model and save computing resources. In addition, the objective function is negatively correlated with the classification loss of the classification model. By adjusting the value of the objective function to the maximum value, the classification loss can be minimized. Therefore, after training the classification model with the optimized initialization parameters, the accuracy of the classification model can be improved.

[0104] The method disclosed herein can be applied to scenarios such as image classification, text classification, audio classification, or video classification.

[0105] In some embodiments, the samples are sample images, and the classification model is an image classification model. Steps S102 to S110 can be replaced by dividing the sample images for training into multiple sub-batches, wherein each sub-batch includes one or more sample images; inputting the sample images of each sub-batch into the image classification model, determining the gradient vector of the image classification error corresponding to each sub-batch relative to the initialization parameters of the image classification model; determining the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the image classification loss of the image classification model; if the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the image classification model according to the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the image classification model as optimized initialization parameters, and training the image classification model with the optimized initialization parameters.

[0106] Based on the method of the above embodiment, the initialization parameters of the image classification model are optimized. In each step, the parameters of the image classification model are derived based on the data of the sample image, and the ultimately optimized initialization parameters are also closely related to the sample image data. During the training process of the image classification model with the optimized initialization parameters, the training efficiency of the image classification model can be improved, and the accuracy of the final image classification can be improved.

[0107] In some embodiments, the sample is a sample text, and the classification model is a text classification model. Steps S102 to S110 can be replaced by dividing the sample text for training into multiple sub-batches, wherein each sub-batch includes one or more sample texts; inputting the sample text of each sub-batch into the text classification model, determining the gradient vector of the text classification error corresponding to each sub-batch relative to the initialization parameters of the text classification model; determining the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the text classification loss of the text classification model; if the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the text classification model based on the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the text classification model as optimized initialization parameters, and training the text classification model with the optimized initialization parameters.

[0108] Based on the method of the above embodiment, the initialization parameters of the text classification model are optimized. In each step, the parameters of the text classification model are obtained based on the data of the sample text, and the optimized initialization parameters are also closely related to the data of the sample text. During the training process of the text classification model with the optimized initialization parameters, the training efficiency of the text classification model can be improved, and the accuracy of the final text classification can be improved.

[0109] In some embodiments, the sample is sample audio, and the classification model is an audio classification model. Steps S102 to S110 can be replaced by dividing the sample audio for training into multiple sub-batches, wherein each sub-batch includes one or more sample audios; inputting the sample audio of each sub-batch into the audio classification model, determining the gradient vector of the audio classification error corresponding to each sub-batch relative to the initialization parameters of the audio classification model; determining the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the audio classification loss of the audio classification model; if the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the audio classification model according to the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the audio classification model as optimized initialization parameters, and training the audio classification model with the optimized initialization parameters.

[0110] Based on the method of the above embodiment, the initialization parameters of the audio classification model are optimized. In each step, the parameters of the audio classification model are obtained based on the sample audio data, and the optimized initialization parameters are closely related to the sample audio data. During the training process of the audio classification model with the optimized initialization parameters, the training efficiency of the audio classification model can be improved, and the accuracy of the final audio classification can be improved.

[0111] In some embodiments, the sample is a sample video, and the classification model is a video classification model. Steps S102 to S110 can be replaced by dividing the sample video for training into multiple sub-batches, wherein each sub-batch includes one or more sample videos; inputting the sample videos of each sub-batch into the video classification model, determining the gradient vector of the video classification error corresponding to each sub-batch relative to the initialization parameters of the video classification model; determining the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the video classification loss of the video classification model; if the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the video classification model according to the objective function; until the value of the objective function reaches a maximum value, using the initialization parameters of the video classification model as optimized initialization parameters, and training the video classification model with the optimized initialization parameters.

[0112] Based on the method of the above embodiment, the initialization parameters of the video classification model are optimized. In each step, the parameters of the video classification model are obtained based on the data of the sample video, and the optimized initialization parameters are closely related to the sample video data. During the training process of the video classification model with the optimized initialization parameters, the training efficiency of the video classification model can be improved, and the accuracy of the final video classification can be improved.

[0113] Since image classification models and video classification models are more complex and have more parameters, the solution disclosed in the present invention is more effective in the fields of image classification and video classification.

[0114] The method disclosed herein can improve the processing speed and efficiency of the processor when executed by a processor of a computer. In some embodiments, the processor divides the samples for training into multiple sub-batches, wherein each sub-batch includes one or more samples; the processor inputs the samples of each sub-batch into a classification model, and determines the gradient vector of the classification error corresponding to each sub-batch relative to the initialization parameters of the classification model; the processor determines the value of the objective function based on the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the classification loss of the classification model; when the value of the objective function does not reach a maximum value, the processor adjusts the initialization parameters of the classification model according to the objective function; until the value of the objective function reaches a maximum value, the processor uses the initialization parameters of the classification model as the optimized initialization parameters, and trains the classification model with the optimized initialization parameters, thereby improving the processing speed and efficiency of the processor.

[0115] The following combination Figure 2 Describe some application examples of the training method of the classification model disclosed in the present invention. Figure 2 As shown in the figure, the parameters of the classification model are first randomly initialized to obtain the initialization parameters. Each batch of training data is divided into multiple subbatches. The gradient of each subbatch is calculated, and then the gradient modulus of each subbatch is calculated, as well as the cosine similarity of the gradients between every two subbatches. Based on the objective function, the gradient modulus, cosine similarity, and parameter modulus are optimized to ultimately obtain the optimal initialization parameters.

[0116] The disclosed solution utilizes optimization surface analysis of the loss function of a deep learning classification model to derive an evaluation index function that simultaneously reflects both the model training dynamics and the final classification performance. This evaluation index function determines the objective function, which only involves first-order derivative calculations and is differentiable with respect to initialization parameters, making it suitable for subsequent optimization algorithms. Initialization parameters are adjusted based on the objective function to obtain the optimal initialization parameters for model training and testing.

[0117] First, the samples in the dataset to be trained are input into the classification model in batches. Each batch of samples is first divided into sub-batches, and then the classification error is calculated for each sub-batch of samples, and the gradient of the classification error with respect to the initialization parameters is calculated. Then, based on the sub-batch gradients, the gradient vector modulus and the similarity of the gradient vectors between different sub-batches are calculated, and the value of the objective function is finally output. Furthermore, the model initialization parameters are adjusted according to the objective function using a gradient descent algorithm. For example, the objective function is maximized through gradient modulus optimization, cosine similarity optimization, and parameter modulus optimization, thereby obtaining initialization parameters corresponding to better training dynamics and better training results.

[0118] In this disclosure, an evaluation index function effectively reflects the model's training dynamics and the quality of the final classification results. After converting the evaluation index function into an objective function, the specific optimization process primarily utilizes first-order approximation and combines it with constrained optimization. By setting the optimization objective and optimization constraints, the problem of some uncalculated quantities in the evaluation index function is resolved. This allows for efficient optimization of the evaluation index function, finding the optimal initialization parameters for the model in a more convenient and efficient manner.

[0119] The present disclosure also provides a training device for a classification model. Figure 3 Provide a description.

[0120] Figure 3 Figure 1 is a structural diagram of some embodiments of the training device for the classification model disclosed herein. Figure 3 As shown, the apparatus 30 of this embodiment includes: a division module 310 , a first determination module 320 , a second determination module 330 , an adjustment module 340 , and a training module 350 .

[0121] The partitioning module 310 is used to partition the samples for training into multiple sub-batches, where each sub-batch includes one or more samples.

[0122] In some embodiments, the partitioning module 310 is configured to determine the number of samples in each sub-batch according to the number of samples, the number of sub-batches, and the overlap rate between sub-batches.

[0123] The first determination module 320 is used to input the samples of each sub-batch into the classification model, and determine the gradient vector of the classification error corresponding to each sub-batch with respect to the initialization parameters of the classification model.

[0124] In some embodiments, the first determination module 320 is used to use the average gradient vector of the classification error corresponding to each sample in each sub-batch relative to the gradient vector of the initialization parameters of the classification model as the gradient vector of the classification error corresponding to each sub-batch relative to the initialization parameters of the classification model.

[0125] The second determination module 330 is used to determine the value of the objective function according to the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the classification loss of the classification model.

[0126] In some embodiments, the objective function includes at least one of a first sub-function and a second sub-function, and the second determination module 330 is used to perform at least one of the following: determining the modulus of the gradient vector corresponding to each sub-batch based on the gradient vector corresponding to each sub-batch, and determining the value of the first sub-function based on the modulus of the gradient vector corresponding to each sub-batch; determining the similarity of the gradient vectors corresponding to every two sub-batches based on the gradient vector corresponding to each sub-batch, and determining the value of the second sub-function based on the similarity of the gradient vectors corresponding to every two sub-batches.

[0127] In some embodiments, the second determining module 330 is configured to determine an average modulus according to the modulus of the gradient vector corresponding to each sub-batch, as the value of the first sub-function.

[0128] In some embodiments, the second determination module 330 is used to determine the cosine similarity of the gradient vectors corresponding to every two subbatches based on the gradient vector corresponding to each subbatch; and determine the average cosine similarity based on the cosine similarity of the gradient vectors corresponding to every two subbatches as the value of the second subfunction.

[0129] In some embodiments, the cosine similarity of the gradient vectors corresponding to every two sub-batches is the ratio of the dot product of the gradient vectors corresponding to every two sub-batches to the modulus of the gradient vectors corresponding to every two sub-batches.

[0130] In some embodiments, the objective function further includes a condition that the maximum modulus of the moduli of the gradient vectors corresponding to each sub-batch does not exceed a preset value.

[0131] In some embodiments, the second determination module 330 is used to perform an optimization surface analysis on the classification loss function of the classification model to determine an evaluation index function, wherein the evaluation index function is a function for determining the upper limit of the classification loss of the classification model; and determine the objective function based on the evaluation index function.

[0132] In some embodiments, the evaluation index function includes a third sub-function for determining the maximum modulus of the modulus of the gradient vector corresponding to each sub-batch, a fourth sub-function for determining the similarity between the gradient vectors corresponding to each two sub-batches, and a fifth sub-function for determining the ratio of the maximum modulus to the minimum modulus of the optimized initialization parameters corresponding to each sub-batch. The second determination module 330 is used to determine the condition that the maximum modulus of the modulus of the gradient vector corresponding to each sub-batch in the objective function does not exceed a preset value based on the third sub-function; determine the second sub-function based on the fourth sub-function; and convert the fifth sub-function into the first sub-function by approximating the optimized initialization parameters corresponding to each sub-batch using the first-order gradient of the initialization parameter and the gradient vector corresponding to each sub-batch.

[0133] The adjustment module 340 is configured to adjust the initialization parameters of the classification model according to the objective function when the value of the objective function has not reached the maximum value.

[0134] In some embodiments, the adjustment module 340 is used to determine the gradient value of the objective function relative to the adjustment vector of the previous cycle; determine the adjustment vector of the current cycle based on the adjustment vector, gradient value and learning rate of the previous cycle; determine the initialization parameters of the classification model of the current cycle based on the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle.

[0135] In some embodiments, the product of the adjustment vector of the current cycle and the initialization parameters of the classification model of the previous cycle is used as the initialization parameters of the classification model of the current cycle.

[0136] The training module 350 is configured to train the classification model with the optimized initialization parameters until the value of the objective function reaches a maximum value, using the initialization parameters of the classification model as the optimized initialization parameters.

[0137] In some embodiments, the training module 350 is used to input samples into a classification model with optimized initialization parameters to obtain classification results; determine the classification loss based on the classification results and the labeling information of the training samples; and adjust the parameters of the classification model based on the classification loss and the classification loss function until the training is completed.

[0138] In some embodiments, the apparatus 30 further includes a classification module 360 ​​for inputting the object to be classified into a trained classification model to obtain a category of the object to be classified; wherein the object to be classified includes an image, text, video or audio.

[0139] The training device of the classification model in the embodiment of the present disclosure can be implemented by various computing devices or computer systems. Figure 4 as well as Figure 5 Provide a description.

[0140] Figure 4 Figure 1 is a structural diagram of some embodiments of the training device for the classification model disclosed herein. Figure 4 As shown, the device 40 of this embodiment includes: a memory 410 and a processor 420 coupled to the memory 410, and the processor 420 is configured to execute the training method of the classification model in any of some embodiments of the present disclosure based on the instructions stored in the memory 410.

[0141] The memory 410 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0142] Figure 5 Figure 1 is a structural diagram of some other embodiments of the training device for the classification model disclosed herein. Figure 5 As shown, the device 50 of this embodiment includes: a memory 510 and a processor 520, which are similar to the memory 410 and the processor 420, respectively. It may also include an input / output interface 530, a network interface 540, a storage interface 550, etc. These interfaces 530, 540, 550 and the memory 510 and the processor 520 can be connected, for example, via a bus 560. Among them, the input / output interface 530 provides a connection interface for input / output devices such as a display, mouse, keyboard, and touch screen. The network interface 540 provides a connection interface for various networked devices, such as a database server or a cloud storage server. The storage interface 550 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0143] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0147] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A classification model training method, comprising: Dividing the samples for training into multiple sub-batches, where each sub-batch includes one or more samples; Inputting the samples of each sub-batch into the classification model, and determining the gradient vector of the classification error corresponding to each sub-batch with respect to the initialization parameters of the classification model; Determining a value of an objective function based on the gradient vector corresponding to each subbatch, wherein the objective function is negatively correlated with the classification loss of the classification model; the objective function includes at least one of a first subfunction and a second subfunction, and determining the value of the objective function includes at least one of the following: determining a modulus of the gradient vector corresponding to each subbatch based on the gradient vector corresponding to each subbatch, and determining a value of the first subfunction based on the modulus of the gradient vector corresponding to each subbatch; determining a similarity between the gradient vectors corresponding to each two subbatches based on the gradient vector corresponding to each subbatch, and determining a value of the second subfunction based on the similarity of the gradient vectors corresponding to each two subbatches; When the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the classification model according to the objective function; until the value of the objective function reaches a maximum value, the initialization parameters of the classification model are used as the optimized initialization parameters, and the classification model with the optimized initialization parameters is trained, Wherein, the sample is a sample image, the classification model is an image classification model, and the classification error is an image classification error; or, The sample is a sample text, the classification model is a text classification model, and the classification error is a text classification error; or The sample is a sample audio, the classification model is an audio classification model, and the classification error is an audio classification error; or The sample is a sample video, the classification model is a video classification model, and the classification error is a video classification error.

2. The training method according to claim 1, wherein: Determining the value of the first sub-function according to the modulus of the gradient vector corresponding to each sub-batch includes: An average modulus is determined according to the modulus of the gradient vector corresponding to each sub-batch, as the value of the first sub-function.

3. The training method according to claim 1, wherein: Determining the similarity of the gradient vectors corresponding to every two subbatches according to the gradient vector corresponding to each subbatch, and determining the value of the second subfunction according to the similarity of the gradient vectors corresponding to every two subbatches includes: Determine the cosine similarity of the gradient vectors corresponding to every two subbatches based on the gradient vector corresponding to each subbatches; An average cosine similarity is determined according to the cosine similarity of the gradient vectors corresponding to each two sub-batches, and is used as the value of the second sub-function.

4. The training method according to claim 1, wherein: The objective function further includes a condition that the maximum modulus of the moduli of the gradient vectors corresponding to each sub-batch does not exceed a preset value.

5. The training method according to claim 1, wherein: The objective function is determined by the following method: Performing an optimization surface analysis on the classification loss function of the classification model to determine an evaluation index function, wherein the evaluation index function is a function for determining an upper limit of the classification loss of the classification model; The objective function is determined according to the evaluation index function.

6. The training method according to claim 5, wherein: The evaluation index function includes a third sub-function for determining the maximum modulus of the modulus of the gradient vector corresponding to each sub-batch, a fourth sub-function for determining the similarity between the gradient vectors corresponding to each two sub-batches, and a fifth sub-function for determining the ratio of the maximum modulus to the minimum modulus of the optimized initialization parameters corresponding to each sub-batch. Determining the objective function according to the evaluation index function includes: Determining, according to the third sub-function, a condition that a maximum modulus of the moduli of the gradient vectors corresponding to each sub-batch in the objective function does not exceed a preset value; determining the second sub-function according to the fourth sub-function; The fifth sub-function is converted into the first sub-function by using the first-order gradient of the initialization parameter and the gradient vector corresponding to each sub-batch to approximate the optimized initialization parameter corresponding to each sub-batch.

7. The training method according to claim 3, wherein: The cosine similarity of the gradient vectors corresponding to each two sub-batches is the ratio of the dot product of the gradient vectors corresponding to each two sub-batches to the module length of the gradient vectors corresponding to each two sub-batches.

8. The training method according to claim 1, wherein: The adjusting the initialization parameters of the classification model according to the objective function includes: Determining a gradient value of the objective function relative to an adjustment vector of a previous cycle; Determining an adjustment vector for the current cycle based on the adjustment vector for the previous cycle, the gradient value, and the learning rate; The initialization parameters of the classification model in the current period are determined according to the adjustment vector of the current period and the initialization parameters of the classification model in the previous period.

9. The training method according to claim 8, wherein: The product of the adjustment vector of the current cycle and the initialization parameters of the classification model in the previous cycle is used as the initialization parameters of the classification model in the current cycle.

10. The training method according to claim 1, wherein: The average gradient vector of the classification error corresponding to each sample in each subbatch relative to the gradient vector of the initialization parameter of the classification model is used as the gradient vector of the classification error corresponding to each subbatch relative to the initialization parameter of the classification model.

11. The training method according to claim 1, wherein: The step of dividing the training samples into multiple sub-batches includes: The number of samples in each sub-batch is determined according to the number of samples, the number of sub-batches, and the overlap rate between sub-batches.

12. The training method according to claim 1, wherein: The training of the classification model with the optimized initialization parameters includes: Inputting the sample into the classification model with the optimized initialization parameters to obtain a classification result; Determining a classification loss based on the classification result and the labeling information of the training samples; According to the classification loss and the classification loss function, the parameters of the classification model are adjusted until the training is completed.

13. The training method according to claim 1, wherein: The classification model is an image classification model, Dividing the sample images for training into a plurality of sub-batches, wherein each sub-batch includes one or more sample images; Inputting the sample images of each subbatch into the image classification model, and determining the gradient vector of the image classification error corresponding to each subbatch relative to the initialization parameters of the image classification model; determining a value of an objective function according to the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the image classification loss of the image classification model; When the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the image classification model according to the objective function; Until the value of the objective function reaches a maximum value, the initialization parameters of the image classification model are used as optimized initialization parameters, and the image classification model with the optimized initialization parameters is trained.

14. The training method according to claim 1, wherein: The classification model is a text classification model, Dividing the sample text for training into multiple sub-batches, wherein each sub-batch includes one or more sample texts; Inputting the sample text of each subbatch into a text classification model, and determining a gradient vector of the text classification error corresponding to each subbatch relative to an initialization parameter of the text classification model; Determining a value of an objective function according to the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the text classification loss of the text classification model; When the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the text classification model according to the objective function; Until the value of the objective function reaches a maximum value, the initialization parameters of the text classification model are used as optimized initialization parameters, and the text classification model with the optimized initialization parameters is trained.

15. The training method according to claim 1, wherein: The classification model is an audio classification model, Dividing the sample audio for training into multiple sub-batches, wherein each sub-batch includes one or more sample audios; Inputting the sample audio of each subbatch into the audio classification model, and determining the gradient vector of the audio classification error corresponding to each subbatch relative to the initialization parameters of the audio classification model; determining a value of an objective function according to the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with the audio classification loss of the audio classification model; When the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the audio classification model according to the objective function; Until the value of the objective function reaches a maximum value, the initialization parameters of the audio classification model are used as optimized initialization parameters, and the audio classification model with the optimized initialization parameters is trained.

16. The training method according to claim 1, wherein: The classification model is a video classification model, Dividing the sample videos for training into multiple sub-batches, wherein each sub-batch includes one or more sample videos; Inputting the sample videos of each subbatch into a video classification model, and determining a gradient vector of the video classification error corresponding to each subbatch relative to an initialization parameter of the video classification model; Determining a value of an objective function according to the gradient vector corresponding to each sub-batch, wherein the objective function is negatively correlated with a video classification loss of the video classification model; When the value of the objective function does not reach a maximum value, adjusting the initialization parameters of the video classification model according to the objective function; Until the value of the objective function reaches a maximum value, the initialization parameters of the video classification model are used as optimized initialization parameters, and the video classification model with the optimized initialization parameters is trained.

17. The training method according to any one of claims 1 to 12, further comprising: Inputting the object to be classified into the trained classification model to obtain the category of the object to be classified; The objects to be classified include images, texts, videos or audios.

18. A training device for a classification model, comprising: A partitioning module, configured to partition the samples for training into a plurality of sub-batches, wherein each sub-batch includes one or more samples; A first determination module is configured to input samples of each subbatch into a classification model and determine a gradient vector of the classification error corresponding to each subbatch relative to an initialization parameter of the classification model; a second determination module, configured to determine a value of an objective function based on the gradient vector corresponding to each subbatch, wherein the objective function is negatively correlated with the classification loss of the classification model; the objective function includes at least one of a first subfunction and a second subfunction, and determining the value of the objective function includes at least one of the following: determining a modulus of the gradient vector corresponding to each subbatch based on the gradient vector corresponding to each subbatch, and determining the value of the first subfunction based on the modulus of the gradient vector corresponding to each subbatch; determining a similarity between the gradient vectors corresponding to every two subbatches based on the gradient vector corresponding to each subbatch, and determining the value of the second subfunction based on the similarity of the gradient vectors corresponding to every two subbatches; an adjustment module, configured to adjust the initialization parameters of the classification model according to the objective function when the value of the objective function has not reached a maximum value; a training module, configured to train the classification model with the optimized initialization parameters using the initialization parameters of the classification model until the value of the objective function reaches a maximum value; Wherein, the sample is a sample image, the classification model is an image classification model, and the classification error is an image classification error; or, The sample is a sample text, the classification model is a text classification model, and the classification error is a text classification error; or The sample is a sample audio, the classification model is an audio classification model, and the classification error is an audio classification error; or The sample is a sample video, the classification model is a video classification model, and the classification error is a video classification error.

19. A training device for a classification model, comprising: processor; as well as A memory coupled to the processor, for storing instructions, wherein when the instructions are executed by the processor, the processor executes the training method of the classification model as described in any one of claims 1-17.

20. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

Images