A model training method, a task execution method and device
By adjusting the weights of training samples and auxiliary features, the target features are determined and the prediction model is trained, thus solving the problem of insufficient accuracy of cue learning methods and achieving higher prediction accuracy and efficiency.
Patent Information
- Application Number
- CN202210494660.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-07
- Publication Date
- 2025-11-28
- Estimated Expiration
- 2042-05-07
AI Technical Summary
In existing technologies, cue learning methods have limited effectiveness in improving the accuracy of prediction models and cannot meet the requirements for performing target tasks.
By acquiring training samples and specifying auxiliary features, adjusting their initial weights to determine target features, and training the prediction model based on the target features, the prediction probability deviation is ensured to be less than the preset deviation, and features that have a negative impact on the prediction results are removed.
It improves the accuracy of prediction results of the prediction model, reduces computation and storage costs, and enhances the efficiency of model training and task execution.
Smart Images
Figure CN114997277B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present specification relates to the technical field of computer technology, and particularly relates to a model training method and a task execution method and device. BACKGROUND
[0002] At present, prompt learning can add additional prompt text in text data before inputting the text data into a pre-trained prediction model, which improves the accuracy of the prediction result of the pre-trained prediction model to a certain extent. However, the accuracy of the prediction result improved in this way is limited and cannot meet the demand of executing a target task.
[0003] Therefore, how to improve the accuracy of the prediction result determined by the prediction model is a problem to be solved. SUMMARY
[0004] The present specification provides a model training method, a task execution method and device to partially solve the above problems existing in the prior art.
[0005] The present specification adopts the following technical solutions:
[0006] The present specification provides a model training method, comprising:
[0007] obtaining a training sample;
[0008] inputting the training sample and the determined specified auxiliary features into the prediction model to determine, according to the initial weights corresponding to the specified auxiliary features, a prediction probability of the training sample under the influence of the specified auxiliary features as a first reference probability;
[0009] adjusting the initial weights corresponding to the specified auxiliary features under the constraint condition that the deviation between the prediction probability obtained by inputting the training sample and the specified auxiliary features with adjusted initial weights into the prediction model and the first reference probability is less than a preset deviation;
[0010] determining a target feature from the specified auxiliary features according to the adjusted weights corresponding to the specified auxiliary features, and training the prediction model based on the target feature.
[0011] Optionally, the specified auxiliary features are determined, specifically comprising:
[0012] obtaining each initial auxiliary feature;
[0013] inputting the training sample and each initial auxiliary feature into the prediction model to determine a prediction probability of the training sample under the influence of each initial auxiliary feature;
[0014] Adjust the initial auxiliary features to determine the specified auxiliary features, with an optimization objective of minimizing deviation between predicted probabilities of the training samples and labels corresponding to the training samples.
[0015] Optionally, the target feature is determined from the specified auxiliary features according to the adjusted weights corresponding to the specified auxiliary features, and specifically includes:
[0016] According to the initial weights corresponding to the specified auxiliary features and the adjusted weights corresponding to the specified auxiliary features, a weight change degree corresponding to each specified auxiliary feature is determined.
[0017] The target feature is determined from the specified auxiliary features according to the weight change degree corresponding to each specified auxiliary feature.
[0018] Optionally, the specified auxiliary features include sub-features of each dimension.
[0019] Before training the prediction model based on the target feature, the method further includes:
[0020] The training samples and the target feature are input into the prediction model to determine, as a second benchmark probability, a predicted probability of the training samples corresponding to the target feature under the influence of the sub-features of each dimension of the target feature according to the initial weights corresponding to the sub-features of each dimension of the target feature.
[0021] The initial weights corresponding to the sub-features of each dimension of the target feature are adjusted with a constraint condition that deviation between a predicted probability obtained by inputting the training samples and the sub-features of each dimension of the target feature with the adjusted initial weights into the prediction model and the second benchmark probability is less than a preset deviation.
[0022] The filtered sub-features are determined from the sub-features of each dimension of the target feature according to the adjusted weights corresponding to the sub-features of each dimension of the target feature, and the prediction model is trained based on the filtered sub-features.
[0023] Optionally, the filtered sub-features are determined from the sub-features of each dimension of the target feature according to the adjusted weights corresponding to the sub-features of each dimension of the target feature, and specifically includes:
[0024] According to the initial weights corresponding to the sub-features of each dimension of the target feature and the adjusted weights corresponding to the sub-features of each dimension of the target feature, a weight change degree corresponding to the sub-features of each dimension of the target feature is determined.
[0025] The filtered sub-features are determined from the sub-features of each dimension of the target feature according to the weight change degree corresponding to the sub-features of each dimension of the target feature.
[0026] Optionally, the method further includes:
[0027] obtaining a specified auxiliary feature corresponding to the target feature, the target feature being different from a task corresponding to the training sample;
[0028] training a model corresponding to the target task according to the target feature and other specified auxiliary features except the specified auxiliary feature corresponding to the target feature.
[0029] Optionally, the prediction model is trained based on the target feature, and specifically includes:
[0030] obtaining an initial auxiliary feature corresponding to the target feature;
[0031] training the prediction model according to the initial auxiliary feature corresponding to the target feature.
[0032] The specification provides a task execution method, including:
[0033] obtaining task data corresponding to a target task and a target feature corresponding to the target task, the target feature being obtained by the above model training method;
[0034] inputting the task data corresponding to the target task and the target feature corresponding to the target task into a pre-trained prediction model to determine a prediction result corresponding to the target task;
[0035] executing the target task according to the prediction result corresponding to the target task.
[0036] The specification provides a model training device, including:
[0037] an obtaining module configured to obtain a training sample;
[0038] an input module configured to input the training sample and each specified auxiliary feature determined into a prediction model to determine, as a first reference probability, a prediction probability of the training sample under the influence of each specified auxiliary feature according to an initial weight corresponding to each specified auxiliary feature;
[0039] an adjusting module configured to adjust the initial weight corresponding to each specified auxiliary feature with a constraint condition that a deviation between a prediction probability obtained by inputting the training sample and each specified auxiliary feature with the adjusted initial weight into the prediction model and the first reference probability is less than a preset deviation;
[0040] a training module configured to determine a target feature from each specified auxiliary feature according to the adjusted weight corresponding to each specified auxiliary feature, and train the prediction model based on the target feature.
[0041] The specification provides a task execution device, including:
[0042] The acquisition module is configured to acquire task data corresponding to a target task and a target feature corresponding to the target task, wherein the target feature is obtained by the method for training the model.
[0043] The determination module is configured to input the task data corresponding to the target task and the target feature corresponding to the target task into a pre-trained prediction model, and determine a prediction result corresponding to the target task.
[0044] The execution module is configured to execute the target task according to the prediction result corresponding to the target task.
[0045] The present specification provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for training the model and the method for executing the task.
[0046] The present specification provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for training the model and the method for executing the task when executing the program.
[0047] The above at least one technical solution adopted by the present specification can achieve the following beneficial effects:
[0048] In the method for training the model and the method for executing the task provided by the present specification, first, a training sample is acquired. Second, the training sample and each specified auxiliary feature determined in advance are input into a prediction model, so as to determine, according to an initial weight corresponding to each specified auxiliary feature, a prediction probability of the training sample under the influence of each specified auxiliary feature as a first reference probability. Then, the initial weight corresponding to each specified auxiliary feature is adjusted under the constraint condition that a deviation between a prediction probability obtained by inputting the training sample and each specified auxiliary feature with the adjusted initial weight into the prediction model and the first reference probability is less than a preset deviation. Finally, a target feature is determined from each specified auxiliary feature according to the adjusted weight corresponding to each specified auxiliary feature, and the prediction model is trained based on the target feature.
[0049] As can be seen from the above method, the initial weight corresponding to each specified auxiliary feature is adjusted under the constraint condition that a deviation between a prediction probability determined by the prediction model and a first reference probability is less than a preset deviation. The target feature having a positive influence on the prediction result of the prediction model is determined from each specified auxiliary feature according to the adjusted weight corresponding to each specified auxiliary feature, so as to improve the accuracy of the prediction result of the prediction model. BRIEF DESCRIPTION OF DRAWINGS
[0050] The accompanying drawings, which are included to provide a further understanding of the present description and constitute a part of the present description, illustrate the illustrative embodiments of the present description and the description thereof serve to explain the present description, and do not constitute an improper limitation on the present description. In the drawings:
[0051] Figure 1 A flowchart of a model training method in the present description;
[0052] Figure 2 A schematic diagram for determining a target feature provided by an embodiment of the present description;
[0053] Figure 3 A schematic diagram for determining a screened sub-feature provided by an embodiment of the present description;
[0054] Figure 4 A flowchart of a task execution method in the present description;
[0055] Figure 5 A schematic diagram of a model training device provided by the present description;
[0056] Figure 6 A schematic diagram of a task execution device provided by the present description;
[0057] Figure 7 A schematic diagram of an electronic device corresponding to Figure 1 provided by the present description. DETAILED DESCRIPTION
[0058] In order to make the purposes, technical solutions and advantages of the present description clearer, the technical solutions of the present description will be described in detail below with the specific embodiments of the present description and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present description, not all the embodiments. Based on the embodiments in the present description, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present description.
[0059] The technical solutions provided by the embodiments of the present description will be described in detail below with reference to the drawings.
[0060] Figure 1 A flowchart of a model training method in the present description, comprising the following steps:
[0061] S100: Obtain a training sample.
[0062] In the embodiments of the present description, the execution subject for training the prediction model can be a server, or an electronic device such as a desktop computer, etc. In order to facilitate the description, the training method of the prediction model provided by the present description will be described below with the server as the execution subject.
[0063] In the embodiments of the present specification, the server can obtain training samples. The training samples mentioned herein can refer to text data or image data.
[0064] S102: input the training samples and the predetermined specified auxiliary features into the prediction model to determine, according to the initial weights corresponding to the specified auxiliary features, the prediction probability of the training samples under the influence of the specified auxiliary features as the first reference probability.
[0065] S104: adjust the initial weights corresponding to the specified auxiliary features under the constraint condition that the deviation between the prediction probability obtained by inputting the training samples and the adjusted initial weights of the specified auxiliary features into the prediction model and the first reference probability is less than a preset deviation.
[0066] In actual applications, in order to apply the pre-trained prediction model to actual tasks, the model parameters of the prediction model need to be fine-tuned through a training sample set, which will bring huge calculation cost (all parameter gradient information and optimizer state information need to occupy a large amount of video memory). Moreover, the server needs to separately save the full model parameters corresponding to each actual task, which will result in high storage cost.
[0067] Therefore, the server can add additional prompt text in the training sample before inputting the training sample into the prediction model to improve the accuracy of the prediction result of the prediction model. For example, the actual task is a classification task to judge the sentiment classification (praise or derogatory) of "the Kung Pao chicken in this store tastes bad". The server can add ", the sentiment expressed by the comment is" in the text "the Kung Pao chicken in this store tastes so-so" and input it into the prediction model to obtain the prediction result of the training sample ("the Kung Pao chicken in this store tastes bad, the sentiment expressed by the comment is anger").
[0068] However, the accuracy of the prediction result improved by this method is limited. The server can add adjustable auxiliary features in the training sample before inputting the training sample into the prediction model to improve the accuracy of the prediction result of the prediction model. The auxiliary features mentioned in the present specification are not data such as specific words and images, but feature vectors. Therefore, the initial auxiliary features mentioned below can be understood as randomly generated feature vectors.
[0069] In the embodiments of the present specification, first, the server can obtain each initial auxiliary feature.
[0070] Secondly, the server can input the training samples and each initial auxiliary feature into the prediction model to determine the prediction probability of the training samples under the influence of each initial auxiliary feature.
[0071] Finally, the server can adjust each initial auxiliary feature to determine each specified auxiliary feature, with the optimization goal of minimizing the deviation between the prediction probability of the training sample and the label corresponding to the training sample. Thus, by adding each specified auxiliary feature to the training sample, the accuracy of the prediction result of the prediction model is improved. The specific formula is as follows:
[0072]
[0073] In the above formula, X e may be used to represent the feature vector corresponding to the training sample. P e may be used to represent the set of feature vectors corresponding to each specified auxiliary feature, where P e = {P1, P2, …, P m}, and m is the number of specified auxiliary features. Y may be used to represent the label corresponding to the training sample. p(Y|[P e ; X e ]) may be used to represent the prediction probability that the prediction result corresponding to the training sample belongs to the label Y.
[0074] In actual applications, not all specified auxiliary features in each specified auxiliary feature have a positive impact on the prediction result of the prediction model, and some specified auxiliary features may have a negative impact on the prediction result of the prediction model, thereby reducing the accuracy of the prediction result of the prediction model. Based on this, the server can determine the impact of each specified auxiliary feature on the prediction result of the prediction model by adjusting the weight corresponding to each specified auxiliary feature.
[0075] In the embodiments of the present specification, the server can input the training sample and the predetermined specified auxiliary features into the prediction model to determine, according to the initial weight corresponding to each specified auxiliary feature, the prediction probability of the training sample under the influence of each specified auxiliary feature as the first reference probability.
[0076] Wherein, one specified auxiliary feature corresponds to one weight, the server can determine the spliced specified auxiliary features according to each specified auxiliary feature and the weight corresponding to each specified auxiliary feature. The specific formula is as follows:
[0077]
[0078] In the above formula, p e = {p1, p2, …, p m}. P m may be used to represent the mth specified auxiliary feature. γ = {γ1, γ2, …, γ m}, γ i ∈ {0, 1}, and γ may be used to represent the set of weights corresponding to each specified auxiliary feature.m The weight corresponding to the mth specified auxiliary feature can be used to represent. The specified auxiliary features after splicing can be used to represent. Based on this, the server can input the training sample and the specified auxiliary features after splicing into the prediction model to determine the prediction probability corresponding to the training sample.
[0079] From the above, the weight corresponding to each specified auxiliary feature is actually used to splice the specified auxiliary features. The splicing here can also be regarded as fusing the specified auxiliary features to obtain an auxiliary feature. Therefore, the weight corresponding to each specified auxiliary feature can also be understood as indicating how much each specified auxiliary feature needs to contribute to the final auxiliary feature in the process of fusing the specified auxiliary features.
[0080] Then, the server can adjust the initial weight corresponding to each specified auxiliary feature under the constraint that the deviation between the prediction probability obtained by inputting the training sample and the adjusted initial weight of each specified auxiliary feature into the prediction model and the first reference probability is less than a preset deviation.
[0081] That is, the server adjusts the initial weight corresponding to each specified auxiliary feature in the process of adjusting the initial weight corresponding to each specified auxiliary feature, so as to determine the adjusted initial weight of each specified auxiliary feature.
[0082] Specifically, when adjusting the initial weight of each specified auxiliary feature, the deviation between the prediction probability determined by inputting the specified auxiliary feature with the adjusted weight and the prediction probability determined by inputting the specified auxiliary feature with the unadjusted weight should not be too large, that is, the probability deviation between the two should be maintained within a small range (even equal). Adjusting the initial weight of each specified auxiliary feature under this constraint condition will inevitably increase the initial weight of some specified auxiliary features and decrease the initial weight of some specified auxiliary features. After several rounds of adjustment, the specified auxiliary feature with a greater impact on the final result, that is, the target feature, can be determined through the adjusted initial weight.
[0083] S106: Determine the target feature from the specified auxiliary features according to the adjusted weight corresponding to each specified auxiliary feature, and train the prediction model based on the target feature.
[0084] In actual applications, some specified auxiliary features have little positive impact on the prediction result of the prediction model, or even have negative impact on the prediction result of the prediction model, and reduce the accuracy of the prediction result of the prediction model. Based on this, the server can determine the specified auxiliary features that have little positive impact on the prediction model or have negative impact on the prediction model, and remove the specified auxiliary features to improve the accuracy of the prediction result of the prediction model.
[0085] In the embodiments of the present specification, the server can determine a target feature from the specified auxiliary features according to the adjusted weight corresponding to each specified auxiliary feature, and train the prediction model based on the target feature.
[0086] Specifically, the server can determine the weight change degree of each specified auxiliary feature according to the initial weight corresponding to each specified auxiliary feature and the adjusted weight corresponding to each specified auxiliary feature.
[0087] Secondly, the server can determine a target feature from the specified auxiliary features according to the weight change degree of each specified auxiliary feature.
[0088] Further, the server can determine the importance of each specified auxiliary feature according to the weight change degree of each specified auxiliary feature, and determine a target feature from the specified auxiliary features according to the importance of each specified auxiliary feature. The specific formula is as follows:
[0089]
[0090] In the above formula, which can be used to represent the importance of the i th specified auxiliary feature (i.e. the importance determined by the weight change degree). L(x) can be used to represent the prediction probability that the prediction result corresponding to the training sample belongs to the label Y. γ i which can be used to represent the weight of the i th specified auxiliary feature. D x which can be used to represent the distribution of the training sample.
[0091] As can be seen from the above formula, under the condition that the deviation between the prediction probability obtained by inputting the training sample and the adjusted initial weight of each specified auxiliary feature into the prediction model and the first reference probability is less than the preset deviation, for each specified auxiliary feature, the greater the weight change degree of the specified auxiliary feature, the greater the gradient change of the specified auxiliary feature. Since the greater the gradient change of the specified auxiliary feature, the greater the impact of the specified auxiliary feature on the prediction result determined by the prediction model, that is, the higher the importance of the specified auxiliary feature. Therefore, the greater the weight change degree of the specified auxiliary feature, the higher the importance of the specified auxiliary feature.
[0092] In the embodiments of this specification, the server can determine the target feature in several ways. For example, the server can remove specified auxiliary features whose importance is less than a set importance threshold from among the specified auxiliary features to determine the target feature. Another example is that the server can sort the importance of each specified auxiliary feature and remove the specified auxiliary features whose sorting number is less than a set number threshold from among the specified auxiliary features to determine the target feature. Specifically, as follows... Figure 2 As shown.
[0093] Figure 2 This is a schematic diagram illustrating the determination of target features as provided in the embodiments of this specification.
[0094] To illustrate this process more vividly, in Figure 2 In this diagram, a rectangle represents a specified auxiliary feature (in practical applications, the actual form of an auxiliary feature is a feature vector, not a specific shape; this is just a shape representation for ease of understanding). Correspondingly, dashed rectangles represent specified auxiliary features that have been removed. The server can reset the weights of specified auxiliary features with importance less than a set importance threshold to 0, thus removing these auxiliary features. Similarly, the server can also reset the weights of specified auxiliary features with sorting numbers less than a set number threshold to 0, thus removing these auxiliary features.
[0095] In practical applications, not all sub-features within each dimension of the specified auxiliary features will positively impact the prediction model's results. Some sub-features may negatively affect the prediction model's accuracy. Therefore, the server can identify and remove sub-features with minimal positive impact or negative impact on the prediction model to improve its accuracy.
[0096] In the embodiments described in this specification, the auxiliary features are specified to include sub-features of each dimension. After the target feature is determined, the server can input the training samples and the target feature into the prediction model to determine the prediction probability of the training samples under the influence of the target feature based on the initial weights corresponding to the sub-features of each dimension of the target feature, which serves as the second baseline probability.
[0097] In this system, each sub-feature of a dimension corresponds to a weight. The server can determine the concatenated sub-features of each dimension based on the sub-features of each dimension and their corresponding weights. The specific formula is as follows:
[0098]
[0099] In the above formula, qe = {q1, q2, …, q m}. q m The δ can be used to characterize the sub-features of the mth dimension. δ = {δ1, δ2, …, δ m}, δ i ∈ {0, 1}, and the δ can be used to characterize the set of weights corresponding to the sub-features of each dimension. δ m The δ can be used to characterize the weight corresponding to the sub-features of the mth dimension. The δ can be used to characterize the sub-features of each dimension after splicing. Based on this, the server can input the training sample and the sub-features of each dimension of the spliced target feature into the prediction model to determine the prediction probability corresponding to the training sample.
[0100] Secondly, the server can adjust the initial weight of the sub-features of each dimension of the target feature, with the constraint that the deviation between the prediction probability obtained by inputting the training sample and the sub-features of each dimension of the adjusted initial weight into the prediction model and the second benchmark probability is less than the preset deviation.
[0101] That is, the server adjusts the initial weight of the sub-features of each dimension of the target feature in the process of adjusting the initial weight of the sub-features of each dimension of the target feature, and adjusts the initial weight of the sub-features of each dimension of the target feature to determine the sub-features of each dimension of the target feature after adjusting the initial weight, with the goal that the deviation between the prediction probability corresponding to the training sample and the second benchmark probability is less than the preset deviation.
[0102] Finally, the server can determine the screened sub-features from the sub-features of each dimension of the target feature according to the adjusted weight corresponding to the sub-features of each dimension of the target feature, and train the prediction model based on the screened sub-features.
[0103] Specifically, the server can determine the weight change degree corresponding to the sub-features of each dimension of the target feature according to the initial weight corresponding to the sub-features of each dimension of the target feature and the adjusted weight corresponding to the sub-features of each dimension of the target feature.
[0104] The server can determine the screened sub-features from the sub-features of each dimension of the target feature according to the weight change degree corresponding to the sub-features of each dimension of the target feature.
[0105] Further, the server can determine the importance degree corresponding to the sub-features of each dimension of the target feature according to the weight change degree corresponding to the sub-features of each dimension of the target feature, and determine the screened sub-features from the sub-features of each dimension of the target feature according to the importance degree corresponding to the sub-features of each dimension of the target feature. The specific formula is as follows:
[0106]
[0107] In the above formula, may be used to represent the importance degree corresponding to the i-th dimension sub-feature. L(x) can be used to represent the prediction probability that the prediction result corresponding to the training sample belongs to the label Y. δ i may be used to represent the weight corresponding to the i-th dimension sub-feature. D x may be used to represent the distribution of the training sample.
[0108] As can be seen from the above formula, in the case of taking the deviation between the prediction probability obtained by inputting the training sample and the adjusted initial weight of each specified auxiliary feature into the prediction model and the second reference probability being less than the preset deviation as a constraint condition, for each dimension sub-feature in the target feature, the greater the weight variation degree corresponding to the dimension sub-feature, the greater the gradient variation of the dimension sub-feature. Since the greater the gradient variation of the dimension sub-feature, the greater the influence of the dimension sub-feature on the prediction result determined by the prediction model. Therefore, the greater the weight variation degree corresponding to the dimension sub-feature, the higher the importance degree corresponding to the dimension sub-feature.
[0109] In the embodiments of the present specification, the server can determine the screened sub-feature in various ways, for example, the server can remove the sub-feature with an importance degree less than a set importance threshold from the sub-feature of each dimension of the target feature to determine the screened sub-feature. For another example, the server can sort the importance degrees corresponding to the sub-features of each dimension of the target feature, remove the sub-feature with a sorting serial number less than a set serial number threshold from the sub-features of each dimension of the target feature, and determine the screened sub-feature. For example, as shown in the following table. Figure 3
[0110] Figure 3 The table provides an example of determining the screened sub-feature provided by the embodiments of the present specification.
[0111] In the table, Figure 3 a rectangle is used to represent a specified auxiliary feature, and a dashed box rectangle is used to represent a removed specified auxiliary feature. A square is used to represent a dimension sub-feature in the specified auxiliary feature. A dashed box square is used to represent a removed dimension sub-feature. The server can set the weight corresponding to the sub-feature with an importance degree less than a set importance threshold to 0, and remove this part of the sub-feature. Of course, the server can also set the weight corresponding to the sub-feature with a sorting serial number less than a set serial number threshold to 0, and remove this part of the sub-feature.
[0112] In the table, a square can also be used to represent a set of a set number of dimension sub-features in the specified auxiliary feature.
[0113] Of course, since the specified auxiliary feature includes sub-features of each dimension, the server can also input the training sample and the predetermined specified auxiliary feature into the prediction model to determine, according to the initial weight corresponding to the sub-feature of each dimension of each specified auxiliary feature, the prediction probability corresponding to the training sample under the influence of the sub-feature of each dimension of each specified auxiliary feature as the first reference probability.
[0114] With the constraint that the deviation between the prediction probability obtained by inputting the training sample and the sub-feature of each dimension of each specified auxiliary feature after adjusting the initial weight into the prediction model and the first reference probability is less than the preset deviation, the initial weight corresponding to the sub-feature of each dimension of each specified auxiliary feature is adjusted.
[0115] According to the adjusted weight corresponding to the sub-feature of each dimension of each specified auxiliary feature, the target sub-feature is determined from the sub-feature of each dimension of each specified auxiliary feature, and the prediction model is trained based on the target sub-feature.
[0116] In practical applications, lottery hypothesis theory refers to the existence of a part of sub-networks in a randomly initialized model. If only the part of sub-networks is retrained, the same model performance as the trained model can be achieved within a similar number of iterations, or even higher model performance than the trained model. The parameters of this part of sub-networks need to be the same as the parameters of this part of sub-networks in the model before training.
[0117] And when the lottery hypothesis theory is applied to the embodiments of the present specification, after determining the target feature from each specified auxiliary feature, the server can obtain the initial auxiliary feature corresponding to the target feature, and train the prediction model according to the initial auxiliary feature corresponding to the target feature.
[0118] Specifically, the server can obtain the initial auxiliary feature corresponding to the target feature. Here, the initial auxiliary feature refers to the specified auxiliary feature before training.
[0119] Secondly, the server can input the training sample and the initial auxiliary feature corresponding to the target feature into the prediction model to determine the prediction probability corresponding to the training sample under the influence of the initial auxiliary feature corresponding to the target feature.
[0120] Finally, the server can adjust the initial auxiliary feature corresponding to the target feature to determine the re-adjusted target feature, with the optimization goal of minimizing the deviation between the prediction probability of the training sample and the label corresponding to the training sample. The server applies the re-adjusted target feature to the target task, which can improve the accuracy of the prediction model in predicting the training sample.
[0121] In the embodiments of the present disclosure, the server can remove different numbers of specified auxiliary features from the specified auxiliary features and remove different numbers of sub-features of each dimension of the specified auxiliary features to obtain target features with different compression ratios. The compression ratio mentioned herein can refer to the ratio of the number of removed specified auxiliary features to the total number of specified auxiliary features, and the ratio of the number of removed sub-features of each dimension of the specified auxiliary features to the total number of sub-features of each dimension. For the target feature corresponding to each compression ratio, the server can input the training sample and the target feature corresponding to the compression ratio into the prediction model to determine the model score of the prediction model under the influence of the target feature corresponding to the compression ratio. The server can select the target feature corresponding to the compression ratio of the prediction model with the highest model score from the model scores of the prediction model under the influence of the target features corresponding to different compression ratios, for execution of the target task.
[0122] In actual applications, in order to improve the training speed of the model corresponding to different tasks and the accuracy of the calculation result, the server can initialize the target feature of the task corresponding to the training sample to the target task after determining the target feature of the task corresponding to the training sample, and improve the training speed of the model corresponding to the target task and the accuracy of the calculation result by adjusting the target feature in the model corresponding to the task corresponding to the training sample.
[0123] In the embodiments of the present disclosure, the server can obtain specified auxiliary features corresponding to the target task, which is different from the task corresponding to the training sample.
[0124] Then, the server can train the model corresponding to the target task according to the target feature and other specified auxiliary features except the specified auxiliary feature corresponding to the target feature.
[0125] It should be noted that the training process of the model corresponding to the target task mentioned herein can be the training process of the prediction model mentioned above, or the training process of other models.
[0126] As can be seen from the above process, the present method can adjust the initial weight corresponding to each specified auxiliary feature by taking the deviation between the prediction probability determined by the prediction model and the first reference probability being less than a preset deviation as a constraint condition. The target feature having a positive influence on the prediction result of the prediction model is determined from each specified auxiliary feature according to the adjusted weight corresponding to each specified auxiliary feature, so as to improve the accuracy of the prediction result of the prediction model.
[0127] After the training of the prediction model is completed, the server can execute the target task by using the prediction result of the target task determined by the prediction model according to the specific process of the present disclosure.Figure 4 As shown.
[0128] Figure 4 A flowchart of a method for performing a task in the present specification.
[0129] S400: Obtain task data corresponding to the target task and target features corresponding to the target task, wherein the target features are obtained by the method for training the model.
[0130] S402: Input the task data corresponding to the target task and the target features corresponding to the target task into the pre-trained prediction model to determine the prediction result corresponding to the target task.
[0131] S404: Perform the target task according to the prediction result corresponding to the target task.
[0132] In the embodiments of the present specification, the server can obtain task data corresponding to the target task and target features corresponding to the target task.
[0133] Secondly, the server can input the task data corresponding to the target task and the target features corresponding to the target task into the pre-trained prediction model to determine the prediction result corresponding to the target task.
[0134] Finally, the server can perform the target task according to the prediction result corresponding to the target task.
[0135] From the above, it can be seen that the server applies the target features obtained in this way to the actual business, which can effectively improve the accuracy of the prediction result determined by the prediction model.
[0136] The above is the method for training the model provided by one or more embodiments of the present specification. Based on the same idea, the present specification also provides a corresponding model training device, as shown in Figure 5 As shown.
[0137] Figure 5 A schematic diagram of a model training device provided by the present specification, comprising:
[0138] The acquisition module 500 is configured to acquire the training sample.
[0139] The input module 502 is configured to input the training sample and the determined specified auxiliary features into the prediction model, so as to determine, according to the initial weight corresponding to each specified auxiliary feature, the prediction probability of the training sample under the influence of each specified auxiliary feature as the first reference probability.
[0140] The adjusting module 504 is configured to adjust the initial weight corresponding to each specified auxiliary feature, with a constraint that a deviation between a prediction probability obtained by inputting the training sample and the adjusted initial weight of each specified auxiliary feature into the prediction model and a first benchmark probability is less than a preset deviation.
[0141] The training module 506 is configured to determine a target feature from the specified auxiliary features according to the adjusted weight corresponding to each specified auxiliary feature, and train the prediction model based on the target feature.
[0142] Optionally, the input module 502 is specifically configured to obtain each initial auxiliary feature, input the training sample and each initial auxiliary feature into the prediction model, determine a prediction probability corresponding to the training sample under the influence of each initial auxiliary feature, and adjust each initial auxiliary feature with an optimization target of minimizing a deviation between the prediction probability of the training sample and a label corresponding to the training sample, to determine each specified auxiliary feature.
[0143] Optionally, the training module 506 is specifically configured to determine a weight change degree corresponding to each specified auxiliary feature according to the initial weight corresponding to each specified auxiliary feature and the adjusted weight corresponding to each specified auxiliary feature, and determine a target feature from the specified auxiliary features according to the weight change degree corresponding to each specified auxiliary feature.
[0144] Optionally, the specified auxiliary feature includes a sub-feature of each dimension.
[0145] The adjusting module 504 is specifically further configured to input the training sample and the target feature into the prediction model, determine a prediction probability corresponding to the training sample under the influence of each sub-feature of each dimension of the target feature according to the initial weight corresponding to each sub-feature of each dimension of the target feature, as a second benchmark probability, adjust the initial weight corresponding to each sub-feature of each dimension of the target feature with a constraint that a deviation between a prediction probability obtained by inputting the training sample and the adjusted initial weight of each sub-feature of each dimension of the target feature into the prediction model and the second benchmark probability is less than a preset deviation, determine a screened sub-feature from each sub-feature of each dimension of the target feature according to the adjusted weight corresponding to each sub-feature of each dimension of the target feature, and train the prediction model based on the screened sub-feature.
[0146] Optionally, the adjusting module 504 is specifically further configured to determine a weight change degree corresponding to each sub-feature of each dimension of the target feature according to the initial weight corresponding to each sub-feature of each dimension of the target feature and the adjusted weight corresponding to each sub-feature of each dimension of the target feature, and determine a screened sub-feature from each sub-feature of each dimension of the target feature according to the weight change degree corresponding to each sub-feature of each dimension of the target feature.
[0147] Optionally, the training module 506 is further configured to obtain specified auxiliary features corresponding to the target task, wherein the target task is different from the task corresponding to the training sample, and to train the model corresponding to the target task based on the target features and other specified auxiliary features besides the specified auxiliary features corresponding to the target features.
[0148] Optionally, the training module 506 is further configured to obtain initial auxiliary features corresponding to the target features, and train the prediction model based on the initial auxiliary features corresponding to the target features.
[0149] Figure 6 A schematic diagram of a task execution device provided in this specification includes:
[0150] The acquisition module 600 is used to acquire the task data and target features corresponding to the target task, wherein the target features are obtained through the above-mentioned model training method.
[0151] The determining module 602 is used to input the task data corresponding to the target task and the target features corresponding to the target task into a pre-trained prediction model to determine the prediction result corresponding to the target task;
[0152] The execution module 604 is used to execute the target task based on the prediction result corresponding to the target task.
[0153] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This provides a method for model training and a method for task execution.
[0154] This instruction manual also provides Figure 7 One of the corresponding Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 7 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The methods for model training and task execution are described. Of course, besides software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.
[0155] It is specified that all the actions of acquisition of signals, information or data in this application are carried out in compliance with the corresponding data protection regulations policy of the country of residence and with the authorization given by the owner of the corresponding device.
[0156] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) such as a field programmable gate array (FPGA) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a piece of PLD by the designer programming it by himself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing a program, and the original code before compilation also has to be written in a specific programming language, which is called a hardware description language (HDL), and there are many types of HDL, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit that implements the logical method flow can be easily obtained.
[0157] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.
[0158] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0159] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of each unit can be implemented in one or more software and / or hardware in implementing the present specification.
[0160] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.
[0161] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.
[0162] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.
[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.
[0164] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0165] The memory can include non-persistent memory and / or storage mechanisms such as, for example, random access memory (RAM), non-volatile memory (NVM), and / or a persistent memory such as, for example, read-only memory (ROM) or flash memory. The memory is an example of computer-readable media.
[0166] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.
[0167] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.
[0168] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0169] The present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.
[0170] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different aspects of the description. For each embodiment, the description focuses on the differences from the other embodiments. In particular, the description of the system embodiments is relatively brief, as the system embodiments are largely analogous to the method embodiments. The relevant parts of the description of the method embodiments are referred to.
[0171] The above only describes the embodiments of the present specification and is not intended to limit the present specification. The present specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of claims of the present specification.
Claims
1. A method for model training, the method comprising: The method comprises the following steps: obtaining training samples, which are text data or image data; inputting the training samples and the determined specified auxiliary features into a prediction model to determine, according to initial weights corresponding to the specified auxiliary features, a prediction probability of the training samples under the influence of the specified auxiliary features as a first reference probability; adjusting the initial weights corresponding to the specified auxiliary features under the constraint that a deviation between a prediction probability obtained by inputting the training samples and the specified auxiliary features with adjusted initial weights into the prediction model and the first reference probability is less than a preset deviation; determining a target feature from the specified auxiliary features according to adjusted weights corresponding to the specified auxiliary features, and training the prediction model based on the target feature. The method comprises the following steps: obtaining initial auxiliary features, which are randomly generated feature vectors; inputting the training samples and the initial auxiliary features into the prediction model to determine a prediction probability of the training samples under the influence of the initial auxiliary features; adjusting the initial auxiliary features to minimize a deviation between the prediction probability of the training samples and a label corresponding to the training samples as an optimization target, and determining the specified auxiliary features.
2. The method of claim 1, wherein, The method comprises the following steps: determining a weight change degree corresponding to each specified auxiliary feature according to initial weights corresponding to the specified auxiliary features and adjusted weights corresponding to the specified auxiliary features; determining the target feature from the specified auxiliary features according to the weight change degree corresponding to each specified auxiliary feature.
3. The method of claim 1, wherein, The specified auxiliary features comprise sub-features of each dimension. Before training the prediction model based on the target feature, the method further comprises the following steps: inputting the training samples and the target feature into the prediction model to determine, according to initial weights corresponding to sub-features of each dimension of the target feature, a prediction probability of the training samples under the influence of the target feature as a second reference probability; adjusting the initial weights corresponding to the sub-features of each dimension of the target feature under the constraint that a deviation between a prediction probability obtained by inputting the training samples and the sub-features of each dimension of the target feature with adjusted initial weights into the prediction model and the second reference probability is less than a preset deviation; determining screened sub-features from the sub-features of each dimension of the target feature according to adjusted weights corresponding to the sub-features of each dimension of the target feature, and training the prediction model based on the screened sub-features.
4. The method of claim 3, wherein, The method comprises the following steps: determining a weight change degree corresponding to each sub-feature of each dimension of the target feature according to initial weights corresponding to the sub-features of each dimension of the target feature and adjusted weights corresponding to the sub-features of each dimension of the target feature; determining the screened sub-features from the sub-features of each dimension of the target feature according to the weight change degree corresponding to each sub-feature of each dimension of the target feature.
5. The method of claim 1, wherein, The method further comprises the following steps: obtaining a specified auxiliary feature corresponding to a target task, the target task being different from a task corresponding to the training sample; training a model corresponding to the target task according to the target feature and other specified auxiliary features except the specified auxiliary feature corresponding to the target feature.
6. The method of claim 1, wherein, training a prediction model based on a target feature, specifically including: obtaining an initial auxiliary feature corresponding to the target feature; training a prediction model according to the initial auxiliary feature corresponding to the target feature.
7. A method of task execution, characterized by, including: obtaining task data corresponding to a target task and a target feature corresponding to the target task, the target feature being obtained by the method in any one of claims 1-6; inputting the task data corresponding to the target task and the target feature corresponding to the target task into a pre-trained prediction model to determine a prediction result corresponding to the target task; performing the target task according to the prediction result corresponding to the target task.
8. An apparatus for model training, the apparatus comprising: including: an obtaining module, configured to obtain a training sample, the training sample being text data or image data; an inputting module, configured to input the training sample and determined specified auxiliary features into a prediction model to determine, as a first benchmark probability, a prediction probability of the training sample under the influence of the specified auxiliary features according to initial weights corresponding to the specified auxiliary features; an adjusting module, configured to adjust the initial weights corresponding to the specified auxiliary features with a constraint condition that a deviation between the prediction probability obtained by inputting the training sample and the adjusted specified auxiliary features into the prediction model and the first benchmark probability is less than a preset deviation; a training module, configured to determine a target feature from the specified auxiliary features according to the adjusted weights corresponding to the specified auxiliary features, and train a prediction model based on the target feature; determining the specified auxiliary features, specifically including: obtaining each initial auxiliary feature, the initial auxiliary feature being a randomly generated feature vector; inputting the training sample and each initial auxiliary feature into a prediction model to determine a prediction probability of the training sample under the influence of each initial auxiliary feature; adjusting each initial auxiliary feature to minimize a deviation between the prediction probability of the training sample and a label corresponding to the training sample as an optimization target, and determining the specified auxiliary features.
9. An apparatus for task execution, the apparatus comprising: including: an obtaining module, configured to obtain task data corresponding to a target task and a target feature corresponding to the target task, the target feature being obtained by the method in any one of claims 1-6; a determining module, configured to input the task data corresponding to the target task and the target feature corresponding to the target task into a pre-trained prediction model to determine a prediction result corresponding to the target task; an executing module, configured to perform the target task according to the prediction result corresponding to the target task.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6 or 7.
11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-6 or 7.