Evaluation method, model training method and related equipment

Abstract

The invention discloses an evaluation and model training method and device, computer equipment and a storage medium. The method comprises the steps of obtaining a first training sample; processing at least one of the training data and the label of the first training sample according to a sample expansion strategy to obtain a second training sample; training the first model by using a second training sample to obtain a second model; processing the training data according to a second model to obtain a prediction label; and determining influence information of the sample expansion strategy on the first model according to the prediction label and the label of the training data. According to the method and the device, the sample expansion strategy most suitable for model training can be selected from the model training by comparing the influence information of different sample expansion strategies on the model training, so that the reasonable sample expansion strategy can be used to improve the training effect of the model when the training samples are insufficient in the subsequent model training process.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an evaluation method, a model training method, an apparatus, a computer device, a computer storage medium, and a computer program product. Background Art

[0002] Currently, more and more manufacturers choose to use machine learning to train models for different purposes in order to provide better services to users. During the model training process, it is often necessary to collect labeled training data in advance to train a more accurate model.

[0003] However, in actual applications, labeling training data usually requires a lot of manpower costs. Therefore, in many scenarios where it is inconvenient to collect labeled training data, people often choose to perform certain transformations on the training data to expand the training data. However, due to the lack of relevant experience, if inappropriate changes are used when expanding the training data, it will affect the training accuracy of subsequent models. Summary of the Invention

[0004] The embodiments of the present application provide an evaluation method, a model training method, an apparatus, a computer device, a computer storage medium, and a computer program product.

[0005] In a first aspect, an embodiment of the present application provides an evaluation method, comprising:

[0006] Obtain a first training sample; the first training sample includes training data and a label of the training data;

[0007] Processing at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample;

[0008] Training the first model using the second training sample to obtain a second model;

[0009] Processing the training data in the first training sample according to the second model to obtain a predicted label;

[0010] Determine the impact of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample

[0011] In a second aspect, an embodiment of the present application provides a model training method, comprising:

[0012] Acquire an initial model and an initial sample for training the initial model; the initial sample includes initial data and a label of the initial data;

[0013] According to the processing task type of the initial model, a target sample expansion strategy corresponding to the processing task type is read from a preset database;

[0014] Processing at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample;

[0015] The initial model is trained according to the target sample to obtain a trained model.

[0016] In a third aspect, an embodiment of the present application provides an evaluation device, comprising:

[0017] An acquisition module, configured to acquire a first training sample; the first training sample includes training data and a label of the training data;

[0018] an expansion module, configured to process at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample;

[0019] A first training module, configured to train the first model using the second training sample to obtain a second model;

[0020] a processing module, configured to process the training data in the first training sample according to the second model to obtain a predicted label;

[0021] A determination module is used to determine the impact information of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample.

[0022] In a fourth aspect, an embodiment of the present application provides a model training device, comprising:

[0023] A second acquisition module is used to acquire an initial model and an initial sample for training the initial model; the initial sample includes initial data and a label of the initial data;

[0024] A reading module is used to read the target sample expansion strategy corresponding to the processing task type from a preset database according to the processing task type of the initial model;

[0025] a transformation module, configured to process at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample;

[0026] The second training module is used to train the initial model according to the target sample to obtain a trained model.

[0027] In a fifth aspect, an embodiment of the present application also provides a computer device comprising a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the evaluation method or the steps of the model training method as described above.

[0028] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for loading by a processor to execute the steps of the evaluation method or the steps of the model training method as described above.

[0029] According to the solution of the embodiment of the present application, after obtaining the first training sample, at least one of the training data and the label of the training data in the first training sample will be processed according to the sample expansion strategy to obtain a second training sample, and then the training data in the first training sample will be verified using the second model trained by the second training sample to obtain a predicted label, wherein the label of the training data can be regarded as the true label, and the predicted label can be understood as the verification information of the model trained by the training sample obtained after processing by the sample expansion strategy. Therefore, based on these two, it can be determined whether the second model trained after processing by the sample expansion strategy has a higher or worse prediction effect, thereby determining the impact information of the sample expansion strategy on the first model, so that the most suitable sample expansion strategy for model training can be selected by comparing the impact information of different sample expansion strategies on model training. Therefore, when the training samples are insufficient in the subsequent model training process, a reasonable sample expansion strategy can be used to improve the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A schematic diagram of a step flow of an evaluation method provided in an embodiment of the present application;

[0032] Figure 2 A schematic diagram of a flow chart of steps for expanding training samples provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a process flow for determining the impact of different strategies on model training results based on label differences provided in an embodiment of the present application;

[0034] Figure 4A schematic diagram of another step flow for determining the impact of different strategies on model training results based on label differences provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of a process flow for associating and storing a sample expansion strategy with a processing task type provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of the steps of a training model provided in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of the structure of an evaluation device provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0039] Figure 9 It is a schematic diagram of the internal structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0041] In order to better understand the evaluation and model training methods, devices, computer equipment and storage media provided by this application, the relevant application scenarios of this application are first explained. Specifically, the evaluation and model training methods provided by this application are mainly applied to the model training scenarios of machine learning. Among them, in related technologies, many manufacturers will choose to use machine learning to train models for different purposes to provide better services to users, such as training models for realizing human-computer dialogue for artificial intelligence customer service, training models for text and image recognition to assist users in retrieval, or training models for product ratings to recommend products to users, etc., especially various pre-trained models have been launched. Manufacturers only need to collect appropriate labeled data according to their own needs to fine-tune the pre-trained model to obtain models suitable for various scenarios and tasks.

[0042] However, the ultimate effectiveness of a model's application often depends on labeled data. The more accurate and comprehensive the labeled data, the better the model's performance. However, labeling data often requires significant labor. Therefore, in many scenarios where collecting labeled training data is inconvenient, users often choose to transform the training data, such as replacing it with similar training data to expand it. However, due to a lack of experience, inappropriate changes to the training data when expanding the training data can affect the subsequent model training results.

[0043] It is precisely to solve the above problems that the present application provides an evaluation and model training method, device, computer equipment and storage medium. By evaluating and recording the impact of sample data obtained under different change strategies on the model effect, it can be used as relevant experience to indicate the subsequent expansion of training data samples in similar model training processes, and can effectively improve the training effect of the model when the training sample data is insufficient. Specifically, the evaluation method and the model training method are usually set in the model evaluation device and the model training device in the form of a computer program. The model evaluation device and the model training device are usually set in the form of a processor in the computer device. The evaluation device in the computer device executes the computer program corresponding to the model evaluation method to execute the evaluation method provided by the present application. The model training device in the computer device executes the computer program corresponding to the model training method to execute the model training method provided by the present application. The following will be explained in detail.

[0044] like Figure 1 As shown, Figure 1 A schematic flow chart of the steps of an evaluation method provided in an embodiment of the present application, specifically including steps S110 to S150:

[0045] S110 , obtaining a first training sample; the first training sample includes training data and a label of the training data.

[0046] In the embodiments of the present application, the first training sample generally refers to a type of sample data that can be used to train a model. It can generally be obtained through network technology, such as crawler technology, or read from a training database. Generally, the training sample generally includes training data and labels for the training data. Specifically, in different application scenarios, that is, in scenarios for training different models, the data type of the first training sample generally varies. For example, taking the aforementioned model for training a human-computer dialogue as an example, the first training sample at this time generally includes different user conversation texts as training data, and the response text corresponding to the conversation text as corresponding labels, such as the training data including texts such as "How should I get to this place?" and "What should I do next?" Alternatively, in the scenario of training a model for image recognition, the first training sample at this time generally includes image data from different sources as training data, and the subject information contained in the image as corresponding labels. In the scenario of training a model for product rating recommendation, the first training sample at this time generally includes different product information, such as product introduction text, product images, etc. as training samples, and the corresponding labels can be used to indicate the user's willingness to purchase such products, etc. Of course, the training samples provided above are only for illustration. In fact, the training data in the training samples can also be videos, audios, etc. in addition to text and images, and their corresponding labels can also be pre-annotated based on actual needs. The embodiment of the present application does not limit the specific data form of the first training sample.

[0047] Of course, it should be noted that since the labels provided above usually need to be manually labeled in advance, in some scenarios where it is not convenient to collect labeled training data, for example, in scenarios involving processing of privacy information such as ID images, the labels of the training data in the first training sample may be missing or incorrect, or the collected training data samples may be insufficient, resulting in the trained model effect being less than ideal.

[0048] S120 : Process at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample.

[0049] In this embodiment, after the training data and the labels of the training data are obtained as mentioned above, at least one of the training data and the labels of the training data will be processed through some preset sample expansion strategies to obtain a second training sample. That is to say, there will be certain differences between the first training sample and the second training sample in the training data or the labels of the training data.

[0050] Specifically, there are various sample expansion strategies for processing training samples, including, for example, modifying the training data or labels, deleting training samples, or repeating training samples. In other words, processing at least one of the training data and its labels according to the sample expansion strategy to obtain a second training sample may typically include one or more of the following steps:

[0051] Transforming the labels in the first training sample to obtain a second training sample containing training data corresponding to different labels;

[0052] Transforming the training data in the first training sample to obtain a second training sample containing different training data corresponding to the same label;

[0053] Repeating a portion of the first training sample in a certain proportion to obtain a second training sample;

[0054] Part of the training samples in the first training samples are deleted according to a certain ratio to obtain the second training samples.

[0055] As can be seen, the above provides several feasible implementation solutions for processing training samples. For example, when the labels in the first training sample are transformed, the second training sample obtained at this time contains the same training data corresponding to different labels, that is, there is data with multiple labels. Specifically, for ease of understanding, let's take the example of training data including (A1, B1), (A2, B2), and (A3, B3), where A1, A2, and A3 are multiple different training data, and B1, B2, and B3 are the labels corresponding to the training data A1, A2, and A3, respectively. In this case, the labels can be adjusted, for example, adjusting B1 to a similar B1' and B2 to a similar B2'. In this case, the second training sample includes (A1, B1), (A2, B2), and (A3, B3), as well as (A1, B1') and (A2, B2'). It can be seen that in the second training sample, the same training data corresponds to multiple different labels. For example, training data A1 corresponds to labels B1 and B1', and training data A2 corresponds to B2 and B2'. Alternatively, the training data in the first training sample can be transformed. In this case, the second training sample will have different training data corresponding to the same label, that is, there is a phenomenon of majority-one-label. Similarly, for ease of understanding, let's take the aforementioned training samples including (A1, B1), (A2, B2), and (A3, B3) as an example. At this time, the training data can be adjusted. For example, the training data A1 is adjusted to a similar A1′, and the training data A2 is adjusted to a similar A2′. At this time, the second training sample obtained includes (A1, B1), (A2, B2), and (A3, B3) as well as (A1′, B1) and (A2′, B2). It can be seen that in the second training sample obtained at this time, there are multiple different training data corresponding to the same label. For example, the training data A1 and A1′ both correspond to the label B1, and the training data A2 and A2′ both correspond to the label B2.

[0056] Of course, in addition to the aforementioned processing of training data or labels, the training samples may be repeated or deleted according to a certain ratio to obtain a second training sample with a larger or smaller number of samples.

[0057] Furthermore, considering that in different application scenarios, training samples and labels can be of different data types, such as text, images, audio, etc., the methods of changing training samples and labels of different data types are generally different. Specifically, for ease of understanding, taking text-type training samples or labels as an example, such as the training samples mentioned above for training human-computer conversation models, changes to the training data or labels can include replacing some words in the training data or labels with synonyms, deleting some words in the training data or labels, or repeating some words in the training reagent library or labels. For image-type data, such as the scenario mentioned above for training image recognition models, some areas of the sample image used for training can be cropped and replaced with corresponding areas in other images, or some areas in the image can be modified to a default color, such as white or black, by masking. Of course, one can also directly copy one area of ​​the image and replace it with another area in the image, etc. In addition, for audio-type data, such as speech datasets, corresponding changes to the phonemes and syllables in the audio can be considered, such as duplication, deletion, modification, etc.

[0058] Of course, the above solutions are only some feasible implementation solutions for sample expansion strategies that can be used to process at least one of the training data and labels. In fact, the sample expansion strategy can also include other strategies. For example, the sample expansion strategy can also be any combination of the above feasible implementation solutions, or it can be used to process part of the training data or labels according to a certain strategy. Figure 2 , Figure 2 A flowchart of the steps for expanding training samples provided in an embodiment of the present application is described in detail as follows.

[0059] In the embodiment of the present application, the weight of each training data is specifically considered to appropriately screen the training samples and expand the selected training samples, thereby improving the processing effect of the training sample data while minimizing the amount of data processing. The weight of the training data here can be understood as a label of the training data, that is, it can be used to indicate the value that the training data can provide when used for model training. Specifically, steps S210 to S230 include:

[0060] S210: Determine a weight coefficient for each training data in the first training sample.

[0061] In the embodiment of the present application, combined with the above-mentioned relevant description, it can be seen that in order to improve the processing effect of the training sample data on the basis of reducing the amount of data processing as much as possible, the weight coefficient of the training data can generally be used to indicate the value that the training data can provide when used for model training. Specifically, the weight coefficient of the training data can generally be the result obtained by inputting the training data into the trained value assessment model for processing. Its value can generally be between 0 and 1. The closer the weight coefficient is to 1, the higher the value that the training data can provide when used for model training. For example, training data with a higher weight coefficient is generally training data with more accurate label results obtained by manual annotation. Conversely, the closer the weight coefficient is to 0, the lower the value that the training data can provide when used for model training. For example, such training data is usually limited to the application scenario, and the unverified training data collected directly from the web page through network technology may have the risk of incorrect labels, so its corresponding weight coefficient is lower. Specifically, the value assessment model can generally be obtained by training a pre-trained model through some training data and labeled value labels, that is, weight coefficient labels, to evaluate the value that the training data can provide when it is subsequently used for model training.

[0062] As a further optional implementation scheme of the present application, in addition to processing the training data to evaluate the weight coefficient indicating the value that the training data can provide when used for model training, the value evaluation model can also output the value evaluation category of the training data. For example, the value evaluation category can be high frequency, that is, the frequency of the training data appearing is high, or accurate and concise, that is, the training data and its corresponding label data are relatively ideal. These value evaluation categories can be better used for subsequent screening of training data so that they can be processed through appropriate sample expansion strategies.

[0063] S220 : Determine a plurality of candidate training data from the training data included in the first training sample according to a weight coefficient of each training data.

[0064] In the embodiment of the present application, in combination with the weight coefficients of the aforementioned training data, the training data included in the first training sample can be screened according to different strategies, corresponding to different training data, and in different proportions to determine multiple candidate training data.

[0065] For example, as a common feasible implementation scheme, for a batch of training data with a higher weight coefficient, such as a batch of training data with a weight coefficient higher than 0.8, multiple candidate training data can be screened out at a higher ratio, for example, 80% of the training data can be screened out as candidate training data, while for a batch of training data with a lower weight coefficient, such as a batch of training data with a weight coefficient lower than 0.2, multiple candidate training data can be screened out at a lower ratio, for example, 20% of the training data can be screened out as candidate training data. Of course, the screening of candidate training data provided above is only a feasible implementation scheme. In fact, under different sample expansion strategies, the division of the weight coefficient range and the proportion of candidate training data screened out within each weight coefficient range can be set differently. For example, for a batch of training data with a weight coefficient higher than 0.6 and less than 0.7, 50% of the training data can be screened out as candidate training data.

[0066] S230 : Process the candidate training data and the labels of the candidate training data according to a sample expansion strategy to obtain a second training sample.

[0067] In an embodiment of the present application, after screening out a suitable proportion of candidate training data from the training data according to the different training data weight coefficients as mentioned above, by further combining the sample expansion strategy provided above to modify, copy or delete the training data, a corresponding second training sample can also be obtained, so as to facilitate the subsequent use of the effect of the model trained according to different sample expansion strategies to determine the impact information of the sample expansion strategy on the model training effect.

[0068] S130: Train the first model using the second training sample to obtain a second model.

[0069] In the embodiment of the present application, after the training sample is processed using any of the aforementioned sample expansion strategies and a second training sample is collected, in order to better determine the impact of the sample expansion strategy on model training, the second training sample can be used to train the preset first model to obtain a second model. Specifically, the process of using the second training sample to train the first model can generally be implemented based on the idea of ​​backpropagation, and specifically, generally includes the following steps:

[0070] (1) Dividing the second training sample into multiple batches according to a preset rule, and in each round of training, inputting one batch of training samples into the first model for prediction to obtain corresponding prediction results;

[0071] (2) Based on the difference between the prediction results and the labels corresponding to the training samples of the batch, the model parameters of the first model are optimized and updated using the idea of ​​back propagation to obtain an updated model;

[0072] (3) Repeatedly input the training samples of the new batch into the updated model to obtain the corresponding prediction results until the difference between the prediction results obtained after several batches and the corresponding labels in the training samples of the corresponding batch meets certain conditions, such as being less than a preset threshold. In this case, the model can be considered to have been optimized and the currently updated model can be used as the trained second model.

[0073] S140: Process the training data in the first training sample according to the second model to obtain a predicted label.

[0074] In order to further evaluate the effect of the trained second model and to determine the impact of the sample expansion strategy on the model, in an embodiment of the present application, the trained second model will be used to re-predict the training data in the first training sample, that is, the training data in the first training sample is input into the second model for processing, thereby obtaining the label predicted by the second model. Specifically, it can be understood that since the second model is trained using a second training sample that is different from the first training sample, the predicted label obtained by the second training model predicting the training data in the first training sample will usually be different from the true label, that is, the label of the training data in the first training sample. This difference can be used to a certain extent to evaluate the impact of the sample expansion strategy on the first model. Please refer to the description of the subsequent step S150 for details.

[0075] S150 : Determine, based on the predicted label and the label of the training data in the first training sample, the impact information of the sample expansion strategy on the first model.

[0076] In an embodiment of the present application, by comparing the predicted labels with the labels of the training data in the first training sample, the effect of the trained second model on the training data processing can be determined, thereby determining the impact information of different sample expansion strategies on the first model. Specifically, the impact information can generally be used to describe the impact of the sample expansion strategy on the model training effect. For example, under normal circumstances, the impact information can generally have a positive impact, that is, the sample expansion strategy is used to process the training samples to improve the training effect of the model, and a negative impact, that is, the sample expansion strategy is used to process the training samples to improve the training effect of the model. In addition, the impact information can further include the degree of impact. For example, the sample expansion strategy is used to process the training samples to greatly improve the training effect of the model, or it may slightly reduce the training effect of the model, and so on.

[0077] Specifically, the impact of the sample expansion strategy on the first model is determined by comparing the predicted labels with the true labels. Usually, the difference between the predicted labels and the true labels is used to determine whether the data with a larger difference or the data with a smaller difference is used as the target data. Based on the number or proportion of this batch of target data in the training data, the evaluation indicators of the second model are determined from multiple dimensions. For details, please refer to Figure 3 , Figure 3 A flowchart of steps for determining the impact of different strategies on model training effects based on label differences provided in an embodiment of the present application, specifically including steps S310 to S330:

[0078] S310 : Determine target data from the training data of the first training sample according to a difference between the predicted label and the label of the training data in the first training sample.

[0079] In the embodiment of the present application, the difference between the predicted label and the label of the training data in the first training sample can usually be obtained by calculating the similarity between the two, or by making a difference between the two. Specifically, for different types of labels, such as the text, image, audio, etc. mentioned above, the specific implementation scheme for calculating the difference can be different. For example, for text-type labels, the difference between labels can be determined by calculating the vector similarity or Euclidean distance between the text vectors corresponding to the two segments of labels. Of course, it is also feasible to determine the difference between labels by fusing cross entropy or other methods. The embodiment of the present application does not limit the specific implementation scheme for calculating the difference between labels. For image-type labels, for example, when a certain area in the image is used as a label, the difference between the labels can be the intersection over union (IOU) of the corresponding areas in the two label images, that is, the ratio of the intersection area of ​​the two areas (borders) to the union area. Of course, it can also be determined based on the statistical results of the difference in pixel values ​​of the corresponding areas in the two label images. In addition, for audio-type labels, the difference can be determined by the difference in phonemes or syllables, or the difference in the pronunciation position of phonemes or syllables.

[0080] On the basis of the above, by utilizing the difference between the predicted label and the label of the training data in the first training sample, it is possible to screen out the training data of the first training sample with a large label difference, such as training data with a larger label difference than a preset threshold, as target data, or screen out the training data with a small label difference, such as training data with a smaller label difference than a preset threshold, as target data. Of course, it should be noted that no matter which method is used, it will not affect the subsequent implementation plan for determining the evaluation index. Therefore, for ease of description, in the embodiment of the present application, the training samples with smaller label differences can be regarded as correctly classified samples, and the training samples with larger label differences can be regarded as incorrectly classified samples, and used as target data for subsequent processing.

[0081] S320: Determine a target evaluation index of the second model according to the amount of target data in the training data.

[0082] In the embodiment of the present application, in combination with the aforementioned solution, after filtering out the target data that is correctly classified or incorrectly classified in the training data, based on the statistics of the number of target data in the training data, such as the percentage of the number, the target evaluation index of the second model can be calculated according to a preset calculation formula. Specifically, among the target evaluation indicators provided in the embodiment of the present application, the evaluation indicators commonly used in the model are generally used, that is, at least one of the recall rate, accuracy rate, and precision rate. Specifically, to facilitate the understanding of the above evaluation indicators, the following will be explained in combination with specific definitions. Specifically, TP describes the number of samples in the training data that are actually positive and predicted to be positive, FP represents the number of samples in the training data that are actually negative and predicted to be positive, FN represents the number of samples in the training data that are actually positive and predicted to be negative, and TN represents the number of samples in the training data that are actually negative and predicted to be negative. It can be understood that TP+FP represents the number of all samples predicted to be positive, TN+FN represents the number of all samples predicted to be negative, TP+FN represents the number of samples that are actually positive, TN+FP represents the number of samples that are actually negative, TP+TN represents the number of samples that are actually predicted to be correct, and FP+FN represents the number of samples that are actually predicted to be incorrect. At this time, for Accuracy, its calculation formula is as follows:

[0083] Accuracy=(TP+TN) / (TP+TN+FP+FN)

[0084] That is, the accuracy describes the proportion of training data that are correctly predicted in all training samples.

[0085] As for the precision, the calculation formula is as follows:

[0086] Precision = TP / (TP+FP)

[0087] That is, the precision is used to describe the proportion of training samples predicted to be positive whose actual results are also positive, that is, the proportion of training data with correct prediction results.

[0088] Correspondingly, the calculation formula for recall is as follows:

[0089] recall=TP / (TP+FN)

[0090] That is, the recall rate is used to describe the proportion of training data that are predicted to be positive among the training samples that are actually positive, that is, the number of training data with correct prediction results.

[0091] Of course, it should be noted that in different application scenarios, based on the different training data, the evaluation indicators that are emphasized are also different. Therefore, the target evaluation indicators actually used can usually include at least one of the recall rate, accuracy rate and precision rate mentioned above. For example, in some training data, if the sample ratio of positive and negative training data is unbalanced, assuming that the proportion of positive samples with actual positive results in the training samples is 90%, and the proportion of negative samples with actual negative results is only 90%, then at this time, even if the model prediction result is always predicted to be positive, 90% accuracy can still be obtained, but the actual performance of the model is poor. At this time, you can consider using precision rate or recall rate as the target evaluation indicator of the second model to more accurately determine the impact of different sample expansion strategies on model training.

[0092] S330: Determine the impact information of the sample expansion strategy on the first model according to the target evaluation index.

[0093] In an embodiment of the present application, after determining the target evaluation index of the second model in the aforementioned manner, the impact information of the sample expansion strategy on the first model can be determined based on the target evaluation index. Furthermore, when there are multiple target evaluation indicators, for example, at least two of the recall rate, accuracy rate, and precision rate, the impact information of the sample expansion strategy on the first model can be determined by weighting the target evaluation index. Specifically, determining the impact information based on the target evaluation index can be by pre-setting different target evaluation index value ranges for different impact information. For example, when the recall rate, accuracy rate, precision rate, or their weighted sum is lower than a certain threshold, it can be considered that the impact information of the sample expansion strategy on the first model is negative, that is, the training effect of the model is reduced. On the contrary, when the recall rate, accuracy rate, precision rate, or their weighted sum is higher than a certain threshold, it can be considered that the impact information of the sample expansion strategy on the model training is positive, that is, the training effect of the model can be improved to a certain extent. The higher the recall rate, accuracy rate, precision rate, or their weighted sum, the better the improvement effect of the model training by the sample expansion strategy.

[0094] Of course, the above solution is only explained by taking the labels in the first training sample as the real labels. In fact, in some application scenarios, the labels in the collected first training samples may be wrong, which will also lead to errors in the information about the impact of the determined sample expansion strategy on model training. Therefore, as another feasible embodiment of this application, please refer to Figure 4 , Figure 4 Another flowchart of determining the impact of different strategies on model training effects based on label differences provided in an embodiment of the present application includes steps S410 to S430:

[0095] S410: Train the first model using the first training sample to obtain a third model.

[0096] In the embodiment of the present application, similar to the step of training the first model using the second training sample in the aforementioned step S130, the first model can also be trained synchronously with the first training sample to obtain the third model. The specific implementation scheme of training the first model using the first training sample can be referred to the aforementioned step S130.

[0097] S420: Process the training data in the first training sample according to the third model to obtain a reference label.

[0098] In an embodiment of the present application, similar to the processing of the training data in the first training sample by the second model in the aforementioned step S140, the corresponding reference label can be obtained by inputting the training data in the first training sample into the third model for processing. It can be understood that since the third model is obtained by training the first model using the first training sample that is different from the second training sample, the result predicted by the third model is usually different from the result predicted by the second model.

[0099] S430 : Determine, based on the predicted label, the reference label, and the label of the training data in the first training sample, the impact information of the sample expansion strategy on the first model.

[0100] In the embodiment of the present application, combined with the above-mentioned related description, it can be known that the impact information of the sample expansion strategy on the first model can be determined by comparing the predicted label, the reference label and the label of the training data in the first training sample. Specifically, the target evaluation index of the second model can be determined by the predicted label and the label of the training data in the first training sample, and the target evaluation index of the third model can be determined based on the reference label and the label of the training data in the first training sample. It can be understood that the target evaluation index of the second model and the target evaluation index of the third model respectively describe the model effect of the model trained before and after sample expansion. Therefore, the impact of the sample expansion strategy on the first model can be determined by comparing the target evaluation index of the second model with the target evaluation index of the third model. For example, when the target evaluation index of the third model is lower than the target evaluation index of the second model, it can be considered that the impact of the sample expansion strategy on the first model is negative, that is, the training effect of the model is reduced, and the greater the difference between the target evaluation index of the third model and the target evaluation index of the second model, the more obvious the reduction in the training effect of the model. Conversely, if the target evaluation index of the third model is higher than the target evaluation index of the second model, it can be considered that the impact of the sample expansion strategy on the first model is positive, that is, the training effect of the model is improved, and the greater the difference between the target evaluation index of the third model and the target evaluation index of the second model, the more obvious the improvement in the training effect of the model.

[0101] And through the method provided above, when the first training sample is processed by different sample expansion strategies to obtain different second training samples, and multiple groups of second models are trained respectively, the impact information of each sample expansion strategy on the first model can be determined. The impact evaluation result can be used as relevant experience to guide the subsequent expansion of training sample data in the model training process. Specifically, considering that different sample expansion strategies have different effects on model training in different scenarios, for example, the same sample expansion strategy has different effects on model training when used for human-computer conversation model training or for image recognition. Therefore, as a further feasible implementation plan of this application, please refer to Figure 5 , Figure 5 A flowchart of steps for associating and storing a sample expansion strategy with a processing task type is provided in an embodiment of the present application, specifically including steps S510 to S530:

[0102] S510: Determine the processing task type of the first model.

[0103] In the embodiments of the present application, combined with the above-mentioned relevant descriptions, it can be seen that the processing task types generally include recognition tasks, classification tasks, and labeling tasks, etc., among which the recognition tasks can be face recognition, emotion recognition, image recognition, etc., the classification tasks can be image classification, fault classification, etc., and the labeling tasks can be named entity labeling tasks, etc. In addition, the application scenarios of image recognition, object classification, etc. mentioned above are also feasible. Of course, in addition to the several feasible solutions provided above, the processing task type can also be determined based on the training requirements of the model. The embodiments of the present application do not make specific restrictions on the processing task type.

[0104] S520: Determine a target sample expansion strategy from the multiple sample expansion strategies according to the impact information.

[0105] In the embodiment of the present application, combined with the above-mentioned relevant description, it can be seen that after obtaining the impact information of multiple different sample expansion strategies on the first model based on different sample expansion strategies, based on the comparison of different impact information, the target sample expansion strategy that can most effectively improve the model training effect can be determined from the sample expansion strategies.

[0106] S530: Associate the processing task type of the first model with the target sample expansion strategy and store them in a preset database.

[0107] In an embodiment of the present application, by associating the target sample expansion strategy that is most effective in improving the model training effect with the processing task type of the first model and storing it in a preset database, the training samples can be processed using the target sample expansion strategy during the training process of the model of the processing task type, thereby improving the training effect of the model.

[0108] According to the solution of the embodiment of the present application, after obtaining the first training sample, at least one of the training data and the label of the training data in the first training sample will be processed according to the sample expansion strategy to obtain a second training sample, and then the training data in the first training sample will be verified using the second model trained by the second training sample to obtain a predicted label, wherein the label of the training data can be regarded as the true label, and the predicted label can be understood as the verification information of the model trained by the training sample obtained after processing by the sample expansion strategy. Therefore, based on these two, it can be determined whether the second model trained after processing by the sample expansion strategy has a higher or worse prediction effect, thereby determining the impact information of the sample expansion strategy on the first model, so that the most suitable sample expansion strategy for model training can be selected by comparing the impact information of different sample expansion strategies on model training. Therefore, when the training samples are insufficient in the subsequent model training process, a reasonable sample expansion strategy can be used to improve the training effect of the model.

[0109] Specifically, to facilitate understanding of the above solution, the following will take the model training process as an example to determine how to expand the sample to improve the model training effect. For details, please refer to Figure 6 , Figure 6 A flowchart of a training model provided in an embodiment of the present application, specifically including steps S610 to S640:

[0110] S610: Acquire an initial model and initial samples for training the initial model.

[0111] In the embodiments of the present application, the initial sample includes initial data and labels for the initial data. Similar to the model training method in related art, the initial model here is typically a network model with parameters initialized based on actual needs and established by the user within an existing model structure. Of course, the initial model can also be a publicly available pre-trained model. Accordingly, the initial data and labels in the initial model are primarily used to complete the training of the initial model.

[0112] S620 , according to the processing task type of the initial model, read from a preset database a target sample expansion strategy corresponding to the processing task type.

[0113] In an embodiment of the present application, the processing task type is similar to the processing task type mentioned in the aforementioned step S510, and generally includes recognition tasks, classification tasks, and labeling tasks, etc., wherein the recognition task can be face recognition, emotion recognition, image recognition, etc., the classification task can be image classification, fault classification, etc., and the labeling task can be a named entity labeling task, etc.

[0114] Since the sample expansion strategies corresponding to different processing task types are stored in the preset database in advance, the corresponding target sample expansion strategy can be read from the preset database based on the processing task type of the initial model. For example, the target sample expansion strategy can be to transform the initial data in the initial sample according to a certain ratio.

[0115] S630: Process at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample.

[0116] In an embodiment of the present application, after determining a target sample expansion strategy that is more suitable for initial model training based on the processing task type, the initial data in the initial sample and at least one of the labels of the initial data are processed based on the target sample expansion strategy to obtain the target sample after data expansion adjustment.

[0117] Of course, it should be noted that, in conjunction with the aforementioned model evaluation, the target change strategy here not only includes adjustments to the initial training data and initial training labels, but also typically includes the extraction ratios of initial data of different values ​​and the processing strategy for the extracted data. For example, for low-quality initial data, a higher ratio can be extracted and adjusted to better expand the sample data, while for high-quality initial data, a lower ratio can be extracted and adjusted to ensure the effectiveness of the sample data for model training.

[0118] S640: Train the initial model according to the target sample to obtain a trained model.

[0119] In an embodiment of the present application, after the expansion and enhancement of the initial sample is completed to obtain the target sample, the target sample can be used based on the idea of ​​machine learning to complete the training of the initial model, thereby obtaining a trained model. At this time, the trained model has a more ideal effect under the corresponding processing task type.

[0120] Of course, as a further feasible implementation scheme of the present application, it is usually necessary to appropriately expand and adjust the initial samples when the initial samples are small and the ideal model cannot be effectively trained. Therefore, before reading the target sample expansion strategy corresponding to the processing task type from the preset database according to the processing task type of the initial model, the method further includes:

[0121] determining the sample size of the initial sample;

[0122] When the number of samples is less than a preset number threshold, the step of reading the target sample expansion strategy corresponding to the processing task type according to the initial model from the preset database is executed.

[0123] In the embodiments of the present application, when it is inconvenient to collect labeled training data and the number of initial samples is small, for example, when the number of samples is less than a preset threshold, the target sample expansion strategy corresponding to the processing task type of the initial model can be read from a preset database. Of course, when the number of samples is sufficient, it is also possible to consider training the initial model directly based on the current sample data to obtain a trained model.

[0124] In order to better implement the evaluation method and model training method provided in the embodiment of the present application, in addition to the evaluation method provided in the embodiment of the present application, this embodiment also provides an evaluation device, which can be integrated into the server. Figure 7 As shown, the evaluation device may include:

[0125] A first acquisition module 710 is configured to acquire a first training sample, wherein the first training sample includes training data and a label of the training data;

[0126] an expansion module 720, configured to process at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample;

[0127] A first training module 730 is configured to train the first model using the second training sample to obtain a second model;

[0128] a processing module 740, configured to process the training data in the first training sample according to the second model to obtain a predicted label;

[0129] The determination module 750 is configured to determine the impact of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample.

[0130] In some embodiments, the expansion module 720 is further used to determine a weight coefficient for each training data in the first training sample; determine a plurality of candidate training data from the training data contained in the first training sample according to the weight coefficient of each training data; and process the candidate training data and the labels of the candidate training data according to the sample expansion strategy to obtain a second training sample.

[0131] In some embodiments, the determination module 750 is also used to determine the target data from the training data of the first training sample based on the difference between the predicted label and the label of the training data in the first training sample; determine the target evaluation index of the second model based on the number of target data in the training data; the target evaluation index includes at least one of recall rate, accuracy rate and precision rate; and determine the impact information of the sample expansion strategy on the initial model based on the target evaluation index.

[0132] In some embodiments, the determination module 750 is also used to train the first model using the first training sample to obtain a third model; process the training data in the first training sample according to the third model to obtain a reference label; and determine the impact information of the sample expansion strategy on the first model based on the predicted label, the reference label and the label of the training data in the first training sample.

[0133] In some embodiments, the determination module 750 is further configured to determine a processing task type of the first model, wherein the processing task type includes at least one of a recognition task, a classification task, and a labeling task;

[0134] A target sample expansion strategy is determined from the multiple sample expansion strategies according to the impact information; and the processing task type of the first model and the target sample expansion strategy are associated and stored in a preset database.

[0135] The embodiment of the present application also provides a model training device, which can be integrated into a server. Figure 8 As shown, the model training device may include:

[0136] A second acquisition module 810 is configured to acquire an initial model and initial samples for training the initial model; the initial samples include initial data and labels of the initial data;

[0137] A reading module 820 is configured to read a target sample expansion strategy corresponding to a processing task type of the initial model from a preset database according to the processing task type of the initial model;

[0138] a transformation module 830, configured to process at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample;

[0139] The second training module 840 is used to train the initial model according to the target sample to obtain a trained model.

[0140] According to the solution of the embodiment of the present application, after obtaining the first training sample, at least one of the training data and the label of the training data in the first training sample will be processed according to the sample expansion strategy to obtain a second training sample, and then the training data in the first training sample will be verified using the second model trained by the second training sample to obtain a predicted label, wherein the label of the training data can be regarded as the true label, and the predicted label can be understood as the verification information of the model trained by the training sample obtained after processing by the sample expansion strategy. Therefore, based on these two, it can be determined whether the second model trained after processing by the sample expansion strategy has a higher or worse prediction effect, thereby determining the impact information of the sample expansion strategy on the first model, so that the most suitable sample expansion strategy for model training can be selected by comparing the impact information of different sample expansion strategies on model training. Therefore, when the training samples are insufficient in the subsequent model training process, a reasonable sample expansion strategy can be used to improve the training effect of the model.

[0141] Based on the same inventive concept, an embodiment of the present application further provides a computer device, which may be a server or terminal device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-mentioned evaluation method or model training method. This implements various functions, such as executing the following steps:

[0142] Obtain a first training sample; the first training sample includes training data and a label of the training data;

[0143] Processing at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample;

[0144] Training the first model using the second training sample to obtain a second model;

[0145] Processing the training data in the first training sample according to the second model to obtain a predicted label;

[0146] Determine the impact of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample

[0147] Alternatively, perform the following steps:

[0148] Acquire an initial model and an initial sample for training the initial model; the initial sample includes initial data and a label of the initial data;

[0149] According to the processing task type of the initial model, a target sample expansion strategy corresponding to the processing task type is read from a preset database;

[0150] Processing at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample;

[0151] The initial model is trained according to the target sample to obtain a trained model.

[0152] According to the solution of the embodiment of the present application, after obtaining the first training sample, at least one of the training data and the label of the training data in the first training sample will be processed according to the sample expansion strategy to obtain a second training sample, and then the training data in the first training sample will be verified using the second model trained by the second training sample to obtain a predicted label, wherein the label of the training data can be regarded as the true label, and the predicted label can be understood as the verification information of the model trained by the training sample obtained after processing by the sample expansion strategy. Therefore, based on these two, it can be determined whether the second model trained after processing by the sample expansion strategy has a higher or worse prediction effect, thereby determining the impact information of the sample expansion strategy on the first model, so that the most suitable sample expansion strategy for model training can be selected by comparing the impact information of different sample expansion strategies on model training. Therefore, when the training samples are insufficient in the subsequent model training process, a reasonable sample expansion strategy can be used to improve the training effect of the model.

[0153] In one embodiment, the computer device is a terminal device, for example, its internal structure diagram can be as follows Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an evaluation method or a model training method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0154] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0155] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0156] Since the computer program stored in the computer-readable storage medium can execute any evaluation method or model training method provided in the embodiments of the present application, the beneficial effects that can be achieved by any evaluation method or model training method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0157] According to the solution of the embodiment of the present application, after obtaining the first training sample, at least one of the training data and the label of the training data in the first training sample will be processed according to the sample expansion strategy to obtain a second training sample, and then the training data in the first training sample will be verified using the second model trained by the second training sample to obtain a predicted label, wherein the label of the training data can be regarded as the true label, and the predicted label can be understood as the verification information of the model trained by the training sample obtained after processing by the sample expansion strategy. Therefore, based on these two, it can be determined whether the second model trained after processing by the sample expansion strategy has a higher or worse prediction effect, thereby determining the impact information of the sample expansion strategy on the first model, so that the most suitable sample expansion strategy for model training can be selected by comparing the impact information of different sample expansion strategies on model training. Therefore, when the training samples are insufficient in the subsequent model training process, a reasonable sample expansion strategy can be used to improve the training effect of the model.

[0158] Based on the same inventive concept, embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0159] It should be noted that the object data (including but not limited to user device information, user personal information, etc.) and conversation data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0160] Any reference to the memory, database or other media used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0161] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0162] In the above-mentioned evaluation device or model training device, computer-readable storage medium, computer equipment, and computer program product embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and beneficial effects of the above-described evaluation device or model training device, computer-readable storage medium, computer program product, computer equipment and its corresponding units can be referred to the description of the evaluation method or model training method in the above embodiments, and will not be repeated here.

[0163] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] The above is a detailed introduction to an evaluation and model training method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An evaluation method, characterized in that: include: Obtain a first training sample; The first training sample includes training data and a label of the training data; Processing at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample; Training the first model using the second training sample to obtain a second model; Processing the training data in the first training sample according to the second model to obtain a predicted label; According to the predicted label and the label of the training data in the first training sample, the influence information of the sample expansion strategy on the first model is determined.

2. The method according to claim 1, characterized in that The processing of at least one of the training data and the label of the training data according to the sample expansion strategy includes: Determining a weight coefficient for each training data in the first training sample; determining a plurality of candidate training data from the training data included in the first training sample according to a weight coefficient of each training data; The candidate training data and the labels of the candidate training data are processed according to a sample expansion strategy to obtain a second training sample.

3. The method according to claim 1, characterized in that The determining, based on the predicted label and the label of the training data in the first training sample, information on the impact of the sample expansion strategy on the first model includes: determining target data from the training data of the first training sample based on a difference between the predicted label and the label of the training data in the first training sample; Determining a target evaluation index of the second model according to the amount of target data in the training data; the target evaluation index includes at least one of recall rate, accuracy rate and precision rate; Determine the impact of the sample expansion strategy on the first model based on the target evaluation index.

4. The method according to claim 1, wherein The determining, based on the predicted label and the label of the training data in the first training sample, information on the impact of the sample expansion strategy on the first model includes: Training the first model using the first training sample to obtain a third model; Processing the training data in the first training sample according to the third model to obtain a reference label; Determine the impact information of the sample expansion strategy on the first model based on the predicted label, the reference label, and the label of the training data in the first training sample.

5. The method according to claim 1, wherein The sample expansion strategy includes multiple strategies; after determining the impact of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample, the method further includes: Determining a processing task type of the first model, where the processing task type includes at least one of a recognition task, a classification task, and a labeling task; determining a target sample expansion strategy from the plurality of sample expansion strategies according to the impact information; The processing task type of the first model and the target sample expansion strategy are associated and stored in a preset database.

6. A model training method, characterized in that: include: Acquire an initial model and initial samples for training the initial model; The initial sample includes initial data and a label of the initial data; According to the processing task type of the initial model, a target sample expansion strategy corresponding to the processing task type is read from a preset database; Processing at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample; The initial model is trained according to the target sample to obtain a trained model.

7. An evaluation device, characterized in that include: A first acquisition module, configured to acquire a first training sample; The first training sample includes training data and a label of the training data; an expansion module, configured to process at least one of the training data and the label of the training data according to a sample expansion strategy to obtain a second training sample; A first training module, configured to train the first model using the second training sample to obtain a second model; a processing module, configured to process the training data in the first training sample according to the second model to obtain a predicted label; A determination module is used to determine the impact information of the sample expansion strategy on the first model based on the predicted label and the label of the training data in the first training sample.

8. A model training device, characterized in that: include: A second acquisition module is used to acquire an initial model and initial samples for training the initial model; The initial sample includes initial data and a label of the initial data; A reading module is used to read the target sample expansion strategy corresponding to the processing task type from a preset database according to the processing task type of the initial model; a transformation module, configured to process at least one of the initial data and the label of the initial data according to the target sample expansion strategy to obtain a target sample; The second training module is used to train the initial model according to the target sample to obtain a trained model.

9. A computer device, characterized in that: It includes a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps of the evaluation method according to any one of claims 1 to 5 or the steps of the model training method according to claim 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute the steps of the evaluation method according to any one of claims 1 to 5 or the steps of the model training method according to claim 6.

Images