Model training data generation method and apparatus, and electronic device

Abstract

This disclosure provides a method, apparatus, and electronic device for generating model training data, relating to the field of artificial intelligence, specifically image annotation and data mining techniques. It can be applied in smart city, urban governance, and emergency management scenarios. The method for generating model training data includes: obtaining first sample data and second sample data from a sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data; calculating a first distribution parameter of the first sample data and determining predicted sample data based on the second sample data; calculating a second distribution parameter of the predicted sample data; and determining the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, wherein the target data is used to train the model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to image annotation and data mining technologies, which can be applied in smart city, urban governance, and emergency management scenarios. Specifically, it relates to a method, device, and electronic device for generating model training data. Background Technology

[0002] With the continuous development of artificial intelligence technology, data is playing an increasingly important role in many business scenarios of artificial intelligence. However, since the same data may appear in different scenarios, it is necessary to screen and determine the data. Currently, manual screening and determination of data is usually adopted. Summary of the Invention

[0003] This disclosure provides a method, apparatus, and electronic device for generating model training data.

[0004] According to a first aspect of this disclosure, a method for generating model training data is provided, comprising:

[0005] Obtain first sample data and second sample data from the sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data;

[0006] Calculate the first distribution parameter of the first sample data, and determine the predicted sample data based on the second sample data;

[0007] Calculate the second distribution parameter of the predicted sample data;

[0008] When the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, the predicted sample data is determined as target data, and the target data is used to train the model.

[0009] According to a second aspect of this disclosure, a model training data generation apparatus is provided, comprising:

[0010] The acquisition module is used to acquire first sample data and second sample data from the sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data;

[0011] The calculation and determination module is used to calculate the first distribution parameter of the first sample data and determine the predicted sample data based on the second sample data.

[0012] The calculation module is used to calculate the second distribution parameters of the predicted sample data;

[0013] The determination module is used to determine the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition. The target data is used to train the model.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory that is communicatively connected to at least one processor; wherein,

[0017] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform any of the methods in the first aspect.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform any of the methods in the first aspect.

[0019] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the methods in the first aspect.

[0020] In this embodiment of the disclosure, when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, the predicted sample data is determined as the target data. The target data can be identified as data worth mining and optimizing, and the target data can be used to train the model. In this way, the efficiency of determining the target data and the accuracy of the determination result are improved.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a method for generating model training data according to an embodiment of this disclosure;

[0023] Figure 2 This is a schematic diagram illustrating the inventive concept of a model training data generation method provided in this embodiment of the disclosure;

[0024] Figure 3 This is a schematic diagram of the process of feature extraction from second sample data in a model training data generation method provided in this embodiment of the disclosure;

[0025] Figure 4This is a flowchart illustrating another method for generating model training data provided in this embodiment of the disclosure;

[0026] Figure 5 This is a schematic diagram of the structure of a model training data generation device provided in an embodiment of this disclosure;

[0027] Figure 6 This is a schematic diagram of the structure of a calculation and determination module included in a model training data generation device provided in this embodiment of the present disclosure;

[0028] Figure 7 This is a schematic diagram of the structure of a prediction submodule included in a model training data generation device provided in an embodiment of this disclosure;

[0029] Figure 8 This is a schematic diagram of the structure of a determining unit included in a model training data generation device provided in an embodiment of this disclosure;

[0030] Figure 9 This is a schematic diagram of the structure of a determining module included in a model training data generation device provided in an embodiment of this disclosure;

[0031] Figure 10 This is a schematic diagram of another model training data generation device provided in this embodiment;

[0032] Figure 11 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] See Figure 1 , Figure 1 A flowchart of a model training data generation method provided in this disclosure embodiment is shown below. Figure 1 As shown, the method for generating model training data includes the following steps:

[0035] Step S101: Obtain first sample data and second sample data from the sample pool. The first sample data is labeled data, and the second sample data is unlabeled data.

[0036] The sample pool can also be called a full dataset database or a sample pool database. The sample pool can store various types of data. The specific types of the first sample data and the second sample data are not limited here. Optionally, the first sample data and the second sample data can be image data. When the first sample data and the second sample data are image data, the above image data can be captured by a camera and stored in the sample pool database.

[0037] Alternatively, the sample pool database can receive image data sent by electronic devices and store the image data in the sample pool database.

[0038] Optionally, the first and second sample data can also be audio data, and both can be acquired by an audio acquisition device and stored in the sample pool database. Furthermore, when the first and second sample data are audio data, the sample pool database can receive the audio data acquired by the electronic device and store the audio data in the sample pool database.

[0039] The first sample data is labeled data. The method of generating the labeled data is not limited here. Optionally, the sample data to be labeled is obtained from the data source and the sample data to be labeled is manually labeled to obtain the first sample data. In this way, the accuracy of the labeled data can be higher, and the first sample data can also be used to train the model to be trained.

[0040] The second sample data is unlabeled data, meaning that the second sample data was not manually labeled.

[0041] It should be noted that the first sample data and the second sample data can belong to the same category of data, and the above category can be understood as data in the same scenario. For example, the first sample data and the second sample data can both be data in the urban management scenario, such as data included in urban management-related regulations, or data such as garbage classification.

[0042] Step S102: Calculate the first distribution parameter of the first sample data, and determine the predicted sample data based on the second sample data.

[0043] As an optional implementation, calculating a first distribution parameter of the first sample data and determining the predicted sample data based on the second sample data includes:

[0044] Determine the first weight of the first sample data and the second weight of the second sample data;

[0045] Calculate a first distribution parameter for the product of the first sample data and the first weight, and determine the predicted sample data based on the product of the second sample data and the second weight.

[0046] In this embodiment of the disclosure, the values ​​of the first distribution parameter and the predicted sample data can be adjusted by adjusting the values ​​of the first weight and the second weight, thereby increasing the adjustment effect on the values ​​of the first distribution parameter and the predicted sample data and making the determination of the values ​​of the first distribution parameter and the predicted sample data more flexible.

[0047] Furthermore, the first weight and the second weight can be different weights. The aforementioned weights can be understood as selection weights. By increasing the second weight and decreasing the first weight, the probability of the second sample data being selected can be increased, and the probability of the first sample data being selected can be decreased. The larger the weight, the greater the importance can be understood. In this way, the probability of the second sample data with greater importance being selected can be increased, and the probability of the first sample data with less importance being selected can be decreased, thereby differentiating the probability of sample data being selected and saving the cost of selecting sample data.

[0048] It should be noted that the specific method for determining the prediction sample data based on the second sample data is not limited here.

[0049] As an optional implementation, determining the prediction sample data based on the second sample data includes:

[0050] The second sample data is input into the target prediction model for prediction, and the third sample data is output. The third sample data is unlabeled data. The target prediction model is a pre-trained model used to predict the sample data.

[0051] If the uncertainty parameter of the third sample data is greater than a preset threshold, the third sample data is determined as the predicted sample data.

[0052] In this embodiment, the second sample data can be input into the target prediction model for prediction to obtain the third sample data. The third sample data can be data stored in the prediction sample pool that belongs to the same category as the second sample data. In this way, by utilizing the prediction capability of the target prediction model, the third sample data is obtained, thereby enhancing the screening performance of the third sample data. When the uncertainty parameter of the third sample data is greater than a preset threshold, it indicates that the third sample data is more likely to be difficult data. The third sample data is then determined as the prediction sample data, thereby reducing the accuracy of the prediction sample data determination result.

[0053] It should be noted that the larger the value of the uncertainty parameter, the greater the likelihood that the corresponding third sample data belongs to difficult cases, and the greater the value to be explored. The aforementioned uncertainty parameter can also be referred to as parametric uncertainty.

[0054] As an optional implementation, determining the prediction sample data based on the second sample data includes:

[0055] The second sample data is input into a preset prediction model for parameter prediction to obtain the uncertainty parameters of the second sample data. The preset prediction model is a pre-trained model used to predict parameters of sample data.

[0056] If the uncertainty parameter of the second sample data is greater than a preset threshold, the second sample data will be determined as the predicted sample data.

[0057] In this embodiment of the disclosure, when the uncertainty parameter of the second sample data is greater than a preset threshold, it indicates that the second sample data is likely to be difficult sample data, and the third sample data is determined as the predicted sample data, thereby reducing the accuracy of the prediction sample data determination result.

[0058] Furthermore, by employing a pre-defined prediction model to predict the uncertainty parameters of the second sample data, and identifying second sample data with uncertainty parameters exceeding a pre-defined threshold as predicted sample data, the diversity and flexibility of the prediction sample data determination method can be increased. Simultaneously, since only the uncertainty parameters need to be predicted, rather than the entire sample data, the computational resource consumption of the pre-defined prediction model is reduced, thereby improving the efficiency of predicting sample data determination.

[0059] As an optional implementation, the second sample data is image data. The step of determining the second sample data as the predicted sample data when the uncertainty parameter of the second sample data is greater than a preset threshold includes:

[0060] The matching score between the instances included in the second sample data and the instance category to which the instances belong is inferred through the preset prediction model, and the matching score is determined as the category contribution of the second sample data.

[0061] The uncertainty parameter is obtained by calculating the matching degree between the category contribution and the pre-acquired category importance through the preset prediction model. The category importance is calculated based on the number of long tails of the instances.

[0062] If the uncertainty parameter is greater than a preset threshold, the second sample data is determined as the predicted sample data.

[0063] Category importance can be calculated using the number of long-tail instances as the unit of measurement.

[0064] In this embodiment of the disclosure, since the importance of the category is taken into account, the predicted sample data can be determined to be difficult example data. Furthermore, the long-tail capability of the category to which the second sample data belongs is taken into account, which enhances the value to be mined from the predicted sample data and improves the accuracy of the determination result of the predicted sample data.

[0065] As an optional implementation, the preset prediction model includes a feature extractor, a first classifier, and a second classifier. The step of inputting the second sample data into the preset prediction model for parameter prediction to obtain the uncertainty parameters of the second sample data includes:

[0066] The second sample data is input into the feature extractor for feature extraction to obtain a first feature vector and a second feature vector.

[0067] The first feature vector is input into the first classifier for data prediction to obtain the first predicted classification data, and the second feature vector is input into the second classifier for data prediction to obtain the second predicted classification data.

[0068] The uncertainty parameter of the second sample data is determined based on the classification difference value between the first predicted classification data and the second predicted classification data.

[0069] The matching degree between the first feature vector and the second feature vector can be greater than or less than the preset matching degree.

[0070] The model structures of the first classifier and the second classifier can be the same, and the first classifier and the second classifier can be trained using the same sample data. That is, it can be considered that the first classifier and the second classifier have a high degree of matching in the classification results of the feature vector.

[0071] In this embodiment of the disclosure, the uncertainty parameter of the second sample data can be determined based on the classification difference value between the first predicted classification data and the second predicted classification data. If the classification difference value is larger, it indicates that the uncertainty parameter of the second sample data is larger; if the classification difference value is smaller, it indicates that the uncertainty parameter of the second sample data is smaller. That is, the uncertainty parameter can be directly determined through the classification difference value, making the determination method of the uncertainty parameter more intuitive and the determination result of the uncertainty parameter more accurate.

[0072] It should be noted that the specific method for determining the uncertainty parameters of the second sample data based on the classification difference between the first and second predicted classification data is not limited here.

[0073] As an optional implementation, determining the uncertainty parameter of the second sample data based on the classification difference value between the first predicted classification data and the second predicted classification data includes:

[0074] Calculate the classification difference value between the first predicted classification data and the second predicted classification data;

[0075] If the classification difference value is within a preset range, the uncertainty parameter of the second sample data is determined to be greater than the preset threshold.

[0076] In this embodiment of the disclosure, when the classification difference value is within a preset range, it indicates that the difference between the prediction classification results of the first classifier and the second classifier for the second sample data is large, thereby indicating that the second sample data is data worth mining and improving the accuracy of the judgment result of the second sample data.

[0077] As an optional implementation, determining the uncertainty parameter of the second sample data based on the classification difference value between the first predicted classification data and the second predicted classification data includes:

[0078] Calculate the classification difference value between the first predicted classification data and the second predicted classification data;

[0079] If the classification difference value is greater than the first preset difference value, the uncertainty parameter of the second sample data is determined to be greater than the preset threshold.

[0080] In this embodiment of the disclosure, when the classification difference value is greater than the first preset difference value, it indicates that the difference between the prediction classification results of the first classifier and the second classifier for the second sample data is large, thereby indicating that the second sample data is data worth mining, and improving the accuracy of the result that the second sample data belongs to data worth mining.

[0081] It should be noted that the difference between this implementation method and the above implementation method lies in the different judgment conditions for determining whether the uncertainty parameter of the second sample data is greater than the preset threshold based on the classification difference value. This increases the diversity and flexibility of the methods for determining whether the uncertainty parameter is greater than the preset threshold.

[0082] It should be noted that in the two embodiments described above, when the prediction classification results of the first classifier and the second classifier for the second sample data differ significantly, it may be because the second sample data is input into the feature extractor for feature extraction, resulting in a difference in the representation of the first feature vector and the second feature vector. That is, the same feature extractor extracts two feature vectors with different representations from the same sample data, and these two feature vectors with different representations are input into the first classifier and the second classifier respectively, resulting in a significant difference in the prediction classification results. Therefore, in this case, the second sample data can also be considered as data worth mining, and the two feature vectors with different representations can be understood as the first feature vector and the second feature vector.

[0083] For example: see Figure 3 ,like Figure 3 As shown, the second sample data can be input into the feature extractor G for feature extraction to obtain the first feature vector F1 and the second feature vector F2. The first feature vector F1 is input into the first classifier for data prediction to obtain the first predicted classification data. The second feature vector F2 is input into the second classifier for data prediction to obtain the second predicted classification data.

[0084] It should be noted that the above classification difference value can be determined by calculating the distance between the first predicted classification data and the second predicted classification data. The above distance can refer to Euclidean distance or cosine distance.

[0085] In addition, there can be multiple predicted sample data, and multiple predicted sample data can be stored in a pre-selection pool. At least some of the predicted sample data can be used to perform the steps in this embodiment, specifically steps S103 and S104. The aforementioned at least some predicted sample data can be the first K predicted sample data when the uncertain parameters of the multiple predicted sample data are arranged from largest to smallest, where K is a positive integer.

[0086] Step S103: Calculate the second distribution parameter of the predicted sample data.

[0087] The first distribution parameter and the second distribution parameter can be collectively referred to as the distribution or the distribution function.

[0088] The first distribution parameter can be calculated using the first formula, which is shown below:

[0089] Δ L (Y L ) = Softmax([δ1, δ 2， …,δ m ,...] T ),

[0090]

[0091] Among them, △ L (Y L ) represents the first distribution function, Softmax is the 0-1 distribution function, and δ1, δ2, δ... m Representing the image Y L In the example, T represents transpose, y L For image Y L The labeled instance (i.e., the first sample data), where I is the label function and m represents the value of the labeled instance.

[0092] The second distribution parameter can be calculated using the second formula, which is shown below:

[0093] Δ U (x u =Softmax([δ1,δ2,…,δ) m ,...] T )

[0094]

[0095] Among them, △ u (x u ) represents the second distribution function, Softmax is the 0-1 distribution function, and δ1, δ2, δ m Let T represent an instance in the image (i.e., the second sample data), and s denote the transpose. k and s j 'represent different instance categories in the second sample data, {s k} and {s j '} represent different instance category groups in the second sample data, ψ m and ψ m 'Represents different instances in the second sample data.

[0096] It should be noted that when calculating the first distribution parameter, the score (i.e., matching degree or similarity) of each instance in the first sample data with the category to which the instance belongs can be calculated, and the first distribution parameter can be determined based on the score. Similarly, when calculating the second distribution parameter, the score of each instance in the second sample data with the category to which the instance belongs can be calculated, and the first distribution parameter can be determined based on the score.

[0097] Step S104: When the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, the predicted sample data is determined as target data, and the target data is used to train the model.

[0098] In this embodiment, both labeled and unlabeled data are used, i.e., an active learning method is adopted. Active learning can help to filter and determine target data. When the target data is applied to model training, it can improve the performance of the model.

[0099] It should be noted that the above model may refer to the preset prediction model mentioned above.

[0100] For example, when the embodiments of this disclosure are applied to a target model, the target model can actively select the data it wants to learn, thereby achieving higher accuracy training of the target model while using less training labeled data and reducing training costs.

[0101] It should be noted that steps S101 to S104 in this embodiment can be understood as an active learning approach considering long-tail categories. Compared with existing active learning methods, this active learning approach considering long-tail categories can help improve the mining of instance data such as key business categories and defect categories. That is, this embodiment can calculate the difference in instance distribution between labeled and unlabeled data, thereby enabling focused consideration of key categories and defect categories to determine target data with higher mining value. Experiments show that, compared with existing active learning methods, this active learning approach considering long-tail categories can significantly improve the identification ability of long-tail distributed data, which can also be referred to as long-tail category data.

[0102] In this embodiment of the disclosure, through steps S101 to S104, when the second distribution parameter and the first distribution parameter satisfy the preset distribution conditions, the predicted sample data is determined as the target data. The target data can be identified as data worth mining and optimizing, and the target data can be used to train the model. In this way, the efficiency of determining the target data and the accuracy of the determination results are improved.

[0103] For example: see Figure 2 , Figure 2 This disclosure provides a technical concept for a model training data generation method. The sample data used in this disclosure is illustrated using image data as an example. Figure 2 As shown, it includes the following steps:

[0104] Step S201: Acquire sample data through a camera;

[0105] Step S202: Store the sample data in the sample pool;

[0106] Step S203: Obtain sample data and determine whether the sample data is the first round data. The first round data can be the first sample data, that is, the labeled data, which can be used to pre-train the model. If it is the first round data, execute step S208; otherwise, execute step S204.

[0107] Step S204: Active learning;

[0108] Step S205: Long-tail categories, obtain target data;

[0109] For details of steps S204 and S205, please refer to the relevant descriptions in the above embodiments.

[0110] Step S206: Whether to continue iterating the model. If iteration is required, return to step S203. If iteration is not required, proceed to step S207.

[0111] Step S207: Model goes online;

[0112] The term "model going online" can be understood as the model being ready for use.

[0113] Step S208: Filter the first round of data and determine the weight and other parameters of the first round of data.

[0114] In this embodiment of the disclosure, by considering the active learning of long-tail categories, the recognition ability of long-tail distribution data can be significantly improved. The aforementioned long-tail distribution data can also be referred to as long-tail category data.

[0115] As an optional implementation, determining the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition includes:

[0116] Calculate the difference in distribution parameters between the second distribution parameter and the first distribution parameter;

[0117] If the difference in the distribution parameters is greater than a second preset difference, the predicted sample data is determined as the target data.

[0118] The difference between the distribution parameters can be obtained by calculating the JS distribution between the second distribution parameter and the first distribution parameter.

[0119] In this embodiment of the disclosure, when the difference in distribution parameters is greater than a second preset difference, it indicates that the predicted sample data is data with greater potential value to be mined. Thus, the predicted sample data can be identified as target data, and the target data can be used to train the model to improve the model's performance.

[0120] As an optional implementation, it also includes:

[0121] The target data is labeled;

[0122] The pre-defined prediction model is trained and updated using the labeled target data.

[0123] It should be noted that, since the existing preset prediction models have large differences in their prediction classification results for target data, it can be considered that the existing preset prediction models have poor recognition performance for target data. Therefore, target data can be considered as data with high value to be mined, and target data can be applied to the training and updating process of preset prediction models to improve the recognition performance of preset prediction models for data.

[0124] In this embodiment of the disclosure, after obtaining the target data, the target data can be labeled to make the labeled target data more accurate and have more dimensions. The labeled target data is then used to train and update the preset prediction model, thereby further enhancing the performance of the preset prediction model.

[0125] For example: see Figure 4 This disclosure illustrates the above content with a specific embodiment, which includes the following steps:

[0126] Step S401: Determine the sample pool, which can also be understood as the entire dataset;

[0127] Step S402: Obtain the first sample data from the sample pool. The first sample data is the labeled data mentioned above.

[0128] Step S403: Obtain the second sample data from the sample pool. The second sample data is the aforementioned unlabeled data.

[0129] Step S404: Calculate the first distribution parameter based on the first sample data;

[0130] Step S405: Input the second sample data into the preset prediction model to predict parameters and obtain the uncertainty parameters of the second sample data;

[0131] Step S406: The second sample data with an uncertainty parameter greater than a preset threshold is determined as the predicted sample data, and the predicted sample data is stored in the pre-selection pool. The pre-selection pool can also be referred to as the unlabeled pre-selection pool.

[0132] Step S407: Obtain the predicted sample data from the pre-selection pool and calculate the second distribution parameter of the predicted sample data;

[0133] Step S408: If the second distribution parameter and the first distribution parameter satisfy the preset distribution conditions, the predicted sample data is determined as the target data;

[0134] Step S409: Label the target data and determine the labeled target data as the first sample data, that is, determine the labeled target data as labeled data, which can be used for model training and updating.

[0135] According to the embodiments disclosed herein, target data can be determined, and the model can be trained and updated based on the target data to improve the performance of the model.

[0136] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a model training data generation device provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the model training data generation device 500 includes:

[0137] The acquisition module 501 is used to acquire first sample data and second sample data from the sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data;

[0138] The calculation and determination module 502 is used to calculate the first distribution parameter of the first sample data and determine the predicted sample data based on the second sample data;

[0139] Calculation module 503 is used to calculate the second distribution parameters of the predicted sample data;

[0140] The determination module 504 is used to determine the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, and the target data is used to train the model.

[0141] Optionally, see Figure 6 The calculation and determination module 502 includes:

[0142] The prediction submodule 5021 is used to input the second sample data into a preset prediction model to perform parameter prediction and obtain the uncertainty parameters of the second sample data. The preset prediction model is a pre-trained model used to perform parameter prediction on the sample data.

[0143] The first determining submodule 5022 is used to determine the second sample data as the predicted sample data when the uncertainty parameter of the second sample data is greater than a preset threshold.

[0144] Optionally, see Figure 7 The preset prediction model includes a feature extractor, a first classifier, and a second classifier. The prediction submodule 5021 includes:

[0145] The feature extraction unit 50211 is used to input the second sample data into the feature extractor to extract features and obtain a first feature vector and a second feature vector.

[0146] The prediction unit 50212 is used to input the first feature vector into the first classifier to perform data prediction and obtain the first predicted classification data, and to input the second feature vector into the second classifier to perform data prediction and obtain the second predicted classification data.

[0147] The determining unit 50213 is used to determine the uncertainty parameter of the second sample data based on the classification difference value between the first predicted classification data and the second predicted classification data.

[0148] Optionally, see Figure 8 The determining unit 50213 includes:

[0149] The calculation subunit 502131 is used to calculate the classification difference value between the first predicted classification data and the second predicted classification data;

[0150] The determination subunit 502132 is used to determine that the uncertainty parameter of the second sample data is greater than the preset threshold when the classification difference value is greater than the first preset difference value.

[0151] Optionally, see Figure 9 The determining module 504 includes:

[0152] The calculation submodule 5041 is used to calculate the difference in distribution parameters between the second distribution parameter and the first distribution parameter;

[0153] The second determining submodule 5042 is used to determine the predicted sample data as target data when the difference in the distribution parameters is greater than a second preset difference.

[0154] Optionally, see Figure 10 The model training data generation device 500 also includes:

[0155] The annotation module 505 is used to annotate the target data;

[0156] The training update module 506 is used to train and update the preset prediction model using the labeled target data.

[0157] The model training data generation apparatus 500 provided in this disclosure can implement all the processes implemented in the model training data generation method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0158] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0159] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0160] like Figure 11 As shown, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from storage unit 1108. The RAM 1103 may also store various programs and data required for the operation of device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0161] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as keyboard, mouse, etc.; output unit 1107, such as various types of monitors, speakers, etc.; storage unit 1108, such as disk, optical disk, etc.; and communication unit 1109, such as network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0162] The computing unit 1101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the model training data generation method. For example, in some embodiments, the model training data generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the model training data generation method described above may be performed. Alternatively, in other embodiments, computing unit 1101 may be configured to perform a model training data generation method by any other suitable means (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0168] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0169] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0170] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating model training data, comprising: Obtain first sample data and second sample data from the sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data; Calculate the first distribution parameter of the first sample data, and determine the predicted sample data based on the second sample data; Calculate the second distribution parameter of the predicted sample data; When the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, the predicted sample data is determined as the target data, and the target data is used to train the model. The step of determining the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition includes: Calculate the JS distribution between the second distribution parameter and the first distribution parameter to obtain the difference in distribution parameters; If the difference in the distribution parameters is greater than a second preset difference, the predicted sample data is determined as the target data.

2. The method according to claim 1, wherein, The step of determining the prediction sample data based on the second sample data includes: The second sample data is input into a preset prediction model for parameter prediction to obtain the uncertainty parameters of the second sample data. The preset prediction model is a pre-trained model used to predict parameters of sample data. If the uncertainty parameter of the second sample data is greater than a preset threshold, the second sample data will be determined as the predicted sample data.

3. The method according to claim 2, wherein, The preset prediction model includes a feature extractor, a first classifier, and a second classifier. The step of inputting the second sample data into the preset prediction model for parameter prediction to obtain the uncertainty parameters of the second sample data includes: The second sample data is input into the feature extractor for feature extraction to obtain a first feature vector and a second feature vector. The first feature vector is input into the first classifier for data prediction to obtain the first predicted classification data, and the second feature vector is input into the second classifier for data prediction to obtain the second predicted classification data. The uncertainty parameter of the second sample data is determined based on the classification difference value between the first predicted classification data and the second predicted classification data.

4. The method according to claim 3, wherein, The step of determining the uncertainty parameter of the second sample data based on the classification difference value between the first predicted classification data and the second predicted classification data includes: Calculate the classification difference value between the first predicted classification data and the second predicted classification data; If the classification difference value is greater than the first preset difference value, the uncertainty parameter of the second sample data is determined to be greater than the preset threshold.

5. The method according to any one of claims 2 to 4, further comprising: The target data is labeled; The pre-defined prediction model is trained and updated using the labeled target data.

6. A model training data generation device, comprising: The acquisition module is used to acquire first sample data and second sample data from the sample pool, wherein the first sample data is labeled data and the second sample data is unlabeled data; The calculation and determination module is used to calculate the first distribution parameter of the first sample data and determine the predicted sample data based on the second sample data. The calculation module is used to calculate the second distribution parameters of the predicted sample data; The determination module is used to determine the predicted sample data as target data when the second distribution parameter and the first distribution parameter satisfy a preset distribution condition, and the target data is used to train the model. The determining module includes: The calculation submodule is used to calculate the JS distribution between the second distribution parameter and the first distribution parameter to obtain the difference in the distribution parameters; The second determining submodule is used to determine the predicted sample data as target data when the difference in the distribution parameters is greater than a second preset difference.

7. The apparatus according to claim 6, wherein, The calculation and determination module includes: The prediction submodule is used to input the second sample data into a preset prediction model to predict parameters and obtain the uncertainty parameters of the second sample data. The preset prediction model is a pre-trained model used to predict parameters of sample data. The first determining submodule is used to determine the second sample data as the predicted sample data when the uncertainty parameter of the second sample data is greater than a preset threshold.

8. The apparatus according to claim 7, wherein, The preset prediction model includes a feature extractor, a first classifier, and a second classifier. The prediction submodule includes: The feature extraction unit is used to input the second sample data into the feature extractor to extract features and obtain a first feature vector and a second feature vector. The prediction unit is used to input the first feature vector into the first classifier to perform data prediction and obtain first predicted classification data, and to input the second feature vector into the second classifier to perform data prediction and obtain second predicted classification data. The determining unit is used to determine the uncertainty parameter of the second sample data based on the classification difference value between the first predicted classification data and the second predicted classification data.

9. The apparatus according to claim 8, wherein, The determining unit includes: A calculation subunit is used to calculate the classification difference value between the first predicted classification data and the second predicted classification data; A sub-unit is defined to determine that, when the classification difference value is greater than a first preset difference value, the uncertainty parameter of the second sample data is greater than the preset threshold value.

10. The apparatus according to any one of claims 7 to 9, further comprising: The annotation module is used to annotate the target data; The training and update module is used to train and update the preset prediction model using labeled target data.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images