A model generation method, device, apparatus, and storage medium

Abstract

The application discloses a model generation method, device and equipment and a storage medium, and belongs to the software technical field.The method comprises the following steps: based on a received new data set, a corresponding test set is extracted; wherein the test set can represent the data distribution of the new data set; based on the data features of the test set, a plurality of similar sample sets, and the corresponding customer data set and stable model of each similar sample set are determined from a preset model library; wherein a plurality of stable models are configured in the model library, the stable model corresponds to a customer data set, and the similar sample set is extracted from the customer data set; a control parameter is calculated based on the similar sample set and the customer data set; wherein the control parameter is used to control the modification of the weight parameter of a certain category of full connection layer in the subsequent steps; and based on the plurality of stable models corresponding to the similar sample set and the control parameter, a stable model corresponding to the new data set is generated.

Description

Technical Field

[0001] This invention relates to a model generation method, apparatus, device, and storage medium, belonging to the field of software technology. Background Technology

[0002] Conventional neural network tasks often address similar client needs. For example, an AI company might have multiple clients in the same industry, all with similar object detection requirements. While serving these diverse clients, the company accumulates a vast amount of data and models. A challenge then becomes: when encountering new clients in the same industry, can the company assist in training their models without disclosing past data?

[0003] New data is often unlabeled, and manual labeling is extremely costly. Trainers or clients can only provide a small amount of labeled data as a test set, leaving a large unlabeled training set as a problem. Self-training uses an existing model to predict the labels for new data, then feeds high-confidence training samples into the new model's training, repeating the prediction and training process. Self-training can alleviate the difficulties of manual labeling to some extent, but it relies on the initial model's ability to predict new data to a certain degree.

[0004] Therefore, how to generate a model with new data using existing training data and a trained model without labeling or with minimal labeling of new data is a technical problem that needs to be solved. Summary of the Invention

[0005] Objective: In view of at least one of the above technical problems, the present invention provides a model generation method, apparatus, device and storage medium that can directly generate a model corresponding to the new dataset based on the existing model and dataset when a new dataset is received, without the need for manual annotation of the new dataset, which can greatly save the manpower and time costs of data annotation and model training.

[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a model generation method, the method comprising:

[0008] Based on the received new dataset, extract the corresponding test set; wherein the test set can characterize the data distribution of the new dataset.

[0009] Based on the data characteristics of the test set, multiple similar sample sets of categories are determined from a preset model library, along with a customer dataset and a stable model corresponding to each similar sample set; wherein, the model library is configured with multiple stable models, each corresponding to a customer dataset, and the similar sample sets are extracted from the customer datasets;

[0010] Control parameters are calculated based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of the weight parameters of a certain category in the fully connected layer in subsequent steps;

[0011] Based on multiple stable models and control parameters corresponding to similar sample sets, a stable model corresponding to the new dataset is generated.

[0012] In some embodiments, the new dataset is a dataset from a new customer or a dataset from an existing customer in a new application scenario.

[0013] In some embodiments, the stable model is represented as a model that can be directly deployed in an application scenario; the customer dataset is a dataset corresponding to the application scenario.

[0014] Furthermore, each stable model is obtained by fine-tuning a pre-trained base model; the training data used for fine-tuning is collected based on the data required for the application scenario.

[0015] In some embodiments, in the model library, each stable model corresponds to a specific customer identifier and a customer dataset; the customer identifier corresponds to an application scenario, and the same customer identifier can correspond to one or more application scenarios; if the same customer identifier corresponds to multiple application scenarios, the customer dataset for each application scenario is different.

[0016] In some embodiments, determining a set of similar samples for multiple categories from a pre-defined model library includes:

[0017] The test set is split according to the classification categories to obtain multiple categories, and each category corresponds to multiple data; the data is image data, text data, or voice data;

[0018] Based on the feature representation of different categories of data, similar customer data are searched from different customer datasets to form a similar sample set.

[0019] In some embodiments, from the perspective of the similarity of feature representations, similar customer data is found from different customer datasets, specifically including:

[0020] The feature vectors of the test set and customer dataset corresponding to the new dataset are mapped to the same feature space to compare the similarity of features;

[0021] If the data is image data, before mapping, the targets in the test set and the customer dataset are cropped based on the annotations to obtain sub-images corresponding to each image containing the target; based on each image in the test set and the sub-images corresponding to each image in each customer dataset, the corresponding image features are generated using the same method to map all image features to the same feature space.

[0022] If the data is text data, word embedding is used to map each word to a continuous vector space, and the semantic similarity between words is converted into the distance between vectors; for text data that is a text segment, the corresponding vector representation is obtained by averaging the word vectors of all words in a text segment; or the text data can be directly input into the BERT pre-trained language model to generate the corresponding text vectors.

[0023] If the data is text data or speech data, the speech data is preprocessed, and features are extracted from the preprocessed speech data to generate speech features; or a deep learning model is used for further feature extraction and mapping; or a pre-trained speech model is directly used to generate the corresponding vector.

[0024] In some embodiments, based on multiple stable models corresponding to similar sample sets and control parameters, a stable model corresponding to a new dataset is generated, including:

[0025] Select one stable model from multiple stable models corresponding to similar sample sets as the initial model;

[0026] The weight parameters of the fully connected layers of each stable model are extracted to obtain multiple weight parameter matrices;

[0027] According to the category, the corresponding learning vector is extracted from each weight parameter matrix, and the learning vector for the corresponding category is calculated in combination with the control parameters;

[0028] The learning vectors of each category are combined to obtain a new weight parameter matrix. The weight parameter matrix of the fully connected layer in the initial model is replaced with the new weight parameter matrix to obtain a stable model corresponding to the new dataset.

[0029] In some embodiments, the control parameters include: the intersection-union ratio (IoU) of the similar sample set and the customer dataset. j And the number S of similar samples covered in the customer dataset j All samples of the corresponding customer dataset j Coverage j According to the category, the corresponding learning vector is extracted from each weight parameter matrix, and the learning vector for the corresponding category is calculated in combination with the control parameters, including:

[0030] V new =∑(iou j *max(0.5,Coverage j ))V j

[0031] In the formula, V new V is the learning vector for the corresponding category. j Let be the learning vector for a certain category in the fully connected layer of the j-th stable model, and be the weight parameters of the column of the weight parameter matrix for this category.

[0032] Furthermore, the intersection-union ratio (IoU) of the similar sample set and the customer dataset... j ,include:

[0033]

[0034] In the formula, KN represents the amount of data in the similar sample set, and Data j This represents the amount of data in the j-th customer dataset, which is a collection of multiple customer datasets.

[0035] Furthermore, the number S of similar samples covered in the customer dataset j All samples of the corresponding customer dataset j Coverage j ,include:

[0036]

[0037] In the formula, S j This represents the amount of data covered in the j-th customer dataset across multiple customer datasets; Data j This represents the amount of data in the j-th customer dataset across multiple customer datasets.

[0038] 8. The model generation method according to claim 1, characterized in that, after generating a stable model corresponding to the new dataset, it further includes:

[0039] The similar sample sets of each category are aggregated as the fine-tuning dataset;

[0040] The stable model corresponding to the new dataset is fine-tuned using the fine-tuning dataset.

[0041] 9. The model generation method according to claim 1, characterized in that, after generating a stable model corresponding to the new dataset, it further includes using a self-training method to improve the accuracy of the stable model corresponding to the new dataset, specifically including:

[0042] Construct training data, wherein the training data includes: (1) fine-tuning the dataset by summarizing similar sample sets of each category; (2) using the stable model corresponding to the new dataset to predict the data in the new dataset, and adding samples with high confidence of the prediction results to the training data;

[0043] The training data is used to train the stable model corresponding to the new dataset, wherein during the training process, the learning parameters of all layers of the model are updated;

[0044] Using the trained model, new data is fed in to predict the data. Samples with high confidence are added to the training data, and the training process is repeated until the preset conditions are met.

[0045] Secondly, the present invention provides a model generation apparatus, the apparatus comprising:

[0046] The extraction module is used to extract a corresponding test set based on the received new dataset; wherein the test set can characterize the data distribution of the new dataset.

[0047] The determination module is used to determine multiple similar sample sets of different categories from a preset model library based on the data features of the test set, as well as the customer dataset and stable model corresponding to each similar sample set; wherein, the model library is configured with multiple stable models, each stable model corresponds to a customer dataset, and the similar sample sets are extracted from the customer dataset;

[0048] The calculation module is used to calculate control parameters based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of the weight parameters of a certain category in the fully connected layer in subsequent steps;

[0049] The generation module is used to generate a stable model corresponding to a new dataset based on multiple stable models and control parameters corresponding to similar sample sets.

[0050] Thirdly, the present invention provides an apparatus comprising,

[0051] Memory;

[0052] processor;

[0053] as well as

[0054] Computer programs;

[0055] The computer program is stored in the memory and configured to be executed by the processor to implement the method described in the first aspect above.

[0056] Fourthly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0057] Beneficial Effects: The model generation method, apparatus, device, and storage medium provided by this invention have the following advantages: Upon receiving a new dataset, this invention can directly generate a model corresponding to the new dataset based on existing models and datasets, without requiring manual annotation of the new dataset. This significantly saves manpower and time costs associated with data annotation and model training. Furthermore, it allows for the reuse of existing models and corresponding datasets without disclosing past data, thus saving resources. Attached Figure Description

[0058] Figure 1 This is a schematic diagram illustrating an application scenario of a model generation method according to an embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram of a model generation method according to an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be used to limit the scope of protection of the present invention.

[0063] like Figure 1 As shown, before detailing the embodiments of the present invention, a specific scenario example is given to illustrate the scenarios in which the technical solution of the present invention can be used. The method of the present invention can be configured in... Figure 1 The server in the system can communicate with multiple clients.

[0064] The client application provides users with a visual interface for model generation and design. It also includes an input interface where users can input new datasets, which can be datasets from new clients or datasets from existing clients for new application scenarios. These new datasets are then transmitted to the server over the network.

[0065] The server has a built-in model library for extracting a corresponding test set based on a received new dataset. Then, based on the data features of the test set, it determines multiple similar sample sets from the pre-defined model library, along with the corresponding customer dataset and stable model. The model library contains multiple stable models, each corresponding to a customer dataset, and the similar sample sets are extracted from the customer dataset. Next, control parameters are calculated based on the similar sample sets and the customer dataset. These control parameters are used to control the modification of weight parameters for a specific category in the fully connected layer in subsequent steps. Finally, based on the multiple stable models corresponding to the similar sample sets and the control parameters, a stable model corresponding to the new dataset is generated. Further, after generating the stable model corresponding to the new dataset, the process includes: summarizing the similar sample sets for each category as a fine-tuning dataset; and using this fine-tuning dataset to fine-tune the stable model corresponding to the new dataset. Alternatively, a self-training method can be used to further improve the accuracy of the stable model.

[0066] The aforementioned server can be an electronic device with a certain computing power. For example, a server can be a distributed system server, or a system with multiple processors, memory, network communication modules, etc., working together. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, a server can also be a new technological means capable of realizing the corresponding functions of the implementation method described in the specification. For example, it can be a new form of "server" based on quantum computing.

[0067] The aforementioned client can be an electronic device with network access capabilities. Specifically, for example, the terminal can be a desktop computer, tablet computer, laptop computer, smartphone, etc. Alternatively, the terminal can also be software that can run on the electronic device.

[0068] The networks described above can be of any type and can use any of the various available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. One or more networks can be local area networks (LANs), Ethernet-based networks, token ring networks, wide area networks (WANs), the Internet, virtual networks, virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0069] Example 1

[0070] Firstly, such as Figure 2 As shown, this embodiment provides a model generation method, including:

[0071] Step S1: Based on the received new dataset, extract the corresponding test set; wherein the test set can characterize the data distribution of the new dataset;

[0072] In this embodiment, the aforementioned new dataset can be a dataset from a new customer or a dataset from an existing customer in a new application scenario. This is because new datasets are often unlabeled images, and unlabeled data cannot be compared in terms of features.

[0073] Therefore, in this embodiment, a portion of the data can be extracted from the new dataset as a test set, which can then be manually labeled, or a standard test set can be provided directly by the client. It should be noted that this test set is capable of representing the data distribution of the new dataset. The specific sampling method used is not limited.

[0074] Step S2: Based on the data features of the test set, determine multiple similar sample sets of different categories from the preset model library, as well as the customer dataset and stable model corresponding to each similar sample set; wherein, the model library is configured with multiple stable models, each stable model corresponds to a customer dataset, and the similar sample sets are extracted from the customer dataset;

[0075] In this embodiment, the model library contains multiple pre-configured stable models. These stable models are those that can be directly deployed in application scenarios, meaning they have extremely high classification accuracy in specific application scenarios. Such models can often be used directly in commercial applications without further fine-tuning.

[0076] In this embodiment, the aforementioned customer dataset corresponds to a dataset specific to an application scenario, rather than a dataset publicly available on the internet. This customer dataset can be directly collected from the customer's application scenario.

[0077] In this embodiment, each stable model can be obtained by fine-tuning a pre-trained base model. The training data used for fine-tuning can be obtained from data collection required for the aforementioned application scenario.

[0078] In this embodiment, the aforementioned basic model can be an object detection network model, an image recognition network model, or, in addition to models in the image domain, models in natural language processing scenarios, such as speech recognition models, text classification models, etc.

[0079] In this implementation, each stable model in the model library can correspond to a specific customer identifier and customer dataset. This customer identifier corresponds to the aforementioned application scenario. The same customer identifier can correspond to multiple application scenarios. For example, customer identifier 001 indicates a customer of a certain bank, and this bank customer has multiple application scenarios, one of which could be an action / behavior recognition application scenario, and another could be a facial recognition application scenario. It is understood that although these two application scenarios correspond to the same customer identifier, their customer datasets should be different.

[0080] Specifically, the aforementioned object detection network models can be R-CNN, Fast R-CNN, YOLO, and NanoDet, etc., while the aforementioned image recognition network models can be VGGNet, ResNet, and DenseNet, etc.

[0081] More specifically, let's take object detection networks as an example. For instance, the basic model mentioned above can be obtained by training conventional object detection networks such as YOLO and NanoDet. Alternatively, a basic model can be obtained by directly acquiring publicly available object detection datasets such as COCO and object365 from the Internet (these training sets are pre-annotated by the resource providers) to pre-train the aforementioned object detection network. It can be understood that this basic model is trained using publicly available object detection datasets. Through multiple rounds of training, the basic model can achieve a high classification accuracy on these publicly available datasets, with a recognition accuracy of around 90%.

[0082] However, these basic models cannot achieve high recognition accuracy in specific application scenarios; they often only achieve a high accuracy rate, such as 70%-80%. This level of accuracy is insufficient for direct deployment in application scenarios and cannot be used commercially.

[0083] Therefore, the basic model needs to be fine-tuned based on the data that needs to be classified in the above application scenarios, so that the fine-tuned basic model also has a high accuracy in specific application scenarios, such as reaching 95% or even 98% or more. Such a model can be called a stable model.

[0084] More specifically, a suitable public dataset can be selected for pre-training of the object detection network. Data of a certain category in a public dataset, such as data labeled "mobile phone," often covers multiple semantics. For example, one semantic state of "mobile phone" could be the semantics expressed by an image of a mobile phone placed still on a plane, while another semantic state could be the semantics expressed by an image of a user using a mobile phone.

[0085] The data samples of these two semantics have feature differences, and the feature differences are quite large, but the pre-trained model cannot reflect these feature differences within the category.

[0086] Unlike publicly available datasets obtained directly from the internet, customer datasets (i.e., datasets corresponding to specific application scenarios, used to fine-tune the base model) often contain data with specific characteristics for a particular category. For example, in applications that recognize user actions, data labeled "mobile phone" often has a narrow semantic meaning, referring to actions such as making calls or taking photos, rather than the phone being stationary on a table. Therefore, compared to publicly available datasets obtained directly from the internet, the range of intra-category feature differences in customer datasets is smaller.

[0087] Therefore, by fine-tuning the training on the customer dataset, the model can be made more targeted to the feature range of the customer dataset.

[0088] It's important to note that during fine-tuning training, all learning layers before the fully connected layers in the neural network need to be frozen. This means that the parameters of convolutional layers and batch-normalized layers should not be updated during fine-tuning; only the parameters of the fully connected layers should be updated. Since the customer dataset, in terms of both categories and the range of features for each category, is smaller than the publicly available dataset, the freezing and fine-tuning training method is sufficient to optimize the model.

[0089] By archiving the finely tuned models (stable models) and their corresponding customer datasets, we can obtain the aforementioned model library. Each stable model and customer dataset in the library targets different data feature directions. This model library will serve as the foundation for model training and optimization for new customers or for existing customers in new application scenarios.

[0090] In this embodiment, the step of determining a similar sample set from a preset model library may include:

[0091] First, the test set needs to be split according to the classification categories, resulting in multiple categories, each corresponding to multiple data points; this data can be image data, text data, or audio data. It's understandable that classification tasks exist for image data, text data, and audio data alike.

[0092] It should be noted that the category refers to the category that needs to be classified in the classification task, that is, the label in the customer dataset, or the output result of the model.

[0093] Then, based on the feature representation of different categories of data, similar customer data are searched from different customer datasets to form a similar sample set.

[0094] Specifically, in this embodiment, similar customer data can be found from different customer datasets from the perspective of feature representation similarity. To characterize feature similarity, the feature vectors of the test set and customer dataset corresponding to the new dataset need to be mapped to the same feature space to compare feature similarity.

[0095] It should be noted that for image data, before mapping, it is necessary to crop out the targets in the test set and the customer dataset based on the annotations (i.e., the bounding boxes). The cropped targets are small images or sub-images.

[0096] Based on the images in the test set and the sub-images corresponding to the images in each customer dataset, the same method is used to generate the corresponding image features, so that all image features are mapped to the same feature space.

[0097] Image features for each sub-image can be generated in various ways. For example, they can be directly input into a classification model, using the output of the last convolutional layer as the image features. This classification model can employ ResNet, VIT, or similar methods.

[0098] In some cases, if the feature map output by the classification model has too high a dimension and is not suitable for comparison, the spatial pooling pyramid can be used to obtain feature vectors as image features.

[0099] It is understandable that although multiple methods can be used to generate image features, considering the need to map the image features of each of the above sub-images to the same feature space, the above-mentioned method of generating corresponding image features using the same method means that only one method can be selected to generate the image features of each sub-image in the test set, as well as the image features of each sub-image in each customer dataset.

[0100] Correspondingly, when the aforementioned data is text or speech data, various methods can be used to map the text or speech data into the same feature space. For text data, word embeddings, such as Word2Vec or GloVe, can be used to map each word to a continuous vector space, and the semantic similarity between words will be converted into the distance between vectors. For text segments, the corresponding vector representation can be obtained by averaging the word vectors of all words in a text segment. Alternatively, the text data can be directly input into a pre-trained language model like BERT, which will generate the corresponding text vectors. In other words, the text vectors are obtained by extracting the output of the intermediate layers of BERT.

[0101] For speech data, compared to text and image data, preprocessing is required. This includes denoising to reduce noise and irrelevant information, followed by feature extraction. For example, the MFCC (Mel-Frequency Cepstral Coefficients) method can be used. The generated speech features can then be further extracted and mapped using deep learning models such as CNN, RNN, LSTM, or Transformer. Similarly, pre-trained speech models can be used to generate corresponding vectors, such as the wav2vec model.

[0102] This implementation method can use the KNN algorithm to find the K feature vectors in the feature space that are closest to each data in each category of the test set.

[0103] Specifically, in the feature space, the KNN algorithm is applied to each data point in each category of the test set, which captures K feature vectors that are similar in features to the data in the test set. Each feature vector represents a data point in the customer dataset.

[0104] K is usually set manually; for example, it can be 100, or it can be adjusted according to the ratio of the customer's training set to the test set.

[0105] More specifically, assuming there are N samples in a certain category in the test set, then for that category, it is necessary to supplement the feature space with KN similar feature vectors, and then find KN similar customer data from the customer dataset based on these KN similar feature vectors.

[0106] These similar customer data often cover different customer datasets. The number of customer data points covered in each dataset is counted. Assuming there are j customer datasets in total, the amount of similar customer data covered in each dataset is S. j .

[0107] The above KN similar samples can be used as the similar sample set of the new dataset. It should be noted that these KN similar samples are only similar samples corresponding to one category in the test set. If there are multiple categories, such as M categories, the data size of the similar sample set should be K*N*M.

[0108] The similar sample sets for each category can be obtained using the methods described above, and will not be elaborated further.

[0109] Understandably, once the similar sample set for each category is determined, it is natural to further determine the customer dataset corresponding to the similar sample set, as well as the stable model.

[0110] Step S3: Calculate control parameters based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of weight parameters of a certain category in the fully connected layer in subsequent steps;

[0111] In this embodiment, the control parameter is used to control the modification of the weight parameters of the fully connected layer for a certain category in subsequent steps. The control parameter includes: the intersection-union ratio (IoU) of the similar sample set and the customer dataset. j And the number S of similar samples covered in the customer dataset j All samples of the corresponding customer dataset j Coverage j .

[0112] One of the parameters is the intersection-union ratio (IoU) between the similar sample set and the customer dataset. j :

[0113]

[0114] In the formula, KN represents the amount of data in the similar sample set, and Data j This represents the data volume of the j-th customer dataset, which is a collection of multiple customer datasets.

[0115] Therefore, iou j This is expressed as the intersection-union ratio (IoU) of the similar sample set and the j-th customer dataset. j The intersection-union ratio (IU) represents the similarity between a certain class of data in the new dataset and various customer datasets. A higher IU indicates higher similarity, and vice versa.

[0116] Another parameter is the number of similar samples S covered in the customer dataset. j All samples of the corresponding customer dataset j The ratio:

[0117]

[0118] In the formula, S j This represents the amount of data covered in the j-th customer dataset across multiple customer datasets; Data j This represents the amount of data in the j-th customer dataset across multiple customer datasets.

[0119] Coverage j This represents the coverage of image features in the new dataset with image features in the j-th customer dataset. The larger the value, the more similar the feature directions of the two datasets are; the smaller the value, the larger the feature range of the customer dataset.

[0120] Note: The two parameters above are discussed according to categories.

[0121] Step S4: Based on multiple stable models and control parameters corresponding to similar sample sets, generate a stable model corresponding to the new dataset.

[0122] In this embodiment, the stable model described above is the stable model corresponding to the similar sample set determined in step S1.

[0123] In this embodiment, the step may include:

[0124] First, select a stable model from multiple stable models corresponding to similar sample sets as the initial model; for example, you can choose one randomly, or you can choose the sample set with the highest similarity to the test set as the initial model.

[0125] Then, the weight parameters of the fully connected layers of each stable model are extracted to obtain multiple weight parameter matrices;

[0126] Next, learning vectors are extracted from each weight parameter matrix according to the specific category, and combined with the control parameters, the merged learning vectors are calculated using the following formula:

[0127] V new =∑(iou j *max(0.5,Coverage j ))V j

[0128] In the formula, V j This is the learning vector for a certain category in the fully connected layer of the j-th stable model, which is also the weight parameter of a certain column in the weight parameter matrix.

[0129] In this formula, via iou j Control V j Usage is important because customer datasets may encompass a wider range of data features, and these features may not necessarily be needed for new datasets; therefore, IOU is required. j For V j Further reduction is required. This formula takes into account the situation where some customer datasets are too large, their features can cover the new dataset, and the proportion of data similar to the new dataset in these customer datasets is too small. Therefore, further reduction is needed, but it is not appropriate to reduce it to less than 0.5.

[0130] Finally, the learning vectors for each category are combined to obtain a new weight parameter matrix, which is then used to replace the weight parameter matrix of the fully connected layer in the initial model. All other model parameters in the initial model remain unchanged except for the fully connected layers.

[0131] Specifically, the role of a fully connected layer can be understood as transforming the feature map output by the last convolutional layer in the network into numerical values ​​for different target categories. Assuming the feature map vector length is 4096 and the number of categories is 100, then the learning parameters of the fully connected layer are a 4096*100 matrix. For this 4096*100 matrix, each column corresponds to a category; in other words, the value in each column controls the network's recognition of that category.

[0132] Therefore, the matrix can be split according to categories, that is, split according to the columns of the matrix. The resulting vectors with a length of 4096 in each column can determine the classification direction of the categories.

[0133] In the model library, the fully connected layers of different base models often have different learning vectors for the same category. For example, if two base models both need to identify "phone", one base model needs to identify a phone placed on a table, while the other base model needs to identify a phone held in the hand. Therefore, the learning vectors of the weight parameter matrices in the fully connected layers of these two base models for the category "phone" are different. More specifically, the learning vectors in the column of the weight parameter matrix for "phone" must be different.

[0134] Therefore, this implementation method combines the learning vectors of the fully connected layers of stable models in the model library to obtain the weight parameter matrix of the fully connected layers. This edited weight parameter matrix then replaces the weight parameter matrix of the fully connected layers in the initial model, resulting in a stable model for the new dataset. This method directly modifies the weight parameter matrix of the fully connected layers, rather than iterating through data training as in traditional training methods, especially for the parameters in the fully connected layers. This avoids labeling the data in the new dataset, saving significant time on manual labeling and model training. Furthermore, for AI companies with large amounts of stable models and customer datasets, this method allows for excellent reuse of already labeled data without concerns about leakage of customer datasets.

[0135] In some cases, the accuracy of the stable model in step S4 can be further improved without manual annotation. For example, the stable model obtained in step S4 can be used as the base model, and the similar sample sets of each category obtained in step S2 can be used as a fine-tuning dataset. This fine-tuning dataset can then be used to fine-tune the stable model obtained in step S4. This method allows the stable model obtained in step S4 to better conform to the feature distribution of the new data.

[0136] In some cases, the accuracy of the stable model in step S4 can be further improved by using self-training methods without manual annotation or with minimal human intervention. For example, training data can be constructed first, consisting of the following components:

[0137] (1) The above-mentioned fine-tuned dataset is also the set of similar samples of each category obtained in step S2.

[0138] (2) Use the stable model obtained in step S4 to predict the data in the new dataset, and add the samples with higher confidence in the prediction results to the training data.

[0139] Then, use the training data described above to train the stable model obtained in step S4. It should be noted that when training the stable model, not only the learning parameters of the fully connected layers need to be updated, but also the learning parameters of other layers, such as the convolutional layers. (This is because when other stable models in the model library are fine-tuned, all layers except the fully connected layers are frozen.)

[0140] The retrained model is then used to predict new data, and samples with high confidence levels are identified. The training process is repeated. During the addition of new data samples, human intervention can be used to screen for error-prone or minority samples.

[0141] After multiple retraining sessions, the self-training process can be terminated. At this point, the method yields a model that can correctly predict new data samples without requiring laborious manual annotation. New data and new models have thus become new case studies in the model market.

[0142] In summary, this invention provides a model generation method that, upon receiving a new dataset, can directly generate a model corresponding to the new dataset based on existing models and datasets, without requiring manual annotation of the new dataset. This significantly reduces the manpower and time costs associated with data annotation and model training. Furthermore, it allows for the reuse of existing models and corresponding datasets without disclosing past data, thus saving resources.

[0143] Example 2

[0144] Secondly, based on Example 1, such as Figure 3 As shown, this embodiment provides a model generation apparatus, including:

[0145] The extraction module is used to extract a corresponding test set based on the received new dataset; wherein the test set can characterize the data distribution of the new dataset.

[0146] The determination module is used to determine multiple similar sample sets of different categories from a preset model library based on the data features of the test set, as well as the customer dataset and stable model corresponding to each similar sample set; wherein, the model library is configured with multiple stable models, each stable model corresponds to a customer dataset, and the similar sample sets are extracted from the customer dataset;

[0147] The calculation module is used to calculate control parameters based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of the weight parameters of a certain category in the fully connected layer in subsequent steps;

[0148] The generation module is used to generate a stable model corresponding to a new dataset based on multiple stable models and control parameters corresponding to similar sample sets.

[0149] Example 3

[0150] Thirdly, based on Example 1, such as Figure 4 As shown, this embodiment provides a device, including,

[0151] Memory;

[0152] processor;

[0153] as well as

[0154] Computer programs;

[0155] The computer program is stored in the memory and configured to be executed by the processor to implement the method described in Embodiment 1.

[0156] Example 4

[0157] Fourthly, based on Embodiment 1, this embodiment provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described in Embodiment 1.

[0158] Example 5

[0159] Fifthly, based on Embodiment 1, this embodiment proposes a hardware system that applies a model generation method from Embodiment 1. This method is applied to the server side.

[0160] A terminal can be an electronic device with network access capabilities. For example, a terminal can be a desktop computer, tablet computer, laptop computer, smartphone, digital assistant, shopping guide terminal, television set, etc.

[0161] A server can be an electronic device with a certain computing power. For example, a server can be a distributed system server, or a system with multiple processors, memory, network communication modules, etc., working together. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, a server can also be a new technological means capable of realizing the corresponding functions of the implementation method described in the manual. For example, it can be a new form of "server" based on quantum computing.

[0162] Example 6:

[0163] Sixthly, based on Embodiment 1, this embodiment also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform a model generation method as described in Embodiment 1.

[0164] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0165] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0166] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0167] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0168] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0169] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0170] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0171] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0172] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0173] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0174] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0175] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this specification, in essence, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A model generation method, characterized in that, The method includes: Based on the received new dataset, extract the corresponding test set; wherein the test set can characterize the data distribution of the new dataset. Based on the data characteristics of the test set, multiple similar sample sets of categories are determined from a preset model library, along with a customer dataset and a stable model corresponding to each similar sample set; wherein, the model library is configured with multiple stable models, each corresponding to a customer dataset, and the similar sample sets are extracted from the customer datasets; Control parameters are calculated based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of the weight parameters of a certain category in the fully connected layer in subsequent steps; Based on multiple stable models and control parameters corresponding to similar sample sets, a stable model corresponding to the new dataset is generated. This includes determining similar sample sets for multiple categories from a pre-defined model library, including: The test set is split according to the classification categories to obtain multiple categories, and each category corresponds to multiple data; the data is image data, text data, or voice data; Based on the feature representation of different categories of data, similar customer data are searched from different customer datasets to form a similar sample set.

2. The model generation method according to claim 1, characterized in that, The new dataset is either a dataset from a new customer or a dataset from an existing customer in a new application scenario. And / or, the stable model is a model that can be directly deployed in the application scenario; the customer dataset is a dataset corresponding to the application scenario; And / or, each stable model is obtained by fine-tuning a pre-trained base model; The training data used for fine-tuning is collected based on the data required for the application scenario; And / or, in the model library, each stable model corresponds to a specific customer identifier and customer dataset; the customer identifier corresponds to an application scenario, and the same customer identifier can correspond to one or more application scenarios; if the same customer identifier corresponds to multiple application scenarios, the customer dataset for each application scenario is different.

3. The model generation method according to claim 1, characterized in that, From the perspective of feature representation similarity, similar customer data is found from different customer datasets, specifically including: The feature vectors of the test set and customer dataset corresponding to the new dataset are mapped to the same feature space to compare the similarity of features; If the data is image data, before mapping, the targets in the test set and the customer dataset are cropped based on the annotations to obtain sub-images corresponding to each image containing the target; based on each image in the test set and the sub-images corresponding to each image in each customer dataset, the corresponding image features are generated using the same method to map all image features to the same feature space. And / or, if the data is text data, a word embedding method is used to map each word to a continuous vector space, and the semantic similarity between words is converted into the distance between vectors; for text data that is a text segment, the corresponding vector representation is obtained by averaging the word vectors of all words in a text segment; or the text data is directly input into the BERT pre-trained language model, and the corresponding text vector is generated by the pre-trained language model. And / or, if the data is text data or speech data, the speech data is preprocessed, and features are extracted from the preprocessed speech data to generate speech features; or a deep learning model is used for further feature extraction and mapping; or a pre-trained speech model is directly used to generate the corresponding vector.

4. The model generation method according to claim 1, characterized in that, Based on multiple stable models corresponding to similar sample sets and control parameters, a stable model corresponding to the new dataset is generated, including: Select one stable model from multiple stable models corresponding to similar sample sets as the initial model; The weight parameters of the fully connected layers of each stable model are extracted to obtain multiple weight parameter matrices; According to the category, the corresponding learning vector is extracted from each weight parameter matrix, and the learning vector for the corresponding category is calculated in combination with the control parameters; The learning vectors of each category are combined to obtain a new weight parameter matrix. The weight parameter matrix of the fully connected layer in the initial model is replaced with the new weight parameter matrix to obtain a stable model corresponding to the new dataset.

5. The model generation method according to claim 1, characterized in that, The control parameters include: the intersection-union ratio of the similar sample set and the customer dataset. and the number of similar samples covered in the customer dataset All samples in the corresponding customer dataset ratio According to the category, the corresponding learning vector is extracted from each weight parameter matrix, and the learning vector for the corresponding category is calculated in combination with the control parameters, including: ， In the formula, This represents the learning vector for the corresponding category. For the first The learning vector for a certain class in the fully connected layer of a stable model is the weight parameter of the column of the weight parameter matrix related to that class.

6. The model generation method according to claim 5, characterized in that, Intersection over Union (IoU) of similar sample sets and customer datasets ,include: ， In the formula, This represents the amount of data in a set of similar samples. This represents the data volume of the j-th customer dataset, which is a collection of multiple customer datasets. And / or, the number of similar samples covered in the customer dataset All samples in the corresponding customer dataset ratio ,include: ， In the formula, This represents the amount of data covered in the j-th customer dataset across multiple customer datasets. This represents the amount of data in the j-th customer dataset across multiple customer datasets.

7. The model generation method according to claim 1, characterized in that, After generating a stable model corresponding to the new dataset, the following steps are also included: The similar sample sets of each category are aggregated as the fine-tuning dataset; The stable model corresponding to the new dataset is fine-tuned using the fine-tuning dataset.

8. The model generation method according to claim 1, characterized in that, After generating a stable model corresponding to the new dataset, the method further includes using self-training to improve the accuracy of the stable model corresponding to the new dataset, specifically including: Construct training data, wherein the training data includes: (1) fine-tuning the dataset by summarizing similar sample sets of each category; (2) using the stable model corresponding to the new dataset to predict the data in the new dataset, and adding samples with high confidence of the prediction results to the training data; The training data is used to train the stable model corresponding to the new dataset, wherein during the training process, the learning parameters of all layers of the model are updated; Using the trained model, new data is fed in to predict the data. Samples with high confidence are added to the training data, and the training process is repeated until the preset conditions are met.

9. A model generation device, characterized in that, The device includes: The extraction module is used to extract a corresponding test set based on the received new dataset; wherein the test set can characterize the data distribution of the new dataset. The determination module is used to determine multiple similar sample sets of different categories from a pre-set model library based on the data features of the test set, as well as the corresponding customer dataset and stable model for each similar sample set; wherein, the model library is configured with multiple stable models, each corresponding to a customer dataset, and the similar sample sets are extracted from the customer datasets; wherein, determining multiple similar sample sets of different categories from the pre-set model library includes: splitting the test set according to the classification categories to obtain multiple categories, each category corresponding to multiple data; the data is image data, text data, or voice data; and based on the feature representation of the data of different categories, searching for similar customer data from different customer datasets to form similar sample sets; The calculation module is used to calculate control parameters based on similar sample sets and customer datasets; wherein the control parameters are used to control the modification of the weight parameters of a certain category in the fully connected layer in subsequent steps; The generation module is used to generate a stable model corresponding to a new dataset based on multiple stable models and control parameters corresponding to similar sample sets.

Images