Data classification method, data classification model training method and device

Abstract

The application discloses a data classification method and a training method and device of a data classification model. The training method of the data classification model comprises the following steps: updating the parameters of a generator for converting data from a source domain to a target domain based on relevant task data in multiple source domains and relevant task data of the target domain; converting and processing each source domain data of a target task by using the generator to obtain each converted data; extracting the features of each converted data; fusing the features of the converted data belonging to the same category of the target task in the multiple source domains to obtain target domain features of each category of the target task; and updating the parameters of the data classification model based on the target domain features of the target task to obtain a trained data classification model. The application can improve the performance of the data classification model.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data classification method, a training method for a data classification model, and an apparatus. Background Technology

[0002] Deep learning, by learning the inherent representation of data, enables algorithms to reach or even surpass human abilities to recognize text, images, and sound. The main processes of current deep learning algorithms include data acquisition, dataset labeling and partitioning, model training and testing, and deployment. During the training of deep learning models, it is typically assumed that the partitioned training and test sets are independent and identically distributed. Methods such as GoogLeNet, ResNet, SENet, the YoLo series, DeepLab V1 / 2 / 3, and UNet are mainstream methods in classification, detection, and semantic segmentation. However, due to the complex and changing testing environment and equipment, the actual test cases may differ from the samples in the training set, leading to a decrease in model performance. Summary of the Invention

[0003] This application provides a data classification method, a training method for a data classification model, and an apparatus, which can improve the performance of the data classification model.

[0004] To achieve the above objectives, this application provides a training method for a data classification model. The training set of this method includes relevant task data in the target domain, relevant task data in multiple source domains, and target task data in multiple source domains. The method includes:

[0005] Based on relevant task data from multiple source domains and relevant task data from the target domain, update the parameters of the generator used to transform data from the source domain to the target domain;

[0006] The generator is used to transform the source domain data of the target task to obtain the transformed data.

[0007] Extract features from each transformed data point;

[0008] The transformation data features belonging to the same category of the target task from multiple source domains are fused to obtain the target domain features of each category of the target task.

[0009] Based on the target domain features of the target task, the parameters of the data classification model are updated to obtain the trained data classification model.

[0010] In one embodiment, updating the parameters of the data classification model based on the target domain features of the target task includes:

[0011] The first differences of multiple source domains are weighted to obtain the feature difference loss, wherein the first difference of each source domain is obtained by utilizing the differences between the data features of each source domain and the features of the target domain in at least some categories of the target task;

[0012] The parameters of the data classification model are updated based on feature difference loss.

[0013] In one embodiment,

[0014] The weight of the first difference in each source domain is positively correlated with the similarity between each source domain and the target domain.

[0015] In one embodiment, parameters are shared among generators corresponding to multiple source domains.

[0016] Based on relevant task data from multiple source domains and relevant task data from the target domain, the parameters of the generator used to transform data from the source domain to the target domain are updated, including: calculating a first adversarial loss based on the discrimination results corresponding to the source domain data of the relevant tasks and the discrimination results of the relevant task data in the target domain; updating the parameters of the generator based on the first adversarial loss; wherein, the discrimination results corresponding to each source domain data of the relevant tasks are obtained by sequentially processing the source domain data of the relevant tasks through generator transformation and discriminator discrimination; the discrimination results of each relevant task data in the target domain are obtained by processing the relevant task data in the target domain through discriminator discrimination.

[0017] The first differences from multiple source domains are weighted to obtain the feature difference loss, including:

[0018] The weight of the first difference of each source domain is calculated using the first adversarial loss corresponding to each source domain, where the first adversarial loss corresponding to each source domain is negatively correlated with the similarity between each source domain and the target domain.

[0019] In one embodiment, a weighted summation of first differences among multiple source domains is performed to obtain a feature difference loss, including:

[0020] The feature difference loss is calculated based on the weighted result of the first difference of multiple source domains and the difference between the data features of different source domains in each category of the target task.

[0021] In one embodiment, based on relevant task data in multiple source domains and relevant task data in the target domain, the parameters of the generator used to transform data from the source domain to the target domain are updated, including:

[0022] The first adversarial loss is calculated based on the discrimination results of the source domain data corresponding to the relevant tasks and the discrimination results of the relevant task data of the target domain. The second adversarial loss is calculated based on the discrimination results of the source domain data corresponding to the target tasks and the discrimination results of the relevant task data of the target domain.

[0023] The generator parameters are updated based on the first and second adversarial losses;

[0024] Specifically, the discrimination results corresponding to each source domain data of the target task are obtained by sequentially processing the source domain data of the target task through generator transformation and discriminator discrimination; the discrimination results corresponding to each source domain data of related tasks are obtained by sequentially processing the source domain data of related tasks through generator transformation and discriminator discrimination; and the discrimination results of each related task data of the target domain are obtained by processing the related task data of the target domain through discriminator discrimination.

[0025] In one embodiment, based on relevant task data in multiple source domains and relevant task data in the target domain, the parameters of the generator used to transform data from the source domain to the target domain are updated, including:

[0026] Calculate the first retention loss, the second retention loss, and the third retention loss, where the first retention loss is the generator's retention loss of the structural features of the source domain data of the relevant task, the second retention loss is the generator's retention loss of the structural features of the target domain data of the relevant task, and the third retention loss is the generator's retention loss of the structural features of the source domain data of the target task.

[0027] The generator parameters are updated based on the first adversarial loss, the second adversarial loss, the first retention loss, the second retention loss, and the third retention loss.

[0028] In one embodiment, the generator parameters are updated based on the first adversarial loss, the second adversarial loss, the first retention loss, the second retention loss, and the third retention loss, including:

[0029] Update the parameters of the generator and discriminator based on the first and second adversarial losses;

[0030] The generator parameters are updated again based on the first retention loss, the second retention loss, and the third retention loss.

[0031] In one embodiment, the data classification model includes a feature extraction unit and a classifier.

[0032] Extracting features from each transformed data, including: extracting features from each transformed data using a feature extraction unit;

[0033] Based on the target domain features of the target task, update the parameters of the data classification model, including:

[0034] The target domain features of the target task are classified using a classifier.

[0035] Based on the classification results of the target domain features, calculate the classification loss;

[0036] The parameters of the data classification model are updated based on the classification loss.

[0037] In one embodiment, the feature extraction unit and the feature extraction module in the generator have the same structure and parameters.

[0038] To achieve the above objectives, this application also provides a data classification method, which includes:

[0039] Obtain the data to be classified;

[0040] The data classification model trained using the above-described data classification model training method is used to classify the data to be classified.

[0041] To achieve the above objectives, this application also provides an electronic device including a processor; the processor is configured to execute instructions to implement the above methods.

[0042] To achieve the above objectives, this application also provides a computer-readable storage medium for storing instruction / program data that can be executed to implement the above methods.

[0043] This application makes full use of knowledge from multiple source domains, learns the mapping from each source domain to the target domain by using multiple related task source domains and target domains, and then sequentially maps, extracts features, and fuses features from multiple source domain samples under the same category of the target task to obtain the target domain features of the target task. This makes the generated target domain features of the target task closer to the real sample features, thereby improving the data processing accuracy of the classification model trained using the generated target domain features of the target task. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 This is a flowchart illustrating one implementation method of the data classification model of this application;

[0046] Figure 2 This is a schematic diagram of the training set in the training method of the data classification model of this application;

[0047] Figure 3 This is a schematic diagram of the feature vector sampling process when calculating the retention loss in the training method of the data classification model of this application;

[0048] Figure 4 This is a schematic diagram of a training architecture used in the training method of the data classification model in this application;

[0049] Figure 5 This is a schematic diagram of the data classification model in the data classification method of this application;

[0050] Figure 6 This is a flowchart illustrating one implementation method of the data classification method of this application;

[0051] Figure 7 This is a schematic diagram of one embodiment of the electronic device of this application.

[0052] Figure 8 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application. In addition, unless otherwise specified (e.g., "or additionally" or "or in alternatives"), the term "or" as used herein refers to a non-exclusive "or" (i.e., "and / or"). Furthermore, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0054] The relevant technology learns the mapping from the source domain to the target domain through relevant data, so that the source domain samples of the target task can obtain a representation close to the target domain of the target task through this mapping. In this way, the training model of the relevant technology can learn the ability to process the target domain of the target task without needing the target domain dataset of the target task.

[0055] However, when there are differences between the source and target domain data and it is difficult to obtain the target domain data in advance, the data processing accuracy of the model trained by this related technology is low.

[0056] Based on this, this application proposes a training method for a classification model. This training method fully utilizes knowledge from multiple source domains, learns the mapping from each source domain to the target domain using multiple related task source domain and target domain data pairs, and then uses the prototypes of the mapped target task source domain samples as the generated target task target domain features, making the generated target task target domain features closer to the real sample features. This can improve the data processing accuracy of the trained model.

[0057] like Figure 1As shown, the training method for the data classification model in this embodiment includes the following steps. It should be noted that the step numbers are for simplification only and are not intended to limit the execution order of the steps. The execution order of each step in this embodiment can be arbitrarily changed without departing from the technical concept of this application.

[0058] S101: Based on the relevant task data in multiple source domains and the relevant task data in the target domain, update the parameters of the generator used to transform data from its corresponding source domain to the target domain.

[0059] Based on the relevant task data of each source domain and the relevant task data of the target domain, the parameters of the generator used to transform data from each source domain to the target domain can be updated so that the generator corresponding to each source domain can be used to transform each sample data of each source domain of the target task into each transformed data of each source domain. Then, the transformed data of the same category in the target task from multiple source domains can be used to determine the target domain features of the same category in the target task, which also facilitates the subsequent training of the data classification model using the target domain features of each category in the target task.

[0060] The training set of the data classification model training method in this application may include relevant task data in the target domain, relevant task data in multiple source domains, and target task data in multiple source domains.

[0061] The data can be audio, images, and / or video, etc., without any restrictions.

[0062] When the data is an image, the difference between the source and target domains lies in the different image acquisition methods. For example, ... Figure 2 As shown, the training set includes data from two source domains and a target domain. One source domain consists of sketch images, and the other consists of visible light images. The target domain consists of line drawings.

[0063] When the data is audio, the difference between the source and target domains lies in the different ways the audio is collected. For example, the training set includes data from two source and target domains: one source domain is cat audio, the other is dog audio, and the target domain is human audio.

[0064] It is understood that the objective task of this application is the classification task of the data classification model of this application. For example, if the classification model of this application is a plant classification model, then the objective task is also a plant classification model. Similarly, if the classification model of this application is an animal classification model, then the objective task is also an animal classification model.

[0065] In this application, the data related to the relevant tasks is only used to provide training data for the generator, so the types of relevant tasks are not limited. In this case, in step S101, the generator can be trained using data from one relevant task or data from multiple relevant tasks. Furthermore, the data for the relevant tasks may not have specific labels for their corresponding tasks. For example, if the relevant task is a plant classification task, then neither the source domain data nor the target domain data of the relevant task need to be labeled with plant categories.

[0066] The generator of this application may include a feature extraction module and a transformation module. The feature extraction module is used to extract features from each source domain data, and the transformation module is used to transform each source domain data feature into a transformed image for each source domain data. In a specific example, assuming the data is an image, the feature extraction module is used to extract features from each source domain image, and the transformation module is used to transform each source domain image feature into a transformed image for each source domain data.

[0067] The structure of the generator is unrestricted. For example, the generator G can be an encoder-decoder structure. That is, G can be represented as an encoder G. enc With decoder G dec That is, G = G dec (G enc (x)), the encoder can be used as a feature extraction module, and the decoder can be used as a transformation module. The encoder can be composed of multiple residual blocks, and the decoder can be composed of transposed convolutional layers connected together.

[0068] In one feasible approach, a generator for transforming data from each source domain to the target domain can be trained based on relevant task data for each source domain and relevant task data for the target domain.

[0069] Alternatively, adversarial training can be used to update the parameters of the generator used to transform data from each source domain to the target domain.

[0070] In a specific example of this implementation, the generator corresponding to each source domain can be used to transform the relevant task data of each source domain into intermediate data of each source domain; the discriminator can be used to determine whether the intermediate data of each source domain is generated data or real data, and the discrimination result of the intermediate data of each source domain can be obtained; the discriminator can also be used to determine whether the relevant task data of the target domain is generated data or real data, and the discrimination result of the relevant task data of the target domain can be obtained; the first adversarial loss of each source domain is calculated based on the discrimination results of the intermediate data of each source domain and the discrimination results of the relevant task data of the target domain; the parameters of the generator and discriminator corresponding to each source domain are updated using the first adversarial loss of each source domain, thus training the generator corresponding to each source domain.

[0071] Among them, the first confrontation loss The calculation formula can be:

[0072]

[0073] Where D(x) rt ) represents the discrimination results of various relevant task data in the target domain; D(G(x) rs )) represents the discrimination results of each intermediate data in the source domain.

[0074] In another specific example of this implementation, the generator parameters for multiple source domains are shared, that is, the parameters of the generators for different source domains are the same. In this case, the total first adversarial loss can be calculated based on the discrimination results of all intermediate data of multiple source domains and the discrimination results of all relevant task data of the target domain. The parameters of the generator and discriminator are updated using the total first adversarial loss, thus training the generator for each source domain.

[0075] In another specific example of this implementation, to ensure that the generator retains the main content information of the data during the transformation of the source domain data, a first retention loss can be calculated using the features generated by the generator from processing relevant task data in the source domain; then, the parameters of the generator are updated using the first retention loss and the first adversarial loss. Specifically, in step S101, updating the parameters of the generator using the first retention loss and the first adversarial loss can be manifested as: calculating a first total loss based on the first retention loss and the first adversarial loss, and updating the parameters of the generator and the discriminator using the first total loss. Alternatively, updating the parameters of the generator using the first retention loss and the first adversarial loss can be manifested as: first updating the parameters of the generator and the discriminator using the first adversarial loss; then updating the parameters of the generator using the first retention loss.

[0076] In another specific example of this implementation, to improve the training effect of the generator, a second retention loss can be calculated using the features generated by the generator from processing relevant task data in the target domain; then, the parameters of the generator are updated using the first retention loss, the second retention loss, and the first adversarial loss. Specifically, in step S101, updating the parameters of the generator using the first retention loss, the second retention loss, and the first adversarial loss can be manifested as: calculating a first total loss based on the first retention loss, the second retention loss, and the first adversarial loss, and updating the parameters of the generator and the discriminator using the first total loss. Alternatively, updating the parameters of the generator using the first retention loss, the second retention loss, and the first adversarial loss can be manifested as: first updating the parameters of the generator and the discriminator using the first adversarial loss; then updating the parameters of the generator using the first retention loss and the second retention loss.

[0077] Among them, the first retention loss The calculation formula is as follows:

[0078]

[0079] Second retention loss The calculation formula is as follows:

[0080]

[0081] in, This indicates that the generated data has passed through the encoder G in the generator. enc The query vector obtained by sampling at layer l is then sampled at different feature layers in the encoder. and These represent the data after passing through encoder G. enc The positive vector and N negative vectors obtained from sampling at layer l, S l This represents the number of spatial locations in the l-th layer feature map, i.e., the width × height of the l-th layer feature map. The feature vector sampling process is as follows: Figure 3 As shown, the mutual information between the input data and the output data after passing through the generator is maximized through multi-source PatchNCE loss, so that the generator can retain the main content information of the data during the transformation of the source domain data.

[0082] In another possible approach, the parameters of the generator used to transform data from each source domain to the target domain can be updated based on the relevant task data for each source domain, the target task data for each source domain, and the relevant task data for the target domain.

[0083] In a specific example of this implementation, a generator corresponding to each source domain can be used to convert the relevant task data of each source domain into relevant intermediate data of each source domain; a discriminator can be used to determine whether the intermediate data of each source domain is generated data or real data, obtaining the discrimination result of the intermediate data of each source domain; a generator corresponding to each source domain can be used to convert the target task data of each source domain into target intermediate data of each source domain; a discriminator can be used to determine whether the target intermediate data of each source domain is generated data or real data, obtaining the discrimination result of the target intermediate data of each source domain; a discriminator can also be used to determine whether the relevant task data of the target domain is generated data or real data. The data is used to obtain the discrimination results of various related task data in the target domain; the first adversarial loss for each source domain is calculated based on the discrimination results of each intermediate data in each source domain and the discrimination results of each related task data in the target domain; the second adversarial loss is calculated based on the discrimination results of each intermediate data in each source domain and the discrimination results of each related task data in the target domain; the parameters of the generator and discriminator corresponding to each source domain are updated using the first and second adversarial losses of each source domain, thus training the generator corresponding to each source domain, so that the generator can be trained with the help of target task data, thereby enabling the generator trained on related task data to be efficiently applied to the transformation of source domain data of the target task.

[0084] Among them, the second resistance loss The calculation formula can be:

[0085]

[0086] Where D(x) rt ) represents the discrimination results of various relevant task data in the target domain; D(G(x) ts )) represents the discrimination results of intermediate data of each target in the source domain.

[0087] In another specific example of this implementation, the generator parameters for multiple source domains are shared, that is, the parameters of the generators for different source domains are the same. In this case, the total first adversarial loss and the total second adversarial loss can be calculated based on the discrimination results of all relevant intermediate data of multiple source domains, the discrimination results of all target intermediate data of multiple source domains, and the discrimination results of all relevant task data of the target domain. The parameters of the generator and the discriminator are updated using the total first adversarial loss and the second adversarial loss, thus training the generator for each source domain.

[0088] In another specific example of this implementation, to ensure that the generator retains the main content information of the data during the transformation of source domain data, a first retention loss can be calculated using the features generated by the generator from processing relevant task data in the source domain; then, the generator parameters are updated using the first retention loss, the first adversarial loss, and the second adversarial loss. Specifically, in step S101, updating the generator parameters using the first retention loss, the first adversarial loss, and the second adversarial loss can be manifested as: calculating a first total loss based on the first retention loss, the first adversarial loss, and the second adversarial loss, and updating the parameters of the generator and the discriminator using the first total loss. Alternatively, updating the generator parameters using the first retention loss, the first adversarial loss, and the second adversarial loss can be manifested as: first updating the parameters of the generator and the discriminator using the first adversarial loss and the second adversarial loss; then updating the generator parameters using the first retention loss.

[0089] In another specific example of this implementation, to improve the training effect of the generator, a second retention loss can be calculated using the features generated by the generator from processing relevant task data in the target domain; then, the parameters of the generator are updated using the first retention loss, the second retention loss, the first adversarial loss, and the second adversarial loss. Specifically, in step S101, updating the parameters of the generator using the first retention loss, the second retention loss, the first adversarial loss, and the second adversarial loss can be manifested as: calculating a first total loss based on the first retention loss, the second retention loss, the first adversarial loss, and the second adversarial loss, and updating the parameters of the generator and the discriminator using the first total loss. Alternatively, updating the parameters of the generator using the first retention loss, the second retention loss, the first adversarial loss, and the second adversarial loss can be manifested as: first updating the parameters of the generator and the discriminator using the first adversarial loss and the second adversarial loss; then updating the parameters of the generator using the first retention loss and the second retention loss.

[0090] In another specific example of this implementation, to improve the training effect of the generator, a third retention loss can be calculated using the features generated by the generator processing the target task data of each source domain; then, the parameters of the generator are updated using the first retention loss, second retention loss, third retention loss, first adversarial loss, and second adversarial loss. Specifically, in step S101, updating the parameters of the generator using the first retention loss, second retention loss, third retention loss, first adversarial loss, and second adversarial loss can be manifested as: calculating a first total loss based on the first retention loss, second retention loss, third retention loss, first adversarial loss, and second adversarial loss, and updating the parameters of the generator and discriminator using the first total loss. Alternatively, updating the parameters of the generator using the first retention loss, second retention loss, third retention loss, first adversarial loss, and second adversarial loss can be manifested as: first updating the parameters of the generator and discriminator using the first adversarial loss and second adversarial loss; then updating the parameters of the generator using the first retention loss, second retention loss, and third retention loss.

[0091] Among them, the third retention loss The calculation formula is as follows:

[0092]

[0093] in, This indicates that the generated data has passed through encoder G in the generator. enc The query vector obtained by sampling at layer l is then sampled at different feature layers in the encoder. and These represent the data after passing through encoder G. enc The positive vector and N negative vectors obtained from sampling at layer l, S l This represents the number of spatial locations in the l-th layer feature map, i.e., the width × height of the l-th layer feature map. The feature vector sampling process is as follows: Figure 3 As shown, the mutual information between the input data and the output data after passing through the generator is maximized through multi-source PatchNCE loss, so that the generator can retain the main content information of the data during the transformation of the source domain data.

[0094] The discriminator structure described above is not limited. In one example, the discriminator can consist of a series of fully connected layers.

[0095] S102: Use the generator to transform the source domain data of the target task to obtain the transformed data.

[0096] Based on the relevant task data of each source domain and the relevant task data of the target domain, after updating the generator parameters, the generator of each source domain can be used to transform each sample data of each source domain of the target task into each transformed data of each source domain. This allows the target domain features of the target task to be generated based on each transformed data of each source domain, which in turn facilitates the training of the classification module based on the target domain features of the target task.

[0097] S103: Extract the features of each transformed data.

[0098] Optionally, the feature extraction unit can be used to extract features from each transformed data in each source domain.

[0099] The structure of the feature extraction unit is not restricted; for example, it can be an encoder structure or a regular convolutional structure.

[0100] In one implementation, the feature extraction unit and the feature extraction module in the generator are independent of each other and do not affect each other. Furthermore, the structures of the feature extraction unit and the feature extraction module in the generator can be different.

[0101] In another implementation, the feature extraction unit and the feature extraction module in the generator have identical structures and parameters; that is, the feature extraction unit and the feature extraction module in the generator share parameters. Thus, when the parameters in the feature extraction module change, the parameters in the feature extraction unit will also change accordingly; conversely, when the parameters in the feature extraction unit change, the parameters in the feature extraction module will also change accordingly.

[0102] S104: Merge the transformation data features of the same category belonging to the target task from multiple source domains to obtain the target domain features of each category of the target task.

[0103] After extracting the features of each transformation data in each source domain, the transformation data features of multiple source domains belonging to the same category of the target task can be fused to obtain the target domain features of the same category of the target task. In this way, the target domain features of each category of the target task can be obtained.

[0104] Optionally, the weighted average of transformation data features belonging to the same category of the target task from multiple source domains can be used as the target domain feature of the same category for the target task. In this way, transformation data features belonging to the same category of the target task from multiple source domains can be fused through weighted averaging. The weights of the transformation data features of each source domain can be set according to the actual situation and are not limited here.

[0105] For example, if the weights of the transformed data features from each source domain are equal, then the transformed data features from multiple source domains belonging to the same category of the target task can be fused by averaging.

[0106] For example, the weights of each source domain can be determined based on the similarity between each source domain and the target domain, where the weights of each source domain can be positively correlated with the similarity between each source domain and the target domain. The similarity between each source domain and the target domain can be obtained using the first adversarial loss of each source domain. Optionally, the similarity between each source domain and the target domain can be negatively correlated with the first adversarial loss of each source domain.

[0107] Optionally, a flattening layer can be used to fuse transformation data features belonging to the same category in multiple source domains that are part of the target task.

[0108] S105: Update the parameters of the data classification model based on the target domain features of the target task.

[0109] After fusing the transformation data features of multiple source domains belonging to the same category of the target task to obtain the target domain features of each category of the target task, the parameters of the classification model can be updated based on the target domain features of the target task. In this way, the mapping from each source domain to the target domain is learned by using multiple related task source domains and target domains. Then, multiple source domain samples under the same category of the target task are sequentially mapped, feature extracted and feature fused to obtain the target domain features of the target task. This makes the generated target domain features of the target task closer to the real sample features, which can improve the data processing accuracy of the classification model trained using the generated target domain features of the target task.

[0110] In one implementation, the parameters of the classification model can be updated based on the target domain features of the target task and the transformed data features of multiple source domains.

[0111] Optionally, the classification loss of the classification model can be calculated using the target domain features of the target task and the transformed data features of multiple source domains; then the classification loss can be used to update the parameters of the classification model.

[0112] Among them, the classification loss L is calculated. C The classification loss function used is unrestricted. For example, the classification loss function can be a multi-class loss function and a cross-entropy loss function as shown below.

[0113]

[0114] in, This represents the classification result for the i-th sample.

[0115] Optionally, the classification model may include a feature extraction unit and a classification module.

[0116] In step S105, the classification module of the classification model can be used to classify the target domain features of the target task and the transformation data features of each source domain, obtaining the classification results of the target domain features of the target task and the classification results of the transformation data features of each source domain. Based on the classification results of the target domain features of the target task, the classification results of the transformation data features of each source domain, the target task category to which the target domain features of the target task belong, and the target task category to which the transformation data features of each source domain belong, the classification loss is calculated. The parameters of the classification model are updated based on the classification loss. Specifically, the target task category to which each transformation data feature of each source domain belongs is equivalent to the target task category to which the corresponding transformation data belongs. The target task category to which each target domain feature of the target task belongs is equivalent to the target task category to which its corresponding target task data belongs.

[0117] In another implementation, the parameters of the classification model can be updated based on the target domain features of the target task and the target task data from multiple source domains.

[0118] Optionally, the classification loss of the classification model can be calculated using the target domain features of the target task and the target task data from multiple source domains; then, the classification loss can be used to update the parameters of the classification model.

[0119] Optionally, the classification module of the classification model can be used to classify the target domain features of the target task and the target task data features of each source domain, obtaining the classification results of the target domain features of the target task and the classification results of the target task data features of each source domain. Based on the classification results of the target domain features of the target task, the classification results of the target task data features of each source domain, the target task category to which the target domain features of the target task belong, and the target task category to which the target task data features of each source domain belong, a classification loss is calculated. The parameters of the classification model are then updated based on the classification loss. Here, the target task category to which each target domain feature of the target task belongs is equivalent to the target task category to which its corresponding target task data belongs.

[0120] Furthermore, feature difference loss can be calculated based on the differences between source domain data features and target domain features in each category of the target task; and the parameters of the classification model can be updated based on the classification loss and feature difference loss. In this way, updating the parameters of the classification model through feature difference loss can make the target domain features extracted by the classification model more similar to the source domain data features of its corresponding category. This allows the feature extraction unit in the classification model to extract the common features of the target domain data and the source domain data, which is beneficial for the classification model to be applicable to the classification processing of data in various domains and improves the data processing effect of the classification model.

[0121] When calculating the feature difference loss, the similarity between each source domain and the target domain can be considered. This allows for dynamic adjustment of the influence of each source domain on the network during the alignment of source domain prototypes with the target domain prototypes. Consequently, source domain prototypes that are closer to the target domain distribution are assigned higher weights, increasing their influence on the classification model and making it better suited for target domain data processing tasks. Specifically, the differences between the data features of each source domain and the features of the target domain in the target task can be weighted to obtain the feature difference loss. The weight of the difference between the data features of each source domain and the features of the target domain can be positively correlated with the similarity between each source and target domain.

[0122] Optionally, the similarity between each source domain and the target domain can be obtained using the first adversarial loss of each source domain. In this case, the similarity between each source domain and the target domain can be negatively correlated with the first adversarial loss of each source domain, thus the weight of the difference between the data features of each source domain and the features of the target domain can be negatively correlated with the first adversarial loss of each source domain. Wherein, the first adversarial loss L of the i-th source domain is used. adv_i Calculate the similarity w between the i-th source domain and the target domain. i The calculation formula is as follows:

[0123]

[0124] Where m is the total number of source domains in the training set.

[0125] In other embodiments, the similarity between each source domain and target domain can also be obtained by calculating the spatial distance between the data features of each source domain and target domain.

[0126] Furthermore, when calculating the feature difference loss, the differences between data features from different source domains in each category of the target task can also be considered. Optionally, the feature difference loss can be calculated based on the differences between source domain data features and target domain features in each category of the target task, as well as the differences between data features from different source domains in each category of the target task.

[0127] Alternatively, the specific formula for calculating the feature difference loss can be as follows:

[0128]

[0129] in, Let represent the data features of category c in the i-th source domain, the j-th source domain, and the target domain, respectively.

[0130] In this scenario, the feature extractor and feature extraction unit used to extract features from the source domain data can have the same structure and parameters. That is, the feature extractor and the feature extraction unit used to extract features from the target domain share parameters. This allows for better utilization of feature difference loss, enabling the feature extraction unit to extract common features from both the target and source domain data. In a specific example, the feature extractor is the feature extraction module in the generator. This means that the feature extraction module and the feature extraction unit in the generator share parameters. Thus, during the generator's transformation of each source domain data to generate a transformed image, the feature extraction module in the generator can input the features extracted from each source domain data into the classifier. The classifier then classifies the corresponding source domain data based on these features, obtaining the classification result for the source domain data corresponding to that feature. By using the feature extraction module in the generator as the feature extractor for extracting features from the source domain data, the training architecture of this training method can be simplified, and repeated feature extraction from the source domain data can be avoided, reducing the amount of data processing during training and improving training efficiency. In another specific example, this feature extractor can also be the feature extraction unit in a classification model.

[0131] In this embodiment, the knowledge of multiple source domains is fully utilized. The mapping from each source domain to the target domain is learned by using multiple related task source domains and target domains. Then, multiple source domain samples under the same category of the target task are sequentially mapped, feature extracted, and feature fused to obtain the target domain features of the target task. This makes the generated target domain features of the target task closer to the real sample features, thereby improving the data processing accuracy of the classification model trained using the generated target domain features of the target task.

[0132] In use Figure 4 In the embodiment of training the classification model using the training architecture shown, adversarial training can be performed based on relevant task data from multiple source domains and relevant task data from the target domain. This updates the parameters of the generator G and discriminator, which are used to transform data from their corresponding source domains to the target domain. The multi-source PatchNCE loss (i.e., the first retention loss, second retention loss, and third retention loss mentioned above) is calculated to update the parameters of the generator G. Finally, the classification loss and feature difference loss are calculated to update the classifier C and the feature extraction unit G. enc By repeating the above process with the parameters set, a multi-source domain zero-shot adaptive network is finally obtained, which includes a classifier C and a feature extraction unit G. enc Data classification model.

[0133] In addition, for each training batch, k samples and k target task source domain data are taken from the relevant task source domain dataset as input to the generator, and mk generated relevant task target domain data and mk generated target task target domain data are obtained. These, together with the real relevant task target domain data, are used as input to the discriminator, so that the parameters of the discriminator and the generator G are updated through the output of the discriminator.

[0134] In passing Figure 4 After training the data classification model using the training architecture shown, as follows: Figure 5 As shown, the feature extraction unit G can be... enc It is separated from classifier C and used as an independent data classification model.

[0135] This application also provides a data classification method, such as Figure 6 As shown, the data classification method may include the following steps. It should be noted that the step numbers are for simplification only and are not intended to limit the execution order of the steps. The execution order of the steps in this embodiment can be arbitrarily changed without departing from the technical concept of this application.

[0136] S201: Obtain the data to be classified.

[0137] The data to be classified can be images, audio, text, or video, etc., without any restrictions.

[0138] S202: Use the data classification model trained using the above data classification model training method to classify the data to be classified.

[0139] The data classification model can include a feature extraction unit and a classifier. The feature extraction unit can extract the features of the data to be classified, and then the classifier can process the features of the data to be classified to obtain the classification result of the data to be classified.

[0140] The structure of the classifier is unrestricted. For example, a classifier can consist of a series of fully connected layers.

[0141] Please see Figure 7 , Figure 7 This is a schematic diagram of one embodiment of the electronic device 20 of this application. The electronic device 20 of this application includes a processor 22, which is used to execute instructions to implement the methods provided by any of the above embodiments of this application and any non-conflicting combinations thereof.

[0142] Processor 22 can also be referred to as CPU (Central Processing Unit). Processor 22 may be an integrated circuit chip with signal processing capabilities. Processor 22 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor can be a microprocessor, or processor 22 can be any conventional processor, etc.

[0143] The electronic device 20 may further include a memory 21 for storing instructions and data required for the processor 22 to run.

[0144] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. The computer-readable storage medium 30 in this embodiment stores instruction / program data 31. When executed, this instruction / program data 31 implements the methods provided in any embodiment of the above-described method of this application, as well as any non-conflicting combination thereof. The instruction / program data 31 can be formed into a program file and stored in the storage medium 30 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium 30 includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or devices such as computers, servers, mobile phones, and tablets.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0148] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A data classification method, characterized in that, The method includes: Obtain the data to be classified; The data classification model trained using the data classification training method is used to classify the data to be classified. The training set includes relevant task data in the target domain, relevant task data in multiple source domains, and source domain data for the target task. The relevant task data is used to train the generator; the source domain data for the target task is used to train the data classification model; the data is audio, image, and / or video. The data classification training method includes: Based on the relevant task data in the multiple source domains and the relevant task data in the target domain, update the parameters of the generator used to transform data from the source domain to the target domain; The generator is used to transform the source domain data of the target task to obtain transformed data. Extract the features of each of the transformed data; The transformation data features belonging to the same category of the target task from the multiple source domains are fused to obtain the target domain features of each category of the target task. Based on the target domain features of the target task, the parameters of the data classification model are updated to obtain the trained data classification model.

2. The data classification method according to claim 1, characterized in that, Updating the parameters of the data classification model based on the target domain features of the target task includes: The first differences of the plurality of source domains are weighted to obtain a feature difference loss, wherein the first difference of each source domain is obtained by utilizing the difference between the data features of each source domain and the features of the target domain in at least a portion of the categories of the target task; Based on the feature difference loss, the parameters of the data classification model are updated; The weight of the first difference of each source domain is positively correlated with the similarity between each source domain and the target domain.

3. The data classification method according to claim 2, characterized in that, The generators corresponding to the multiple source domains share parameters. The step of updating the parameters of the generator used to transform data from the source domain to the target domain based on the relevant task data in the plurality of source domains and the relevant task data in the target domain includes: calculating a first adversarial loss based on the discrimination results corresponding to the source domain data of the relevant tasks and the discrimination results of the relevant task data in the target domain; and updating the parameters of the generator based on the first adversarial loss; wherein, the discrimination results corresponding to each source domain data of the relevant tasks are obtained by sequentially processing the source domain data of the relevant tasks through the generator transformation process and the discriminator discrimination process; and the discrimination results of each relevant task data in the target domain are obtained by processing the relevant task data of the target domain through the discriminator discrimination process. The weighting of the first differences among the multiple source domains to obtain the feature difference loss includes: The weight of the first difference of each source domain is calculated using the first adversarial loss corresponding to each source domain, wherein the first adversarial loss corresponding to each source domain is negatively correlated with the similarity between each source domain and the target domain.

4. The data classification method according to claim 2, characterized in that, The weighting of the first differences among the multiple source domains to obtain the feature difference loss includes: The feature difference loss is calculated based on the weighted result of the first difference of the multiple source domains and the difference between the data features of different source domains in each category of the target task.

5. The data classification method according to claim 1, characterized in that, The step of updating the parameters of the generator used to transform data from the source domain to the target domain based on relevant task data in the multiple source domains and relevant task data in the target domain includes: A first adversarial loss is calculated based on the discrimination results corresponding to the source domain data of the relevant task and the discrimination results of the relevant task data of the target domain; a second adversarial loss is calculated based on the discrimination results corresponding to the source domain data of the target task and the discrimination results of the relevant task data of the target domain. The generator parameters are updated based on the first adversarial loss and the second adversarial loss; Specifically, the discrimination results corresponding to each source domain data of the target task are obtained by sequentially processing each source domain data of the target task through the generator transformation process and the discriminator discrimination process; the discrimination results corresponding to each source domain data of the related tasks are obtained by sequentially processing each source domain data of the related tasks through the generator transformation process and the discriminator discrimination process; and the discrimination results of each related task data of the target domain are obtained by processing each related task data of the target domain through the discriminator discrimination process.

6. The data classification method according to claim 5, characterized in that, The step of updating the parameters of the generator used to transform data from the source domain to the target domain based on relevant task data in the multiple source domains and relevant task data in the target domain includes: Based on the first adversarial loss and the second adversarial loss, update the parameters of the generator and the discriminator; Calculate a first retention loss, a second retention loss, and a third retention loss, wherein the first retention loss is the retention loss of the generator for the structural features of the source domain data of the relevant task, the second retention loss is the retention loss of the generator for the structural features of the target domain data of the relevant task, and the third retention loss is the retention loss of the generator for the structural features of the source domain data of the target task. The generator parameters are updated again based on the first retention loss, the second retention loss, and the third retention loss.

7. The data classification method according to claim 1, characterized in that, The data classification model includes a feature extraction unit and a classifier. The step of extracting features from each of the transformed data includes: extracting features from each of the transformed data using the feature extraction unit; Updating the parameters of the data classification model based on the target domain features of the target task includes: The classifier is used to classify the target domain features of the target task. Based on the classification results of the target domain features, calculate the classification loss; The parameters of the data classification model are updated based on the classification loss. The feature extraction unit and the feature extraction module in the generator have the same structure and parameters.

8. An electronic device, characterized in that, The electronic device includes a processor for executing instructions to implement the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instruction / program data for execution to implement the method of any one of claims 1-7.

Images