Data tag identification method and device, and storage medium

Abstract

The application provides a data label identification method and device and a storage medium, and the method comprises the following steps: obtaining to-be-identified data; processing the to-be-identified data by using a preset joint discriminative multiple order moment alignment model to determine a target label of the to-be-identified data; wherein the preset joint discriminative multiple order moment alignment model is obtained by jointly iteratively training based on the inter-domain error, the intra-class error between a source domain sample set and a target domain sample set, and the source domain error of the source domain sample set. Since the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set are fully considered, the source domain sample set and the target domain sample set are jointly trained to obtain the preset joint discriminative multiple order moment alignment model, so that the model can improve the identification accuracy of data in different fields.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of computer and internet, and particularly relates to a data label identification method and device and storage medium. BACKGROUND

[0002] Data label identification has a large number of applications in related fields, and a data label identification model fully utilizes the knowledge and labels of a source domain through transfer learning to improve the performance of data label identification as much as possible in the case of lacking label data. At present, two transfer learning algorithms are included:

[0003] The first method realizes domain adaptation under multiple source domains based on partial feature alignment, includes a feature extraction module, a feature selection module of partial feature extraction and two classifiers for adversarial training, so as to obtain features highly related to the target domain in the source domain. However, this method does not consider that due to the difference in data distribution under different domains, the reliability of different source domains to the target domain is different, but directly sums up the difference between the class center point of each source domain and the class center point of the target domain, which will bring errors to the recognition accuracy of the model to the target domain.

[0004] The second method uses subspace / mapping to simultaneously map the data of the source domain and the target domain to a new subspace, and uses the source domain features in the new subspace to build a classifier, which is used for prediction of the target domain. However, this method ignores the problem that edge distribution adaptation and conditional distribution adaptation are not equally important, so it will inevitably bring errors to the data recognition accuracy under different domains. Therefore, the problem in related technologies is that the accuracy is poor when facing data recognition of different domains. SUMMARY

[0005] The data label identification method, device and storage medium provided by the embodiment of the present application can improve the recognition accuracy of different domain data.

[0006] The technical solution of the present application is implemented as follows:

[0007] The embodiment of the present application provides a data label identification method, comprising:

[0008] Obtaining the data to be identified;

[0009] Processing the data to be identified by using a preset joint discriminative multi-order moment alignment model to determine the target label of the data to be identified; wherein

[0010] The preset joint discriminative multi-order moment alignment model is obtained by joint iterative training based on the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label.

[0011] In the above scheme, before the preset joint discriminative multi-order moment alignment model is used to process the to-be-identified data to determine the target label of the to-be-identified data, the method further comprises:

[0012] Obtaining the source domain sample set and the target domain sample set; the source domain sample set comprises a plurality of source domain sample matrices corresponding to the plurality of source domain samples, and the source domain sample label corresponding to each source domain sample; the target domain sample set comprises a plurality of target domain sample matrices corresponding to the plurality of target domain samples;

[0013] Calculating the inter-domain error parameter, the intra-class error parameter, and the source domain error parameter of the source domain sample set between the source domain sample set and the target domain sample set by using an initial joint discriminative multi-order moment alignment model;

[0014] Determining a joint error parameter by using the inter-domain error parameter, the intra-class error parameter, and the source domain error parameter;

[0015] Iteratively adjusting the model parameters of the initial joint discriminative multi-order moment alignment model based on the joint error parameter until a training condition is met to stop, thereby obtaining the preset joint discriminative multi-order moment alignment model.

[0016] In the above scheme, the initial joint discriminative multi-order moment alignment model is used to calculate the inter-domain error parameter, the intra-class error parameter, and the source domain error parameter of the source domain sample set between the source domain sample set and the target domain sample set, comprising:

[0017] Inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model to obtain a plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices, and a plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices;

[0018] Based on the plurality of source domain sample vectors and the plurality of target domain sample vectors, the inter-domain error parameter is calculated;

[0019] Based on the plurality of source domain sample vectors and a plurality of preset source domain deep feature center vectors, a first intra-class error parameter is calculated;

[0020] calculate a second intra-class error parameter based on the plurality of target domain sample vectors and a plurality of preset target domain deep feature center vectors;

[0021] calculate the intra-class error parameter by the first intra-class error and the second intra-class error;

[0022] calculate the source domain error parameter by the plurality of source domain sample vectors and the plurality of source domain sample labels.

[0023] In the above scheme, the determination of the joint error parameter based on the domain error parameter, the intra-class error parameter and the source domain error parameter comprises:

[0024] multiply the domain error parameter by a first preset balance coefficient to obtain a first parameter;

[0025] multiply the intra-class error parameter by a second preset balance coefficient to obtain a second parameter;

[0026] add the first parameter, the second parameter and the source domain error parameter to obtain the joint error parameter.

[0027] In the above scheme, the input of the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model to obtain the corresponding plurality of source domain sample vectors of the plurality of source domain sample matrices and the corresponding plurality of target domain sample vectors of the plurality of target domain sample matrices comprises:

[0028] input the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model;

[0029] perform feature extraction on the plurality of source domain sample matrices and the plurality of target domain sample matrices by an initial feature extractor to obtain a plurality of source domain intermediate matrices corresponding to the plurality of source domain sample matrices and a plurality of target domain intermediate matrices corresponding to the plurality of target domain sample matrices;

[0030] perform deep feature extraction on the plurality of source domain intermediate matrices and the plurality of target domain intermediate matrices by an initial field adapter to obtain the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0031] In the above scheme, the calculation of the domain error parameter based on the plurality of source domain sample vectors and the plurality of target domain sample vectors comprises:

[0032] add the plurality of source domain sample vectors, and divide the sum by the length of the plurality of source domain sample vectors corresponding to the first matrix to obtain a source domain empirical expectation vector, add the plurality of target domain sample vectors, and divide the sum by the length of the plurality of target domain sample vectors corresponding to the second matrix to obtain a target domain empirical expectation vector;

[0033] calculate a K-order source domain central moment vector by using the source domain empirical expectation vector and the first matrix, and calculate a K-order target domain central moment vector by using the target domain empirical expectation vector and the second matrix; K is an integer greater than 1;

[0034] calculate the inter-domain error parameter based on the source domain empirical expectation vector, the target domain empirical expectation vector, the K-order source domain central moment vector, and the K-order target domain central moment vector.

[0035] In the above scheme, the K-order source domain central moment vector is calculated by using the source domain empirical expectation vector and the first matrix, and the K-order target domain central moment vector is calculated by using the target domain empirical expectation vector and the second matrix, including:

[0036] subtract each row in the first matrix from the source domain empirical expectation vector to obtain a source domain difference value matrix, and raise the source domain difference value matrix to the power of i to obtain an i-order source domain difference value matrix; i is an integer greater than or equal to 1 and less than K;

[0037] sum each row vector in the i-order source domain difference value matrix, and divide the sum by the length of the i-order source domain difference value matrix to obtain an i-order source domain central moment vector, until i is equal to k, and then the K-order source domain central moment vector is obtained;

[0038] subtract each row in the second matrix from the target domain empirical expectation vector to obtain a target domain difference value matrix, and raise the target domain difference value matrix to the power of i to obtain an i-order target domain difference value matrix;

[0039] sum each row vector in the i-order target domain difference value matrix, and divide the sum by the length of the i-order target domain difference value matrix to obtain an i-order target domain central moment vector, until i is equal to k, and then the K-order target domain central moment vector is obtained.

[0040] In the above scheme, the inter-domain error parameter is calculated based on the source domain empirical expectation vector, the target domain empirical expectation vector, the K-order source domain central moment vector, and the K-order target domain central moment vector, including:

[0041] calculate an expected two-norm of a difference value between the source domain empirical expectation vector and the target domain empirical expectation vector, and divide the expected two-norm by a preset threshold to obtain a first intermediate error parameter;

[0042] obtaining a central moment domain two-norm of a difference between the i-th order source domain central moment vector and the i-th order target domain central moment vector, dividing the central moment domain two-norm by the preset threshold raised to the i-th power to obtain an i-th order error parameter, until i equals K, to obtain a K-th order error parameter;

[0043] obtaining a sum of the first intermediate error parameter and the i-th order error parameter to the K-th order error parameter to obtain the inter-domain error parameter.

[0044] In the above scheme, after the plurality of source domain sample matrices and the plurality of target domain sample matrices are input into the initial joint discriminative multi-order moment alignment model to obtain the corresponding plurality of source domain sample vectors of the plurality of source domain sample matrices and the corresponding plurality of target domain sample vectors of the plurality of target domain sample matrices, and before the second intra-class error parameter is calculated based on the plurality of target domain sample vectors and the plurality of preset target domain deep feature center vectors, the method further comprises:

[0045] obtaining, by the label classifier in the initial joint discriminative multi-order moment alignment model, a plurality of node output values of each target domain sample vector through processing of the plurality of target domain sample vectors;

[0046] determining, in the plurality of node output values, a maximum node output value as an output value of the corresponding target domain sample vector, and further obtaining a plurality of output values of the plurality of target domain sample vectors;

[0047] determining, in the plurality of output values, at least one output value greater than a threshold value, and finding, by using the at least one output value, at least one target domain sample label corresponding to at least one target domain sample vector corresponding to the at least one output value.

[0048] In the above scheme, after the plurality of source domain sample matrices and the plurality of target domain sample matrices are input into the initial joint discriminative multi-order moment alignment model to obtain the corresponding plurality of source domain sample vectors of the plurality of source domain sample matrices and the corresponding plurality of target domain sample vectors of the plurality of target domain sample matrices, and before the first intra-class error parameter is calculated based on the plurality of source domain sample vectors and the plurality of preset source domain deep feature center vectors, the method further comprises:

[0049] initializing a plurality of source domain sample labels to obtain a plurality of initial deep feature center vectors, calculating a deep feature center difference vector of each source domain sample label by using the plurality of initial deep feature center vectors, and calculating a next deep feature center vector of the each source domain sample label by using the deep feature center difference vector;

[0050] The next time deep feature center vector is taken as the initial deep feature center vector of each source domain sample label for iterative calculation until the deep feature center difference vector reaches the second predetermined condition to stop, and a source domain deep feature center vector of each source domain sample label is obtained.

[0051] In the above scheme, after the plurality of source domain sample matrices and the plurality of target domain sample matrices are input into the initial joint discriminative multi-order moment alignment model to obtain a plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and a plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices, and before the second intra-class error parameter is calculated based on the plurality of target domain sample vectors and a plurality of preset target domain deep feature center vectors, the method further comprises:

[0052] A plurality of target initial deep feature center vectors are obtained by initializing a plurality of target domain sample labels, a target deep feature center difference vector of each target domain sample label is calculated through the plurality of target initial deep feature center vectors, and a next target deep feature center vector of each target domain sample label is calculated through the target deep feature center difference vector.

[0053] The next target deep feature center vector is taken as the target initial deep feature center vector of each target domain sample label for iterative calculation until the target deep feature center difference vector reaches the second predetermined condition to stop, and a target domain deep feature center vector of each target domain sample label is obtained.

[0054] In the above scheme, the plurality of target initial deep feature center vectors are obtained by initializing a plurality of target domain sample labels, a target deep feature center difference vector of each target domain sample label is calculated through the plurality of target initial deep feature center vectors, and a next target deep feature center vector of each target domain sample label is calculated through the target deep feature center difference vector, which comprises:

[0055] The sum of the difference between the plurality of target domain sample vectors and the corresponding target initial deep feature center vectors is calculated.

[0056] The sum of the difference is divided by the sum of the number of the plurality of target domain sample vectors and a preset value to obtain the target deep feature center difference vector.

[0057] Each target initial deep feature center vector is subtracted by the product of the corresponding target deep feature center difference vector and a second preset value to obtain the next target deep feature center vector.

[0058] In the scheme, the first intra-class error parameter is calculated based on the plurality of source domain sample vectors and a plurality of preset source domain deep feature center vectors, comprising:

[0059] The square of the two-norm of each source domain sample vector and the corresponding source domain deep feature center vector is calculated to obtain a source domain square value,

[0060] The source domain square values corresponding to the plurality of source domain sample vectors are added, and the first intra-class error parameter is obtained by comparing the third preset value.

[0061] In the scheme, the second intra-class error parameter is calculated based on the plurality of target domain sample vectors and a plurality of preset target domain deep feature center vectors, comprising:

[0062] The square of the two-norm of each target domain sample vector and the corresponding target domain deep feature center vector is calculated to obtain a target domain square value,

[0063] The target domain square values corresponding to the plurality of target domain sample vectors are added, and the second intra-class error parameter is obtained by comparing the third preset value.

[0064] In the scheme, the intra-class error parameter is calculated by the first intra-class error and the second intra-class error, comprising:

[0065] The first product of the first intra-class error and the fourth preset value is calculated;

[0066] The second product of the second intra-class error and the second constant minus the fourth preset threshold difference is calculated;

[0067] The first product and the second product are added to obtain the intra-class error parameter.

[0068] In the scheme, the source domain error parameter is calculated based on the plurality of source domain sample vectors and the plurality of source domain sample labels, comprising:

[0069] The first natural logarithm of the length of each source domain sample vector and the second natural logarithm of the difference between the each source domain sample vector and the second constant are calculated;

[0070] The third product is obtained by multiplying the first natural logarithm and the preset expected output vector length corresponding to the each source domain sample vector;

[0071] The fourth product is obtained by multiplying the second natural logarithm and the difference between the preset expected output vector length and the second constant;

[0072] The third product and the fourth product are added to obtain the intermediate source domain error parameter corresponding to the each source domain sample vector, and a plurality of intermediate source domain error parameters are obtained.

[0073] adding the plurality of intermediate source domain error parameters to obtain an error sum, obtaining a negative number of a ratio of the error sum to a number of the plurality of source domain sample vectors, and obtaining the source domain error parameter.

[0074] In the scheme, the processing of the to-be-identified data by using the preset joint discriminative multi-order moment alignment model to determine the target label of the to-be-identified data comprises:

[0075] processing the to-be-identified data by using the preset joint discriminative multi-order moment alignment model to obtain a predicted output value;

[0076] determining the target label corresponding to the to-be-identified data from a plurality of labels by using the predicted output value.

[0077] The embodiment of the present application further provides a data label identification device, comprising:

[0078] a data acquisition unit configured to acquire to-be-identified data;

[0079] a processing unit configured to process the to-be-identified data by using a preset joint discriminative multi-order moment alignment model to determine a target label of the to-be-identified data; wherein

[0080] the preset joint discriminative multi-order moment alignment model is obtained by joint iterative training based on an inter-domain error, an intra-class error between a source domain sample set and a target domain sample set, and a source domain error of the source domain sample set; a plurality of source domain samples in the source domain sample set and a plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label.

[0081] The embodiment of the present application further provides a data label identification device comprising a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the steps in the above method when executing the program.

[0082] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method.

[0083] In the embodiment of the present application, the data to be identified is acquired; the data to be identified is processed by using a preset joint discriminative multiple order moment alignment model to determine the target label of the data to be identified; wherein the preset joint discriminative multiple order moment alignment model is obtained by joint iterative training based on the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label. Since the present scheme fully considers the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set, the preset joint discriminative multiple order moment alignment model is obtained by joint training of the source domain sample set and the target domain sample set, so that the model can improve the recognition accuracy of data in different fields. BRIEF DESCRIPTION OF DRAWINGS

[0084] Figure 1 An optional flowchart of a data label recognition method provided by the embodiment of the present application is shown in the figure.

[0085] Figure 2 An optional effect diagram of a data label recognition method provided by the embodiment of the present application is shown in the figure.

[0086] Figure 3 An optional flowchart of a data label recognition method provided by the embodiment of the present application is shown in the figure.

[0087] Figure 4 An optional effect diagram of a data label recognition method provided by the embodiment of the present application is shown in the figure.

[0088] Figure 5 An optional effect diagram of a data label recognition method provided by the embodiment of the present application is shown in the figure.

[0089] Figure 6 A structure diagram of a data label recognition device provided by the embodiment of the present application is shown in the figure.

[0090] Figure 7 An optional flowchart of a data label recognition device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application is further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of the present application.

[0092] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets as each other and as all possible embodiments, and can be combined with each other, without conflicts.

[0093] If the similar description of "first / second" appears in the invention document, the following description is added, in the following description, the terms "first\second\third" referred to only distinguish similar objects, not represent the specific order of the object, it can be understood that "first\second\third" can be exchanged in the specific order or sequence, if allowed, so that the embodiments of the invention described here can be implemented in the order other than illustrated or described here.

[0094] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. The terminology used in the description herein is for describing the embodiments of the invention only and is not intended to be limiting of the invention.

[0095] Please refer to Figure 1 , an optional flow diagram of the data tag identification method provided by the embodiments of the invention will be described with reference to the steps shown in the figure. Figure 1

[0096] S101, acquire the data to be identified.

[0097] In the embodiments of the invention, the data tag identification device acquires the data to be identified.

[0098] In the embodiments of the invention, the data tag identification device acquires the data to be identified sent by other servers or terminals or clients.

[0099] The data to be identified can include image data, video data, audio data, text data, etc.

[0100] In the embodiments of the invention, the data tag identification device can include a server or terminal with data processing function.

[0101] S102, process the data to be identified by using a preset joint discriminative multi-order moment alignment model to determine the target label of the data to be identified; wherein the preset joint discriminative multi-order moment alignment model is obtained by joint iterative training based on the domain error between the source domain sample set and the target domain sample set, the intra-class error, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label.

[0102] ​In the embodiment of the present application, the data label recognition device processes the to-be-recognized data by using the preset joint discriminative multi-order moment alignment model to determine the target label of the to-be-recognized data.

[0103] The preset joint discriminative multi-order moment alignment model is obtained by joint iterative training based on the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label.

[0104] In the embodiment of the present application, the data label recognition device can obtain the source domain sample set and the target domain sample set; the source domain sample set includes a plurality of source domain sample matrices corresponding to the plurality of source domain samples, and the source domain sample label corresponding to each source domain sample; the target domain sample set includes a plurality of target domain sample matrices corresponding to the plurality of target domain samples; the inter-domain error parameter, the intra-class error parameter between the source domain sample set and the target domain sample set, and the source domain error parameter of the source domain sample set are calculated by using an initial joint discriminative multi-order moment alignment model; the joint error parameter is determined by using the inter-domain error parameter, the intra-class error parameter and the source domain error parameter; the model parameters of the initial joint discriminative multi-order moment alignment model are iteratively adjusted based on the joint error parameter until the training condition is met to stop, and the preset joint discriminative multi-order moment alignment model is obtained.

[0105] In the embodiment of the present application, the data in the source domain sample set, the data in the target domain sample set and the to-be-recognized data can be different data fields, but the three should belong to the same data category. For example, when the data in the target domain sample set and the data in the source domain sample set are image data, the to-be-recognized data should also be image data. The data (with label) in the source domain sample set can be an image set (such as a handwritten data set), which is composed of several single-channel original black and white images, each image corresponds to a string of handwritten numbers, so each image corresponds to a label (0-9).

[0106] In the embodiment of the present application, the plurality of source domain samples and the plurality of target domain samples can include image data, video data, audio data, text data, etc.

[0107] Exemplarily, in combination with Figure 2 The field self-adaptive algorithm based on joint discriminative multi-order moment alignment can be applied to the image recognition field, and for the case of cross-image set, the algorithm can accurately extract deep information of different domain images, thereby effectively improving the recognition accuracy of target domain images.

[0108] The application provides a domain adaptation algorithm based on joint discriminative multi-order moment alignment.

[0109] The application provides a domain adaptation algorithm based on joint discriminative multi-order moment alignment.

[0110] The application provides a domain adaptation algorithm based on joint discriminative multi-order moment alignment. Figure 3As shown in the figure. The feature extractor extracts the deep features of the source domain samples and the target domain samples through a deep convolutional network. The domain adapter aligns and narrows the deep features of the source domain samples and the target domain samples extracted by the feature extractor through the calculation of the center loss and the center moment difference. The label classifier is trained through the calculation of the cross-entropy loss to output the prediction result (target domain category) of the model for the target domain. This method learns the discriminative features of different domains by jointly training the labeled samples of the source domain and the unlabeled samples of the target domain, and aligns the features learned by the deep convolutional network of the two domains through multiple order moments, thereby realizing the domain adaptation of the model. By considering the source domain loss, the center loss and the domain difference loss at the same time, the training of the model can improve the prediction accuracy of the model for the target domain as much as possible without reducing the convergence rate.

[0111] As shown in the figure. The feature extractor extracts the deep features of the source domain samples and the target domain samples through a deep convolutional network. The domain adapter aligns and narrows the deep features of the source domain samples and the target domain samples extracted by the feature extractor through the calculation of the center loss and the center moment difference. The label classifier is trained through the calculation of the cross-entropy loss to output the prediction result (target domain category) of the model for the target domain. This method learns the discriminative features of different domains by jointly training the labeled samples of the source domain and the unlabeled samples of the target domain, and aligns the features learned by the deep convolutional network of the two domains through multiple order moments, thereby realizing the domain adaptation of the model. By considering the source domain loss, the center loss and the domain difference loss at the same time, the training of the model can improve the prediction accuracy of the model for the target domain as much as possible without reducing the convergence rate. Figure 4 As shown in the figure. The feature extractor extracts the deep features of the source domain samples and the target domain samples through a deep convolutional network. The domain adapter aligns and narrows the deep features of the source domain samples and the target domain samples extracted by the feature extractor through the calculation of the center loss and the center moment difference. The label classifier is trained through the calculation of the cross-entropy loss to output the prediction result (target domain category) of the model for the target domain. This method learns the discriminative features of different domains by jointly training the labeled samples of the source domain and the unlabeled samples of the target domain, and aligns the features learned by the deep convolutional network of the two domains through multiple order moments, thereby realizing the domain adaptation of the model. By considering the source domain loss, the center loss and the domain difference loss at the same time, the training of the model can improve the prediction accuracy of the model for the target domain as much as possible without reducing the convergence rate.

[0112] In the embodiment of the application, the to-be-identified data is obtained; the to-be-identified data is processed by using a preset joint discriminative multi-order moment alignment model to determine the target label of the to-be-identified data; wherein the preset joint discriminative multi-order moment alignment model is obtained by jointly iterative training based on the domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data domains, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label. Since the present scheme fully considers the domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set, the preset joint discriminative multi-order moment alignment model is obtained by jointly training the source domain sample set and the target domain sample set, so the model can improve the recognition accuracy of data in different domains.

[0113] In some embodiments, Figure 1 As shown in the figure. The feature extractor extracts the deep features of the source domain samples and the target domain samples through a deep convolutional network. The domain adapter aligns and narrows the deep features of the source domain samples and the target domain samples extracted by the feature extractor through the calculation of the center loss and the center moment difference. The label classifier is trained through the calculation of the cross-entropy loss to output the prediction result (target domain category) of the model for the target domain. This method learns the discriminative features of different domains by jointly training the labeled samples of the source domain and the unlabeled samples of the target domain, and aligns the features learned by the deep convolutional network of the two domains through multiple order moments, thereby realizing the domain adaptation of the model. By considering the source domain loss, the center loss and the domain difference loss at the same time, the training of the model can improve the prediction accuracy of the model for the target domain as much as possible without reducing the convergence rate.

[0114] S103, obtain a source domain sample set and a target domain sample set.

[0115] In the embodiment of the present application, the data label recognition device acquires the source domain sample set and the target domain sample set.

[0116] The source domain sample set comprises: a plurality of source domain sample matrices corresponding to a plurality of source domain samples, and a source domain sample label corresponding to each source domain sample.

[0117] In the embodiment of the present application, the data label recognition device can acquire the source domain sample set and the target domain sample set locally. The data label recognition device can also acquire the source domain sample set and the target domain sample set from other servers or terminals.

[0118] In the embodiment of the present application, the plurality of source domain samples have a plurality of corresponding source domain sample labels. The label can be a number or identification information representing the attribute of the corresponding sample. The plurality of source domain samples are processed to obtain a plurality of corresponding source domain sample matrices. The plurality of target domain samples can also be processed to obtain a plurality of target domain sample matrices. For example, when the source domain sample is a picture, the source domain sample can be processed to obtain a three-channel matrix.

[0119] The video data, audio data and text data can be calculated to obtain their respective matrices through corresponding matrix algorithms.

[0120] In the embodiment of the present application, the data label recognition device obtains a source domain sample set D S with labels n and a target domain sample set D T without labels m The data distributions P(X S ) and P(X T ) of the two fields are different, and the goal of the domain adaptation algorithm is to use the labeled data D S to learn a classifier f to predict the label Y T of the target domain D T . Wherein, xi represents a source domain sample, yi represents a source domain sample, and n is the number of source domain samples. Xj represents a target domain sample.

[0121] In the embodiment of the present application, the data label recognition device calculates the inter-domain error parameters, the intra-class error parameters between the source domain sample set and the target domain sample set, and the source domain error parameters of the source domain sample set by using the initial joint discriminative multi-order moment alignment model.

[0122] In the embodiment of the present application, the data label recognition device calculates the inter-domain error parameters, the intra-class error parameters between the source domain sample set and the target domain sample set, and the source domain error parameters of the source domain sample set by using the initial joint discriminative multi-order moment alignment model.

[0123] In the embodiment of the present application, the data label recognition device inputs a plurality of source domain samples and a plurality of target domain samples into an initial joint discriminative multi-order moment alignment model, and obtains a plurality of source domain sample vectors of the plurality of source domain samples and a plurality of target domain sample vectors corresponding to the plurality of target domain samples through processing of the initial joint discriminative multi-order moment alignment model. The data label recognition device calculates domain error parameters, intra-class error parameters and source domain error parameters of a source domain sample set in combination with the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0124] In the embodiment of the present application, the data label recognition device calculates the multi-order central moment difference between the source domain sample set and the target domain sample set, the center loss and the cross-entropy loss of the source domain sample set by using the initial joint discriminative multi-order moment alignment model.

[0125] S105, determining a joint error parameter by using the domain error parameter, the intra-class error parameter and the source domain error parameter.

[0126] In the embodiment of the present application, the data label recognition device determines the joint error parameter by using the domain error parameter, the intra-class error parameter and the source domain error parameter.

[0127] In the embodiment of the present application, the data label recognition device can multiply the domain error parameter, the intra-class error parameter and the source domain error parameter by respective corresponding coefficients, and then sum to obtain the joint error parameter.

[0128] S106, iteratively adjusting model parameters of the initial joint discriminative multi-order moment alignment model based on the joint error parameter until a training condition is met, and stopping to obtain a preset joint discriminative multi-order moment alignment model.

[0129] In the embodiment of the present application, the data label recognition device iteratively adjusts model parameters of the initial joint discriminative multi-order moment alignment model based on the joint error parameter until a training condition is met, and stops to obtain a preset joint discriminative multi-order moment alignment model.

[0130] In the embodiment of the present application, the data label recognition device adjusts weight parameters of each layer network of the initial joint discriminative multi-order moment alignment model in the back propagation training process based on the joint error parameter, and stops until a predetermined training number is reached or a loss function of the initial joint discriminative multi-order moment alignment model converges, to obtain a preset joint discriminative multi-order moment alignment model.

[0131] Since the present scheme fully considers the domain error between the source domain sample set and the target domain sample set, the intra-class error and the source domain error of the source domain sample set, the model can improve the recognition accuracy of different field data.

[0132] In some embodiments, S104 shown can also be implemented by S107 to S112, which will be described in combination with each step.

[0133] S107, inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-order moment alignment model to obtain a plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and a plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices.

[0134] In the embodiment of the application, the data label recognition device inputs the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model to obtain the plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and the plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices.

[0135] In the embodiment of the application, the data label recognition device inputs the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model, and obtains the plurality of source domain intermediate matrices corresponding to the plurality of source domain sample matrices and the plurality of target domain intermediate matrices corresponding to the plurality of target domain sample matrices through the feature extractor. Then, the plurality of source domain intermediate matrices and the plurality of target domain intermediate matrices are processed through the data adapter to obtain the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0136] S108, calculating an inter-domain error parameter based on the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0137] In the embodiment of the application, the data label recognition device calculates the inter-domain error parameter based on the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0138] S109, calculating a first intra-class error parameter based on the plurality of source domain sample vectors and a plurality of preset source domain deep feature center vectors.

[0139] In the embodiment of the application, the data label recognition device calculates the first intra-class error parameter based on the plurality of source domain sample vectors and the plurality of preset source domain deep feature center vectors.

[0140] S110, calculating a second intra-class error parameter based on the plurality of target domain sample vectors and a plurality of preset target domain deep feature center vectors.

[0141] In the embodiment of the application, the data label recognition device calculates the second intra-class error parameter based on the plurality of target domain sample vectors and the plurality of preset target domain deep feature center vectors.

[0142] S111, calculating an intra-class error parameter through the first intra-class error and the second intra-class error.

[0143] In the embodiment of the present application, the data label recognition device calculates the intra-class error parameter through the first intra-class error and the second intra-class error.

[0144] In the embodiment of the present application, the data label recognition device calculates the source domain error parameter through the plurality of source domain sample vectors and the plurality of source domain sample labels.

[0145] In the embodiment of the present application, the data label recognition device calculates the source domain error parameter through the plurality of source domain sample vectors and the plurality of source domain sample labels.

[0146] In some embodiments, S105 shown can also be implemented through S113 to S115, which will be described in combination with each step.

[0147] S113, multiplying the inter-domain error parameter by a first preset balance coefficient to obtain a first parameter.

[0148] In the embodiment of the present application, the data label recognition device multiplies the inter-domain error parameter by a first preset balance coefficient to obtain a first parameter.

[0149] The first preset balance coefficient is a coefficient obtained in the model training process. In the embodiment of the present application, the first preset balance coefficient is not limited.

[0150] S114, multiplying the intra-class error parameter by a second preset balance coefficient to obtain a second parameter.

[0151] In the embodiment of the present application, the data label recognition device multiplies the intra-class error parameter by a second preset balance coefficient to obtain a second parameter.

[0152] The second preset balance coefficient is a coefficient obtained in the model training process. In the embodiment of the present application, the second preset balance coefficient is not limited.

[0153] S115, adding the first parameter, the second parameter and the source domain error parameter to obtain a joint error parameter.

[0154] In the embodiment of the present application, the data label recognition device adds the first parameter, the second parameter and the source domain error parameter to obtain a joint error parameter.

[0155] For example, the data label recognition device can calculate the joint error parameter through formula (1) to obtain the loss function L.

[0156] L = L s + λ d L d + λ c L c (1)

[0157] The λ dis a first preset balancing coefficient, and is a second preset balancing coefficient. The and are trade-off coefficients for controlling the balance between different loss functions in the training process. d is an inter-domain error parameter, and L c is an intra-domain error parameter, and L s is a source domain error parameter. The data label recognition device multiplies the by the d multiplies the by the d obtains a first parameter, and multiplies the by the c obtains a second parameter, and adds the first parameter, the second parameter and the source domain error parameter to obtain the loss function L.

[0158] In the back propagation training process of the model, the loss function is minimized by adjusting the network weight parameters of each layer in the model, so that the model can learn the features of domain invariance and class discriminativeness. The final optimization objective of the model is calculated as formula (2).

[0159]

[0160] wherein x s is a source domain sample, y s is a label of the source domain sample, x t is a target domain sample. When the loss function L is less than a certain threshold or the number of iterations reaches a preset value, the model training is completed.

[0161] In the embodiment of the application, in the back propagation training process of the model, the loss function is minimized by adjusting the network weight parameters of each layer in the model, so that the model can learn the features of domain invariance and class discriminativeness, thereby improving the accuracy of data label recognition.

[0162] In some embodiments, S107 shown can also be implemented through S116 to S118, which will be described in combination with each step.

[0163] S116, input the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-order moment alignment model.

[0164] In the embodiment of the application, the data label recognition device inputs the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-order moment alignment model.

[0165] S117, perform feature extraction on the plurality of source domain sample matrices and the plurality of target domain sample matrices through an initial feature extractor to obtain a plurality of source domain intermediate matrices corresponding to the plurality of source domain sample matrices, and a plurality of target domain intermediate matrices corresponding to the plurality of target domain sample matrices.

[0166] In the embodiment of the present application, the data label recognition device extracts features of a plurality of source domain sample matrices and a plurality of target domain sample matrices through an initial feature extractor to obtain a plurality of source domain intermediate matrices corresponding to the plurality of source domain sample matrices and a plurality of target domain intermediate matrices corresponding to the plurality of target domain sample matrices.

[0167] In the embodiment of the present application, the initial feature extractor is constructed by using a deep convolutional network, the plurality of source domain sample matrices and the plurality of target domain sample matrices are input into the initial feature extractor, and deep features of the samples are obtained. The deep convolutional network is composed of a plurality of convolutional layers and a plurality of max-pooling layers, and the output units of the last max-pooling layer are tiled to convert them into one-dimensional vectors.

[0168] The expression formula (3) of the convolution operation is as follows:

[0169]

[0170] In the formula, f(·) represents an activation function, '*' represents a one-dimensional convolution operator, wj represents the jth convolution kernel, b is a bias, Cin is the number of input channels, and xi represents an input feature vector of the ith channel.

[0171] The expression formula of the max-pooling operation is as follows:

[0172]

[0173] In the formula, cj,k represents the kth element value of the jth output channel of the convolutional layer, and K is the size of the convolution kernel.

[0174] In the embodiment of the present application, the data label recognition device extracts deep features of the plurality of source domain intermediate matrices and the plurality of target domain intermediate matrices through the initial domain adapter to obtain a plurality of source domain sample vectors and a plurality of target domain sample vectors.

[0175] In the embodiment of the present application, the data label recognition device extracts deep features of the plurality of source domain intermediate matrices and the plurality of target domain intermediate matrices through the initial domain adapter to obtain a plurality of source domain sample vectors and a plurality of target domain sample vectors.

[0176] In the embodiment of the present application, the initial domain adapter is constructed by using a deep fully connected network, and deep features of the source domain and the target domain samples are further extracted through the domain adapter according to the deep features of the plurality of source domain sample matrices and the plurality of target domain sample matrices obtained by the feature extractor.

[0177] The output expression formula (4) of the hidden layer node in the fully connected network is as follows:

[0178]

[0179] In the formula, xi represents the value of the ith input node, represents the weight value from the i-th input node to the h-th hidden layer node.

[0180] In the embodiment of the present application, more accurate data information can be extracted by deep extraction of the plurality of source domain samples and the plurality of target domain samples, so that the recognition performance of the trained model is better.

[0181] In some embodiments, S108 shown can also be implemented by S119 to S121, which will be described in combination with each step.

[0182] S119, add the plurality of source domain sample vectors, and then divide the length of the plurality of source domain sample vectors corresponding to the first matrix to obtain a source domain empirical expectation vector; add the plurality of target domain sample vectors, and then divide the length of the plurality of target domain sample vectors corresponding to the second matrix to obtain a target domain empirical expectation vector.

[0183] In the embodiment of the present application, the data label recognition device adds the plurality of source domain sample vectors, and then divides the length of the plurality of source domain sample vectors corresponding to the first matrix to obtain a source domain empirical expectation vector; adds the plurality of target domain sample vectors, and then divides the length of the plurality of target domain sample vectors corresponding to the second matrix to obtain a target domain empirical expectation vector.

[0184] In the embodiment of the present application, the data label recognition device can calculate the source domain empirical expectation vector E(H) through formula (5).

[0185]

[0186] In the formula, H={h1,…,h n}, n is the total number of the plurality of source domain sample vectors. H is the first matrix. The data label can add h1 to h n , and then divide the length of H to obtain E(H).

[0187] In the embodiment of the present application, the data label recognition device can obtain the target domain empirical expectation vector through the same method.

[0188] S120, calculate a K-order source domain central moment vector through the source domain empirical expectation vector and the first matrix, and calculate a K-order target domain central moment vector through the target domain empirical expectation vector and the second matrix.

[0189] In the embodiment of the present application, the data label recognition device calculates a K-order source domain central moment vector through the source domain empirical expectation vector and the first matrix, and calculates a K-order target domain central moment vector through the target domain empirical expectation vector and the second matrix. K is an integer greater than 1.

[0190] In some embodiments, S120 shown can also be implemented by S1201 to S1204, which will be described in combination with each step.

[0191] S1201, subtract the source domain empirical expectation vector from each row in the first matrix to obtain a source domain difference matrix, and then raise the source domain difference matrix to the i-th power to obtain an i-th order source domain difference matrix.

[0192] In the embodiment of the application, the data label recognition device subtracts the source domain empirical expectation vector from each row in the first matrix to obtain a source domain difference matrix, and then raises the source domain difference matrix to the i-th power to obtain an i-th order source domain difference matrix. i is an integer greater than or equal to 1 and less than K.

[0193] S1202, sum each row vector in the i-th order source domain difference matrix, and then compare the length of the i-th order source domain difference matrix to obtain an i-th order source domain central moment vector, until i is equal to k, and then a K-th order source domain central moment vector is obtained.

[0194] In the embodiment of the application, the data label recognition device sums each row vector in the i-th order source domain difference matrix, and then compares the length of the i-th order source domain difference matrix to obtain an i-th order source domain central moment vector, until i is equal to k, and then a K-th order source domain central moment vector is obtained.

[0195] In the embodiment of the application, the data label recognition device can calculate the K-th order source domain central moment vector C k (H)

[0196] C k (H)=E[(H-E(H) k ] (6)

[0197] Wherein, E[ ] represents the empirical expectation vector of *, H is the first matrix, E(H) is the source domain empirical expectation vector, and (H-E(H)) is the k-th order source domain difference matrix.

[0198] S1203, subtract the target domain empirical expectation vector from each row in the second matrix to obtain a target domain difference matrix, and then raise the target domain difference matrix to the i-th power to obtain an i-th order target domain difference matrix.

[0199] In the embodiment of the application, the data label recognition device subtracts the target domain empirical expectation vector from each row in the second matrix to obtain a target domain difference matrix, and then raises the target domain difference matrix to the i-th power to obtain an i-th order target domain difference matrix.

[0200] S1204, sum each row vector in the i-th order target domain difference matrix, and then compare the length of the i-th order target domain difference matrix to obtain an i-th order target domain central moment vector, until i is equal to k, and then a K-th order target domain central moment vector is obtained.

[0201] In the embodiment of the present application, the data label recognition device obtains the sum of each row vector in the i-th order target domain difference value matrix, and then obtains the i-th order target domain central moment vector by dividing the module length of the i-th order target domain difference value matrix, until i equals k, and then obtains the K-th order target domain central moment vector.

[0202] In the embodiment of the present application, the data label recognition device obtains the sum of each row vector in the i-th order target domain difference value matrix, and then obtains the i-th order target domain central moment vector by dividing the module length of the i-th order target domain difference value matrix, until i equals k, and then obtains the K-th order target domain central moment vector.

[0203] In the embodiment of the present application, the data label recognition device obtains the sum of each row vector in the i-th order target domain difference value matrix, and then obtains the i-th order target domain central moment vector by dividing the module length of the i-th order target domain difference value matrix, until i equals k, and then obtains the K-th order target domain central moment vector.

[0204] In some embodiments, S121 shown can also be implemented by S1211 to S1213, which will be described in combination with each step.

[0205] S1211, obtains the expected two-norm of the difference between the source domain expected experience vector and the target domain expected experience vector, divides the expected two-norm by the preset threshold value to obtain a first intermediate error parameter.

[0206] In the embodiment of the present application, the data label recognition device obtains the expected two-norm of the difference between the source domain expected experience vector and the target domain expected experience vector, and divides the expected two-norm by the preset threshold value to obtain a first intermediate error parameter.

[0207] S1212, obtains the central moment domain two-norm of the difference between the i-th order source domain central moment vector and the i-th order target domain central moment vector, divides the central moment domain two-norm by the i-th power of the preset threshold value to obtain the i-th order error parameter, until i equals K, and obtains the K-th order error parameter.

[0208] In the embodiment of the present application, the data label recognition device obtains the central moment domain two-norm of the difference between the i-th order source domain central moment vector and the i-th order target domain central moment vector, and divides the central moment domain two-norm by the i-th power of the preset threshold value to obtain the i-th order error parameter, until i equals K, and obtains the K-th order error parameter.

[0209] S1213, obtains the sum of the first intermediate error parameter and the i-th order error parameter to the K-th order error parameter to obtain the inter-domain error parameter.

[0210] In the embodiment of the present application, the data label recognition device obtains the sum of the first intermediate error parameter and the i-th order error parameter to the K-th order error parameter to obtain the inter-domain error parameter.

[0211] In this embodiment of the invention, the data label recognition device calculates the inter-domain error parameters of the source and target domains based on the output features of the source and target domain samples obtained by the domain adapter in the last fully connected layer, using the difference in multi-order central moments, to measure the differences in the distribution of data features in different domains.

[0212] The inter-domain error parameter is defined as shown in formula (7):

[0213]

[0214] In the formula, the first part represents the depth features H of the source and target domains. s and H t The average difference between the probability distributions, the second part represents H s and H t The higher-order differences in probability distributions between them. K is the maximum order of the difference in central moments, and a and b are the lower and upper bounds of Hs and Ht, respectively. E(H s Let E(H) be the empirical expectation vector of the source domain. t Let C be the empirical expectation vector of the target domain and the source domain. k (H s Let C be the source domain central moment vector. k (H t ) is the central moment vector of the target domain.

[0215] In this embodiment of the invention, by calculating the inter-domain error parameters between the source domain sample set and the target domain sample set, and updating the parameters in the backpropagation of the model using the inter-domain error parameters, a preset model with better recognition performance is obtained.

[0216] In some embodiments, S107 to S110 may also include S122 to S124, which will be described in conjunction with each step.

[0217] S122. The output values ​​of multiple nodes of each target domain sample vector are obtained by processing multiple target domain sample vectors through the label classifier in the initial joint discriminative multi-moment alignment model.

[0218] In this embodiment of the invention, the data label recognition device obtains multiple target domain sample vectors by processing the label classifier in the initial joint discriminative multi-moment alignment model, and the output values ​​of multiple nodes of each target domain sample vector.

[0219] S123. Determine the output value of the node with the largest output value among multiple node output values ​​as the output value of the corresponding target domain sample vector, and thus obtain multiple output values ​​of multiple target domain sample vectors.

[0220] In the embodiment of the present application, the data label recognition device determines the maximum node output value as the output value of the corresponding target domain sample vector among the multiple node output values, and further obtains multiple output values of multiple target domain sample vectors.

[0221] In the embodiment of the present application, at least one output value greater than the threshold value is determined among the multiple output values, and at least one target domain sample label corresponding to at least one target domain sample vector corresponding to the at least one output value is found.

[0222] In the embodiment of the present application, at least one output value greater than the threshold value is determined among the multiple output values, and at least one target domain sample label corresponding to at least one target domain sample vector corresponding to the at least one output value is found.

[0223] In the embodiment of the present application, since the target domain sample is not labeled, a pseudo label is assigned to the target domain sample according to the output encoding of the classifier layer obtained after the initial model is input into the target domain sample Then, high-confidence target domain samples with a prediction probability greater than a given threshold μ are selected, and the pseudo label is assigned to these reliable target domain samples. The pseudo label is calculated by formula (8) to obtain the target domain sample label

[0224]

[0225] In the formula, The actual output value of the target domain sample at the kth node of the label classifier.

[0226] In some embodiments, S125 to S126 can also be implemented between S107 to S109 as shown, which will be described in combination with each step.

[0227] S125, initializing multiple source domain sample labels to obtain multiple initial deep feature center vectors, calculating a deep feature center difference vector of each source domain sample label through the multiple initial deep feature center vectors, and calculating a next deep feature center vector of each source domain sample label through the deep feature center difference vector.

[0228] In the embodiment of the present application, the data label recognition device initializes multiple source domain sample labels to obtain multiple initial deep feature center vectors, calculates a deep feature center difference vector of each source domain sample label through the multiple initial deep feature center vectors, and calculates a next deep feature center vector of each source domain sample label through the deep feature center difference vector.

[0229] In the embodiment of the present application, the data label recognition device calculates the sum of differences between the plurality of source domain sample vectors and the corresponding initial deep feature center vectors. The sum of the differences is divided by the sum of the number of the plurality of source domain sample vectors and a preset value to obtain a deep feature center difference vector. Each initial deep feature center vector is subtracted by the product of the corresponding deep feature center difference vector and a second preset value to obtain a next deep feature center vector.

[0230] In the embodiment of the present application, the data label recognition device iteratively calculates the next deep feature center vector as the initial deep feature center vector of each source domain sample label until the deep feature center difference vector reaches a second predetermined condition, and stops to obtain the source domain deep feature center vector of each source domain sample label.

[0231] In the embodiment of the present application, the data label recognition device iteratively calculates the next deep feature center vector as the initial deep feature center vector of each source domain sample label until the deep feature center difference vector reaches a second predetermined condition, and stops to obtain the source domain deep feature center vector of each source domain sample label.

[0232] In the embodiment of the present application, the second predetermined condition can be that the deep feature center difference vector reaches a certain interval.

[0233] In some embodiments, S127 to S128 can also be implemented between S107 to S110 as shown, which will be described in combination with each step.

[0234] S127, initializing a plurality of target domain sample labels to obtain a plurality of target initial deep feature center vectors, calculating a target deep feature center difference vector of each target domain sample label through the plurality of target initial deep feature center vectors, and calculating a next target deep feature center vector of each target domain sample label through the target deep feature center difference vector.

[0235] In the embodiment of the present application, the data label recognition device initializes a plurality of target domain sample labels to obtain a plurality of target initial deep feature center vectors, calculates a target deep feature center difference vector of each target domain sample label through the plurality of target initial deep feature center vectors, and calculates a next target deep feature center vector of each target domain sample label through the target deep feature center difference vector.

[0236] In the embodiment of the present application, the data label recognition device calculates the sum of differences between the plurality of target domain sample vectors and the corresponding target initial deep feature center vectors. The sum of the differences is divided by the sum of the number of the plurality of target domain sample vectors and a preset value to obtain a target deep feature center difference vector. Each target initial deep feature center vector is subtracted by the product of the corresponding target deep feature center difference vector and a second preset value to obtain a next target deep feature center vector.

[0237] In this embodiment of the invention, the data label recognition device learns the discriminative features between different categories in the source and target domains simultaneously using joint center loss based on the output features of the source and target domain samples obtained by the domain adapter in the last fully connected layer, thereby expanding the discriminability between different categories in different domains.

[0238] The definition of center loss is as shown in formula (9):

[0239]

[0240] In the formula, xi represents the feature vector of the i-th target domain sample. Let represent the deep feature center vector of the class to which the i-th sample belongs, and m be the total number of samples in the target domain.

[0241] In each iteration, the center is calculated by averaging the features of different classes. The gradient of Lc with respect to xi is... The update equations are calculated as shown in formulas (10)-(12):

[0242]

[0243]

[0244]

[0245] In the above formula, α is the second preset value used for the learning rate of the control center, and α∈[0,1]. δ represents y i =j means δ is 1, and 0 when they are not equal. The initial depth feature center vector of the target. Characteristic y i When =j, calculate the sum of the differences between the sample vectors of multiple target domains and the corresponding initial depth feature center vectors of the target. Characteristic y i When =j, the number of sample vectors in multiple target domains is incremented by 1.

[0246] S128. The next target depth feature center vector is used as the target initial depth feature center vector of each target domain sample label for iterative calculation until the target depth feature center difference vector reaches the second predetermined condition, and the target domain depth feature center vector of each target domain sample label is obtained.

[0247] In the embodiment of the present application, the data label recognition device takes the next target depth feature center vector as the target initial depth feature center vector of each target domain sample label for iterative calculation until the target depth feature center difference vector reaches the second predetermined condition to stop, and obtains the target domain depth feature center vector of each target domain sample label.

[0248] In the embodiment of the present application, the second predetermined condition can be that the target depth feature center difference vector reaches a certain interval.

[0249] In some embodiments, S109 to S111 shown can also be implemented through S129 to S135, which will be described in combination with each step.

[0250] S129, square the two-norm of each source domain sample vector and the corresponding source domain depth feature center vector to obtain a source domain square value.

[0251] In the embodiment of the present application, the data label recognition device squares the two-norm of each source domain sample vector and the corresponding source domain depth feature center vector to obtain a source domain square value.

[0252] S130, add the source domain square values corresponding to the plurality of source domain sample vectors, and compare the sum with a third preset value to obtain a first intra-class error parameter.

[0253] In the embodiment of the present application, the data label recognition device adds the source domain square values corresponding to the plurality of source domain sample vectors, and compares the sum with a third preset value to obtain a first intra-class error parameter.

[0254] The third preset value can be 2, and the third preset value is not limited in the embodiment of the present application.

[0255] In the embodiment of the present application, the data label recognition device can calculate the first intra-class error parameter through formula (13)

[0256] Wherein, The i-th source domain sample vector is represented by xi. The i-th source domain depth feature center vector is represented by μi. The corresponding source domain depth feature center vector is represented by μi.

[0257] S131, square the two-norm of each target domain sample vector and the corresponding target domain depth feature center vector to obtain a target domain square value.

[0258] In the embodiment of the present application, the data label recognition device squares the two-norm of each target domain sample vector and the corresponding target domain depth feature center vector to obtain a target domain square value.

[0259] S132, add the target domain square values corresponding to the plurality of target domain sample vectors, and obtain a second intra-class error parameter by comparing the result with a third preset value.

[0260] In the embodiment of the application, the data label recognition device adds the target domain square values corresponding to the plurality of target domain sample vectors, and obtains a second intra-class error parameter by comparing the result with a third preset value.

[0261] In the embodiment of the application, the data label recognition device can calculate the second intra-class error parameter L t c .

[0262]

[0263] wherein, represents the i-th target domain sample vector, represents the corresponding source domain deep feature center vector.

[0264] S133, obtain a first product of the first intra-class error and a fourth preset value.

[0265] In the embodiment of the application, the data label recognition device obtains a first product of the first intra-class error and a fourth preset value.

[0266] S134, obtain a second product of the second intra-class error and a second difference value obtained by subtracting a fourth preset threshold value from a constant.

[0267] In the embodiment of the application, the data label recognition device obtains a second product of the second intra-class error and a second difference value obtained by subtracting a fourth preset threshold value from a constant.

[0268] S135, add the first product and the second product to obtain an intra-class error parameter.

[0269] In the embodiment of the application, the data label recognition device adds the first product and the second product to obtain an intra-class error parameter.

[0270] In the embodiment of the application, the data label recognition device can calculate the intra-class error parameter L c (x s ,x t ):

[0271]

[0272] wherein, is the first intra-class error, is the second intra-class error, and β is a constant.

[0273] In some embodiments, S112 shown can also be implemented by S136 to S140, which will be described in combination with respective steps.

[0274] In the embodiment of the present application, the preset model with better recognition performance is obtained by calculating the intra-class error parameters between the source domain sample set and the target domain sample set, and updating the parameters in the back propagation of the model through the intra-class error parameters.

[0275] S136, the first natural logarithm of the length of each source domain sample vector and the second natural logarithm of the difference between each source domain sample vector and the second constant are obtained.

[0276] In the embodiment of the present application, the data label recognition device obtains the first natural logarithm of the length of each source domain sample vector and the second natural logarithm of the difference between each source domain sample vector and the second constant.

[0277] S137, the first natural logarithm is multiplied by the length of the preset expected output vector corresponding to each source domain sample vector to obtain a third product.

[0278] In the embodiment of the present application, the data label recognition device multiplies the first natural logarithm by the length of the preset expected output vector corresponding to each source domain sample vector to obtain a third product.

[0279] S138, the second natural logarithm is multiplied by the difference between the length of the preset expected output vector and the second constant to obtain a fourth product.

[0280] In the embodiment of the present application, the data label recognition device multiplies the second natural logarithm by the difference between the length of the preset expected output vector and the second constant to obtain a fourth product.

[0281] S139, the third product and the fourth product are added to obtain an intermediate source domain error parameter corresponding to each source domain sample vector, and then a plurality of intermediate source domain error parameters are obtained.

[0282] In the embodiment of the present application, the data label recognition device adds the third product and the fourth product to obtain an intermediate source domain error parameter corresponding to each source domain sample vector, and then a plurality of intermediate source domain error parameters are obtained.

[0283] S140, the plurality of intermediate source domain error parameters are added to obtain a sum of errors, and the negative of the ratio of the sum of errors to the number of the plurality of source domain sample vectors is obtained to obtain a source domain error parameter.

[0284] In the embodiment of the present application, the data label recognition device adds the plurality of intermediate source domain error parameters to obtain a sum of errors, and obtains the negative of the ratio of the sum of errors to the number of the plurality of source domain sample vectors to obtain a source domain error parameter.

[0285] In the embodiment of the present application, the output features of the source domain and the target domain samples obtained by the domain adapter at the last full connection layer are used to predict the labels of the samples by a label classifier. The label classifier is composed of a full connection network, the output size of the network is the number of label categories, and Softmax is used as the activation function. The calculation is as shown in formula (16):

[0286]

[0287] In the formula, xi is the output value of the i th neuron, and n is the number of label categories. The objective function of the label classifier is to minimize the classification error on the source domain data, which is defined as the source domain error parameter loss, and the calculation is as shown in formula (17):

[0288]

[0289] In the formula, n s is the number of source domain samples, is the expected output vector of the i th source domain sample (a vector set in advance), is the actual output vector of the i th source domain sample.

[0290] In the embodiment of the present application, the source domain error parameter of the source domain sample set is calculated, the parameters are updated in the back propagation of the model through the source domain error parameter, and a preset model with better recognition performance is obtained.

[0291] In some embodiments, Figure 1 The S102 shown can also be implemented by S141 to S142, which will be described in combination with each step.

[0292] S141, using a preset joint discriminative multi-order moment alignment model to process the data to be identified, to obtain a predicted output value.

[0293] In the embodiment of the present application, the data label recognition device uses a preset joint discriminative multi-order moment alignment model to process the data to be identified, to obtain a predicted output value.

[0294] S142, using the predicted output value to determine the corresponding target label in multiple labels.

[0295] In the embodiment of the present application, the data label recognition device uses the predicted output value to determine the corresponding target label in multiple labels.

[0296] In the embodiment of the present application, the data label recognition device locally stores a mapping relationship between multiple labels and multiple output values.

[0297] When the model training is completed, the data to be identified is input into the model, and the target label y t, calculate as formula (18):

[0298]

[0299] In the formula, denotes the actual output value of the data to be identified at the k node of the label classifier.

[0300] In some embodiments, referring to Figure 5 , Figure 5 An optional flowchart of the data label recognition method provided by the embodiments of the present application will be described in conjunction with the steps.

[0301] S201, obtaining sample data of a source domain and a target domain.

[0302] In the embodiments of the present application, the data label recognition device obtains sample data of a source domain and a target domain.

[0303] S202, extracting deep features of the source domain and target domain samples by a feature extractor.

[0304] In the embodiments of the present application, the data label recognition device extracts deep features of the source domain and target domain samples by a feature extractor.

[0305] S203, further extracting deep features of the source domain and target domain samples.

[0306] In the embodiments of the present application, the data label recognition device further extracts deep features of the source domain and target domain samples.

[0307] S204, calculating an inter-domain error parameter of the source domain and target domain samples.

[0308] In the embodiments of the present application, the data label recognition device calculates an inter-domain error parameter of the source domain and target domain samples.

[0309] S205, calculating an intra-class error parameter of the source domain and target domain samples.

[0310] In the embodiments of the present application, the data label recognition device calculates an intra-class error parameter of the source domain and target domain samples.

[0311] S206, calculating a source domain error parameter of the source domain samples by a label classifier.

[0312] In the embodiments of the present application, the data label recognition device calculates a source domain error parameter of the source domain samples by a label classifier.

[0313] S207, obtaining pseudo labels of the target domain samples by a label classifier.

[0314] In the embodiment of the present application, the data label recognition device obtains the pseudo label of the target domain sample through the label classifier.

[0315] S208, update the model parameter by integrating the inter-domain error parameter, the intra-class error parameter and the source domain error parameter.

[0316] In the embodiment of the present application, the data label recognition device updates the model parameter by integrating the inter-domain error parameter, the intra-class error parameter and the source domain error parameter.

[0317] S209, judge whether the iteration stopping condition is reached.

[0318] In the embodiment of the present application, the data label recognition device judges whether the iteration stopping condition is reached.

[0319] S210, output the prediction result of the model on the target domain through the label classifier.

[0320] In the embodiment of the present application, the data label recognition device outputs the prediction result of the model on the target domain through the label classifier.

[0321] The present application firstly constructs a feature extractor through a deep convolutional network, which is used to extract deep features of source domain and target domain samples, then aligns and narrows the deep features of source domain and target domain samples through a domain adapter, and finally outputs the prediction result of the target domain sample through a label classifier. The domain adapter adopts a joint discriminative multiple moment alignment algorithm, which firstly calculates the multiple central moment difference and aligns the deep features of source domain and target domain samples obtained by the feature extractor, then calculates the joint center loss of source domain and target domain samples and clusters according to the known label of the source domain sample and the pseudo label of the target domain sample obtained by the label classifier, and finally updates the parameters of the model by comprehensively considering the source domain loss, the center loss and the domain difference loss. When the model training is completed, the class of the target domain sample is predicted and output through the label classifier.

[0322] Referring to Figure 6 , Figure 6 The structural schematic diagram of the data label recognition device provided by the embodiment of the present application is shown.

[0323] The embodiment of the present application further provides a data label recognition device 800, which comprises a data acquisition unit 803 and a processing unit 804.

[0324] The data acquisition unit 803 is used to acquire the to-be-recognized data.

[0325] The processing unit 804 is used to process the to-be-recognized data by using a preset joint discriminative multiple moment alignment model, and determine the target label of the to-be-recognized data; wherein

[0326] The preset joint discriminative multiple order moment alignment model is obtained through joint iterative training based on the inter-domain error, the intra-class error between the source domain sample set and the target domain sample set, and the source domain error of the source domain sample set; the plurality of source domain samples in the source domain sample set and the plurality of target domain samples in the target domain sample set belong to different data fields, the plurality of source domain samples each correspond to a source domain sample label, and the plurality of target domain samples each do not have a corresponding label.

[0327] In the embodiment of the application, the data acquisition unit 803 in the data label recognition device 800 is configured to acquire the source domain sample set and the target domain sample set; the source domain sample set includes a plurality of source domain sample matrices corresponding to the plurality of source domain samples, and the source domain sample labels corresponding to each source domain sample; the target domain sample set includes a plurality of target domain sample matrices corresponding to the plurality of target domain samples; the processing unit 804 in the data label recognition device is configured to calculate the inter-domain error parameter, the intra-class error parameter between the source domain sample set and the target domain sample set, and the source domain error parameter of the source domain sample set by using the initial joint discriminative multiple order moment alignment model; determine the joint error parameter by using the inter-domain error parameter, the intra-class error parameter and the source domain error parameter; iteratively adjust the model parameters of the initial joint discriminative multiple order moment alignment model based on the joint error parameter until the training condition is met, and stop, to obtain the preset joint discriminative multiple order moment alignment model.

[0328] In the embodiment of the application, the processing unit 804 in the data label recognition device 800 is configured to input the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multiple order moment alignment model to obtain a plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices, and a plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices; calculate the inter-domain error parameter based on the plurality of source domain sample vectors and the plurality of target domain sample vectors; calculate a first intra-class error parameter based on the plurality of source domain sample vectors and a plurality of preset source domain deep feature center vectors; calculate a second intra-class error parameter based on the plurality of target domain sample vectors and a plurality of preset target domain deep feature center vectors; calculate the intra-class error parameter by using the first intra-class error and the second intra-class error; and calculate the source domain error parameter by using the plurality of source domain sample vectors and the plurality of source domain sample labels.

[0329] In the embodiment of the application, the processing unit 804 in the data label recognition device 800 is configured to multiply the inter-domain error parameter by a first preset balance coefficient to obtain a first parameter; multiply the intra-class error parameter by a second preset balance coefficient to obtain a second parameter; and add the first parameter, the second parameter and the source domain error parameter to obtain the joint error parameter.

[0330] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to input the plurality of source domain sample matrices and the plurality of target domain sample matrices into the initial joint discriminative multi-moment alignment model; to perform feature extraction on the plurality of source domain sample matrices and the plurality of target domain sample matrices through an initial feature extractor to obtain a plurality of source domain intermediate matrices corresponding to the plurality of source domain sample matrices and a plurality of target domain intermediate matrices corresponding to the plurality of target domain sample matrices; and to perform deep feature extraction on the plurality of source domain intermediate matrices and the plurality of target domain intermediate matrices through an initial neighborhood adapter to obtain the plurality of source domain sample vectors and the plurality of target domain sample vectors.

[0331] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to add the plurality of source domain sample vectors, and then compare them with the magnitude of the first matrix corresponding to the plurality of source domain sample vectors to obtain the source domain empirical expectation vector; add the plurality of target domain sample vectors, and then compare them with the magnitude of the second matrix corresponding to the plurality of target domain sample vectors to obtain the target domain empirical expectation vector; calculate the K-order source domain central moment vector by using the source domain empirical expectation vector and the first matrix; calculate the K-order target domain central moment vector by using the target domain empirical expectation vector and the second matrix; K is an integer greater than 1; and calculate the inter-domain error parameter based on the source domain empirical expectation vector, the target domain empirical expectation vector, the K-order source domain central moment vector, and the K-order target domain central moment vector.

[0332] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to obtain a source domain difference matrix by subtracting the source domain empirical expectation vector from each row of the first matrix, and then raising the source domain difference matrix to the power of i to obtain the i-th order source domain difference matrix; i is an integer greater than or equal to 1 and less than K; the sum of the vectors in each row of the i-th order source domain difference matrix is ​​calculated and then divided by the magnitude of the i-th order source domain difference matrix to obtain the i-th order source domain central moment vector, until i equals k, and thus obtaining the K-th order source domain central moment vector; the target domain difference matrix is ​​obtained by subtracting the target domain empirical expectation vector from each row of the second matrix, and then raising the target domain difference matrix to the power of i to obtain the i-th order target domain difference matrix; the sum of the vectors in each row of the i-th order target domain difference matrix is ​​calculated and then divided by the magnitude of the i-th order target domain difference matrix to obtain the i-th order target domain central moment vector, until i equals k, and thus obtaining the K-th order target domain central moment vector.

[0333] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to calculate the expected second norm of the difference between the source domain expected vector and the target domain expected vector, and divide the expected second norm by a preset threshold to obtain a first intermediate error parameter; calculate the central moment domain second norm of the difference between the i-th order source domain central moment vector and the i-th order target domain central moment vector, and divide the central moment domain second norm by the preset threshold to the power of i to obtain the i-th order error parameter, until i equals K to obtain the K-th order error parameter; and calculate the sum of the first intermediate error parameter and the error parameters from the i-th order to the K-th order to obtain the inter-domain error parameter.

[0334] In this embodiment of the invention, the processing unit 804 in the data label recognition device 800 is used to process the plurality of target domain sample vectors through the label classifier in the initial joint discriminative multi-moment alignment model to obtain multiple node output values ​​for each target domain sample vector; the node output value with the largest value among the multiple node output values ​​is determined to be the output value of the corresponding target domain sample vector, thereby obtaining multiple output values ​​of the plurality of target domain sample vectors; at least one output value greater than a threshold is determined among the multiple output values, and at least one target domain sample label corresponding to the at least one target domain sample vector is found through the at least one output value.

[0335] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to initialize multiple source domain sample tags to obtain multiple initial depth feature center vectors, calculate the depth feature center difference vector of each source domain sample tag using the multiple initial depth feature center vectors, calculate the next depth feature center vector of each source domain sample tag using the depth feature center difference vector, and iteratively calculate the next depth feature center vector as the initial depth feature center vector of each source domain sample tag until the depth feature center difference vector reaches a second predetermined condition, thereby obtaining the source domain depth feature center vector of each source domain sample tag.

[0336] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to initialize multiple target domain sample tags to obtain multiple target initial depth feature center vectors, calculate the target depth feature center difference vector for each target domain sample tag using the multiple target initial depth feature center vectors, calculate the next target depth feature center vector for each target domain sample tag using the target depth feature center difference vector, and iteratively calculate the next target depth feature center vector as the target initial depth feature center vector for each target domain sample tag until the target depth feature center difference vector reaches a second predetermined condition, thereby obtaining the target domain depth feature center vector for each target domain sample tag.

[0337] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to calculate the sum of the differences between the plurality of target domain sample vectors and the corresponding target initial depth feature center vectors; divide the sum of the differences by the sum of the number of the plurality of target domain sample vectors and a preset value to obtain the target depth feature center difference vector; subtract the product of the corresponding target depth feature center difference vector and a second preset value from each target initial depth feature center vector to obtain the next target depth feature center vector.

[0338] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to calculate the square of the L2 norm of each source domain sample vector and the corresponding source domain depth feature center vector to obtain the source domain square value; the source domain square values ​​corresponding to the plurality of source domain sample vectors are added together and divided by a third preset value to obtain the first intra-class error parameter.

[0339] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to calculate the square of the L2 norm of each target domain sample vector and the corresponding target domain depth feature center vector to obtain the target domain square value; the target domain square values ​​corresponding to the multiple target domain sample vectors are added together and divided by a third preset value to obtain the second intraclass error parameter.

[0340] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to calculate the first product of the first intra-class error and the fourth preset value; calculate the second product of the second intra-class error and the difference between the constant and the fourth preset threshold; and add the first product and the second product to obtain the intra-class error parameter.

[0341] In this embodiment of the invention, the processing unit 804 in the data tag identification device 800 is used to calculate the first natural logarithm of the magnitude of each source domain sample vector and the second natural logarithm of the difference between each source domain sample vector and the second constant; multiply the first natural logarithm by the magnitude of the preset expected output vector corresponding to each source domain sample vector to obtain a third product; multiply the second natural logarithm by the difference between the magnitude of the preset expected output vector and the second constant to obtain a fourth product; add the third product and the fourth product to obtain the intermediate source domain error parameter corresponding to each source domain sample vector, and then obtain multiple intermediate source domain error parameters; add the multiple intermediate source domain error parameters to obtain the sum of errors, and calculate the negative of the ratio of the sum of errors to the number of multiple source domain sample vectors to obtain the source domain error parameter.

[0342] In this embodiment of the invention, the processing unit 804 in the data tag recognition device 800 is used to process the data to be identified using the preset joint discriminative multi-moment alignment model to obtain a predicted output value; and to determine the corresponding target tag among multiple tags using the predicted output value.

[0343] In this embodiment of the invention, the data to be identified is acquired by the data acquisition unit 803; the data to be identified is processed by the processing unit 804 using a preset joint discriminative multi-moment alignment model to determine the target label of the data to be identified; wherein, the preset joint discriminative multi-moment alignment model is obtained by joint iterative training based on the inter-domain error and intra-class error between the source domain sample set and the target domain sample set, as well as the source domain error of the source domain sample set; multiple source domain samples in the source domain sample set belong to different data domains than multiple target domain samples in the target domain sample set, multiple source domain samples have corresponding source domain sample labels, and multiple target domain samples do not have corresponding labels. Since this scheme fully considers the inter-domain error and intra-class error between the source domain sample set and the target domain sample set, as well as the source domain error of the source domain sample set, and obtains the preset joint discriminative multi-moment alignment model by jointly training the source domain sample set and the target domain sample set, this model can improve the recognition accuracy of data from different domains.

[0344] It should be noted that, in the embodiments of the present invention, if the above-described data tag identification method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a data tag identification device (which may be a personal computer, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of the present invention are not limited to any specific hardware and software combination.

[0345] Correspondingly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0346] Correspondingly, this embodiment of the invention provides a data tag identification device, including a memory 802 and a processor 801. The memory 802 stores a computer program that can run on the processor 801. When the processor 801 executes the program, it implements the steps in the above method.

[0347] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the descriptions of the method embodiments of the present invention for understanding.

[0348] It should be noted that, Figure 7 This is a schematic diagram of a hardware entity of the data tag recognition device provided in an embodiment of the present invention, such as... Figure 7 As shown, the hardware entity of the data tag identification device 800 includes: a processor 801 and a memory 802, wherein;

[0349] The processor 801 typically controls the overall operation of the data tag identification device 800.

[0350] The memory 802 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 801 and the various modules in the data tag identification device 800. It can be implemented by flash memory or random access memory (RAM).

[0351] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0352] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0353] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the apparatus or units can be electrical, mechanical, or other forms.

[0354] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0355] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0356] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0357] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0358] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data tag recognition method, characterized in that, include: Obtain the data to be identified; The data to be identified includes: image data, video data, audio data, and text data; The data to be identified is processed using a preset joint discriminative multi-order moment alignment model to determine the target label of the data to be identified; wherein, The preset joint discriminative multi-moment alignment model is obtained by joint iterative training based on the inter-domain error and intra-class error between the source domain sample set and the target domain sample set, as well as the source domain error of the source domain sample set. The source domain sample set includes: multiple source domain sample matrices corresponding to multiple source domain samples, and source domain sample labels corresponding to each source domain sample; the target domain sample set includes: multiple target domain sample matrices corresponding to multiple target domain samples; wherein, the multiple source domain sample matrices and the multiple target domain sample matrices are input into an initial joint discriminative multi-moment alignment model to obtain multiple source domain sample vectors corresponding to the multiple source domain sample matrices, and multiple target domain sample vectors corresponding to the multiple target domain sample matrices; based on the multiple source domain sample vectors and the multiple target domain sample vectors, the inter-domain error is calculated; based on the multiple source domain sample vectors and preset multiple source domain depth features... The first intra-class error is calculated using the feature center vector; the second intra-class error is calculated based on the multiple target domain sample vectors and the preset multiple target domain deep feature center vectors; the intra-class error is calculated using the first intra-class error and the second intra-class error; the source domain error is calculated using the multiple source domain sample vectors and the multiple source domain sample labels; the multiple source domain samples in the source domain sample set and the multiple target domain samples in the target domain sample set belong to different data domains, the multiple source domain samples all have corresponding source domain sample labels, and the multiple target domain samples do not have corresponding labels; the multiple source domain samples and the multiple target domain samples include: image data, video data, audio data, and text data.

2. The data tag identification method according to claim 1, characterized in that, Before processing the data to be identified using a preset joint discriminative multi-moment alignment model to determine the target label of the data to be identified, the method further includes: Obtain the source domain sample set and the target domain sample set; The inter-domain error and intra-class error between the source domain sample set and the target domain sample set, as well as the source domain error of the source domain sample set, are calculated using the initial joint discriminative multi-moment alignment model. The joint error is determined using the inter-domain error, the intra-class error, and the source domain error; The model parameters of the initial joint discriminative multi-moment alignment model are iteratively adjusted based on the joint error until the training conditions are met, thereby obtaining the preset joint discriminative multi-moment alignment model.

3. The data tag identification method according to claim 1, characterized in that, The step of inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-moment alignment model to obtain the plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and the plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices includes: The multiple source domain sample matrices and the multiple target domain sample matrices are input into the initial joint discriminative multi-moment alignment model; The initial feature extractor extracts features from the multiple source domain sample matrices and the multiple target domain sample matrices to obtain multiple source domain intermediate matrices corresponding to the multiple source domain sample matrices and multiple target domain intermediate matrices corresponding to the multiple target domain sample matrices. Deep feature extraction is performed on the multiple source domain intermediate matrices and the multiple target domain intermediate matrices using an initial domain adapter to obtain the multiple source domain sample vectors and the multiple target domain sample vectors.

4. The data tag identification method according to claim 1, characterized in that, The step of calculating the inter-domain error based on the plurality of source domain sample vectors and the plurality of target domain sample vectors includes: Add the multiple source domain sample vectors together and then compare them with the magnitude of the first matrix corresponding to the multiple source domain sample vectors to obtain the source domain empirical expectation vector. Add the multiple target domain sample vectors together and then compare them with the magnitude of the second matrix corresponding to the multiple target domain sample vectors to obtain the target domain empirical expectation vector. The K-order source domain central moment vector is calculated using the source domain empirical expectation vector and the first matrix, and the K-order target domain central moment vector is calculated using the target domain empirical expectation vector and the second matrix; K is an integer greater than 1. The inter-domain error is calculated based on the source domain empirical expectation vector, the target domain empirical expectation vector, the K-order source domain central moment vector, and the K-order target domain central moment vector.

5. The data tag identification method according to claim 1, characterized in that, After inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-moment alignment model to obtain the plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and the plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices, and before calculating the second intra-class error based on the plurality of target domain sample vectors and the preset plurality of target domain deep feature center vectors, the method further includes: The multiple target domain sample vectors are processed by the label classifier in the initial joint discriminative multi-moment alignment model, and the multiple node output values ​​of each target domain sample vector are obtained. The node with the largest output value among the multiple node output values ​​is determined to be the output value of the corresponding target domain sample vector, thereby obtaining multiple output values ​​of the multiple target domain sample vectors; Among the plurality of output values, at least one output value greater than a threshold is determined, and at least one target domain sample label corresponding to at least one target domain sample vector is found through the at least one output value.

6. The data tag identification method according to claim 1, characterized in that, After inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-moment alignment model to obtain the plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and the plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices, and before calculating the first intra-class error based on the plurality of source domain sample vectors and the preset plurality of source domain deep feature center vectors, the method further includes: Multiple source domain sample labels are initialized to obtain multiple initial depth feature center vectors. The depth feature center difference vector of each source domain sample label is calculated using the multiple initial depth feature center vectors. The next depth feature center vector of each source domain sample label is calculated using the depth feature center difference vector. The next depth feature center vector is used as the initial depth feature center vector for each source domain sample label and iteratively calculated until the depth feature center difference vector reaches the second predetermined condition, thereby obtaining the source domain depth feature center vector for each source domain sample label.

7. The data tag identification method according to claim 1, characterized in that, After inputting the plurality of source domain sample matrices and the plurality of target domain sample matrices into an initial joint discriminative multi-moment alignment model to obtain the plurality of source domain sample vectors corresponding to the plurality of source domain sample matrices and the plurality of target domain sample vectors corresponding to the plurality of target domain sample matrices, and before calculating the second intra-class error based on the plurality of target domain sample vectors and the preset plurality of target domain deep feature center vectors, the method further includes: Multiple target domain sample labels are initialized to obtain multiple target initial depth feature center vectors. The target depth feature center difference vector of each target domain sample label is calculated using the multiple target initial depth feature center vectors. The next target depth feature center vector of each target domain sample label is calculated using the target depth feature center difference vector. The next target depth feature center vector is used as the target initial depth feature center vector of each target domain sample label for iterative calculation until the target depth feature center difference vector reaches the second predetermined condition, thereby obtaining the target domain depth feature center vector of each target domain sample label.

8. The data tag identification method according to claim 1, characterized in that, The calculation of the first intra-class error based on the multiple source domain sample vectors and the preset multiple source domain depth feature center vectors includes: Calculate the square of the L2 norm of each source domain sample vector and the corresponding source domain deep feature center vector to obtain the source domain square value; The first intra-class error is obtained by adding the squared values ​​of the source domains corresponding to the multiple source domain sample vectors and comparing them with a third preset value.

9. The data tag identification method according to claim 1, characterized in that, The calculation of the second intra-class error based on the multiple target domain sample vectors and the preset multiple target domain depth feature center vectors includes: Calculate the square of the L2 norm of each target domain sample vector and the corresponding target domain depth feature center vector to obtain the target domain square value; The squared values ​​of the target domains corresponding to the multiple target domain sample vectors are added together and compared with a third preset value to obtain the second intra-class error.

10. The data tag identification method according to claim 1, characterized in that, The calculation of the source domain error using the multiple source domain sample vectors and multiple source domain sample labels includes: Find the first natural logarithm of the magnitude of each source domain sample vector, and the second natural logarithm of the difference between each source domain sample vector and the second constant; Multiply the first natural logarithm by the preset expected output vector magnitude corresponding to each source domain sample vector to obtain the third product; Multiply the second natural logarithm by the difference between the preset expected output vector magnitude and the second constant to obtain the fourth product; Add the third product to the fourth product to obtain the intermediate source domain error corresponding to each source domain sample vector, and then obtain multiple intermediate source domain errors; The source domain errors are obtained by summing the errors of the multiple intermediate source domains, and the negative of the ratio of the sum of errors to the number of sample vectors in the multiple source domains is calculated.

11. The data tag identification method according to claim 1, characterized in that, The process of using a preset joint discriminative multi-moment alignment model to process the data to be identified and determine the target label of the data to be identified includes: The data to be identified is processed using the preset joint discriminative multi-moment alignment model to obtain the predicted output value; The target label is determined from multiple labels using the predicted output value.

12. A data tag identification device, characterized in that, include: The data acquisition unit is used to acquire the data to be identified. The data to be identified includes: image data, video data, audio data, and text data; The processing unit is used to process the data to be identified using a preset joint discriminative multi-moment alignment model to determine the target label of the data to be identified; wherein, The preset joint discriminative multi-moment alignment model is obtained by joint iterative training based on the inter-domain error and intra-class error between the source domain sample set and the target domain sample set, as well as the source domain error of the source domain sample set. The source domain sample set includes: multiple source domain sample matrices corresponding to multiple source domain samples, and source domain sample labels corresponding to each source domain sample; the target domain sample set includes: multiple target domain sample matrices corresponding to multiple target domain samples; wherein, the multiple source domain sample matrices and the multiple target domain sample matrices are input into an initial joint discriminative multi-moment alignment model to obtain multiple source domain sample vectors corresponding to the multiple source domain sample matrices, and multiple target domain sample vectors corresponding to the multiple target domain sample matrices; based on the multiple source domain sample vectors and the multiple target domain sample vectors, the inter-domain error is calculated; based on the multiple source domain sample vectors and preset multiple source domain depth features... The first intra-class error is calculated using the feature center vector; the second intra-class error is calculated based on the multiple target domain sample vectors and the preset multiple target domain deep feature center vectors; the intra-class error is calculated using the first intra-class error and the second intra-class error; the source domain error is calculated using the multiple source domain sample vectors and the multiple source domain sample labels; the multiple source domain samples in the source domain sample set and the multiple target domain samples in the target domain sample set belong to different data domains, the multiple source domain samples all have corresponding source domain sample labels, and the multiple target domain samples do not have corresponding labels; the multiple source domain samples and the multiple target domain samples include: image data, video data, audio data, and text data.

13. A data tag identification device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the program to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 11.

Images