Data classification method, device, computer readable storage medium and terminal device

Abstract

The present application belongs to the field of machine learning technology, and in particular, relates to a data classification method, device, computer-readable storage medium and terminal device. The method includes: obtaining target data; using a feature extraction network to extract features from the target data to obtain target data features; using a fully connected layer to estimate the distribution of the feature distribution to which the target data features belong to obtain a target feature distribution; based on the target feature distribution, resampling the target data features to obtain target resampling features; classifying the target data based on the target resampling features and a multi-center classification mechanism to obtain a target data category. In the above method, resampling can extract richer robust features; the multi-center classification mechanism adapts to the target data at a fine-grained level to reduce the probability that the sample is at the classification boundary. Accordingly, accurate data classification can be maintained in the face of unknown and diverse adversarial attacks.

Description

Technical Field

[0001] The present application belongs to the field of machine learning technology, and in particular, relates to a data classification method, apparatus, computer-readable storage medium, and terminal device. Background Art

[0002] With the development of machine learning and artificial intelligence technology, adversarial attacks have become an increasingly serious security issue. Adversarial attacks can deceive deep neural networks through carefully designed tiny perturbations that are not easily perceived by humans, causing them to make incorrect classifications during testing or applications. In order to deal with adversarial attacks, existing technologies generally use strategies such as adversarial training, defensive distillation, gradient masking, and input transformation to improve the robustness of deep neural networks. However, most of the existing strategies are only effective against specific types of adversarial attacks, but are difficult to deal with unknown adversarial attacks, resulting in low adversarial robustness in actual data classification applications. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a data classification method, apparatus, computer-readable storage medium and terminal device to solve the problem of low adversarial robustness of data classification methods in the prior art.

[0004] A first aspect of an embodiment of the present application provides a data classification method, which may include:

[0005] Get target data;

[0006] Extracting features from the target data using a feature extraction network to obtain target data features;

[0007] Using a fully connected layer to estimate the distribution of the target data features, to obtain a target feature distribution;

[0008] Based on the target feature distribution, feature resampling is performed on the target data feature to obtain a target resampling feature;

[0009] The target data is classified based on the target resampling features and the multi-center classification mechanism to obtain a target data category.

[0010] In a specific implementation of the first aspect, the using a fully connected layer to perform distribution estimation on a normal distribution to which the target data feature belongs to obtain a target feature distribution includes:

[0011] Using the first fully connected layer to estimate the mean of the distribution of the target data feature, to obtain an estimated mean of the target data feature;

[0012] Using the second fully connected layer to estimate the variance of the distribution of the target data feature, to obtain the estimated variance of the target data feature;

[0013] The target feature distribution is determined based on the estimated mean and the estimated variance.

[0014] In a specific implementation of the first aspect, classifying the target data based on the target resampling feature and the multi-center classification mechanism to obtain the target data category includes:

[0015] Respectively calculating the correlation between the target resampling feature and each category center corresponding to each category to obtain each correlation degree;

[0016] The category corresponding to the maximum value of each of the correlation degrees is determined as the target data category.

[0017] In a specific implementation of the first aspect, the feature extraction process, the distribution estimation process, the feature resampling process, and the data classification process are reasoning processes of a multi-center random deep neural network, and the learning process of the multi-center random deep neural network includes:

[0018] Establishing a training sample set; wherein the training sample set includes a preset number of training samples and corresponding real categories;

[0019] Using an initial deep neural network to perform data classification on each of the training samples in the training sample set to obtain an output category of each of the training samples;

[0020] Based on the true categories and output categories corresponding to each of the training samples, the initial deep neural network and the center are learned to obtain the multi-center random deep neural network.

[0021] In a specific implementation of the first aspect, the initial deep neural network includes an initial feature extraction network, an initial fully connected layer, and initial centers of each category;

[0022] The using of the initial deep neural network to perform data classification on each of the training samples in the training sample set to obtain an output category of each of the training samples includes:

[0023] Extracting features from the training samples using an initial feature extraction network to obtain initial sample features;

[0024] Using the initial fully connected layer to estimate the distribution of the initial sample features, to obtain the sample feature distribution;

[0025] Based on the sample feature distribution, feature resampling is performed on the initial sample feature to obtain a sample resampling feature;

[0026] Based on the correlation between the sample resampling features and the initial centers of the various categories, the output category of the training sample is determined.

[0027] In a specific implementation of the first aspect, resampling the initial sample features based on the sample feature distribution to obtain sample resampling features includes:

[0028] Determining a random perturbation amount of the initial sample characteristics from a standard normal distribution;

[0029] The sample resampling feature is determined in the sample feature distribution based on the random perturbation amount of the initial sample feature.

[0030] In a specific implementation of the first aspect, the initial deep neural network and the center are learned based on the real category and the output category corresponding to each of the training samples to obtain the multi-center random deep neural network, including:

[0031] Calculating classification loss based on the true category and output category of the training sample;

[0032] Calculating distribution estimation loss based on the sample feature distribution of the training sample and a standard normal distribution;

[0033] Calculate the multi-center loss based on the degree of difference between the initial centers of each category;

[0034] Based on the classification loss, the distribution estimation loss and the multi-center loss, the initial deep neural network and the center are learned to obtain the multi-center random deep neural network.

[0035] A second aspect of an embodiment of the present application provides a data classification device, which may include:

[0036] An acquisition module is used to acquire target data;

[0037] An extraction module, used to extract features from the target data using a feature extraction network to obtain target data features;

[0038] An estimation module, used for performing distribution estimation on the target data feature using a fully connected layer to obtain a target feature distribution;

[0039] A sampling module, used for performing feature resampling on the target data feature based on the target feature distribution to obtain a target resampling feature;

[0040] A classification module is used to classify the target data based on the target resampling characteristics and the multi-center classification mechanism to obtain a target data category.

[0041] In a specific implementation of the second aspect, the estimation module includes:

[0042] A mean estimation submodule, used to perform mean estimation on the distribution of the target data feature using the first fully connected layer to obtain an estimated mean of the target data feature;

[0043] A variance estimation submodule, configured to perform variance estimation on the distribution of the target data feature using the second fully connected layer to obtain an estimated variance of the target data feature;

[0044] A distribution determination submodule is used to determine the target feature distribution based on the estimated mean and the estimated variance.

[0045] In a specific implementation of the second aspect, the classification module includes:

[0046] A correlation degree calculation submodule is used to respectively calculate the correlation degree between the target resampling feature and each category center corresponding to each category, so as to obtain each correlation degree;

[0047] The category determination submodule is used to determine the category corresponding to the maximum value of each of the correlation degrees as the target data category.

[0048] In a specific implementation of the second aspect, the data classification device further includes:

[0049] A sample set establishment module, used to establish a training sample set; wherein the training sample set includes a preset number of training samples and corresponding real categories;

[0050] A data classification module, used to perform data classification on each of the training samples in the training sample set using an initial deep neural network to obtain an output category of each of the training samples;

[0051] The parameter learning module is used to learn the initial deep neural network and the center based on the real category and the output category corresponding to each of the training samples to obtain the multi-center random deep neural network.

[0052] In a specific implementation of the second aspect, the data classification module includes:

[0053] A feature extraction module, used to extract features from the training samples using an initial feature extraction network to obtain initial sample features;

[0054] A distribution estimation module, used to estimate the distribution of the initial sample features using the initial fully connected layer to obtain the sample feature distribution;

[0055] A resampling module, used for performing feature resampling on the initial sample features based on the sample feature distribution to obtain sample resampling features;

[0056] The category determination module is used to determine the output category of the training sample based on the correlation between the sample resampling feature and the initial center of each category.

[0057] In a specific implementation of the second aspect, the resampling module includes:

[0058] A disturbance amount determination submodule, used for determining the random disturbance amount of the initial sample characteristics from a standard normal distribution;

[0059] The resampling submodule is used to determine the sample resampling feature in the sample feature distribution based on the random perturbation amount of the initial sample feature.

[0060] In a specific implementation of the second aspect, the parameter learning module includes:

[0061] A classification loss calculation submodule, used to calculate the classification loss based on the true category and output category of the training sample;

[0062] A distribution estimation loss calculation submodule, used to calculate the distribution estimation loss based on the sample feature distribution of the training sample and the standard normal distribution;

[0063] A multi-center loss calculation submodule is used to calculate the multi-center loss based on the degree of difference between the initial centers of each category;

[0064] A parameter learning submodule is used to learn the initial deep neural network and the center based on the classification loss, the distribution estimation loss and the multi-center loss to obtain the multi-center random deep neural network.

[0065] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data classification methods are implemented.

[0066] The fourth aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned data classification methods when executing the computer program.

[0067] A fifth aspect of an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the steps of any one of the above-mentioned data classification methods.

[0068] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain target data; use a feature extraction network to extract features from the target data to obtain target data features; use a fully connected layer to estimate the distribution of the target data features to obtain target feature distribution; based on the target feature distribution, perform feature resampling on the target data features to obtain target resampling features; perform data classification on the target data based on the target resampling features and a multi-center classification mechanism to obtain target data categories. In the embodiments of the present application, resampling can extract richer robust features; the multi-center classification mechanism adapts to the target data at a fine-grained level to reduce the probability that the sample is at the classification boundary. Accordingly, accurate data classification can be maintained in the face of unknown and diverse adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 A schematic diagram of the data classification process in an embodiment of the present application;

[0071] Figure 2 Schematic diagram for correlation degree calculation;

[0072] Figure 3 Schematic diagram for calculating training loss;

[0073] Figure 4 This is a flow chart of an embodiment of a data classification method in an embodiment of the present application;

[0074] Figure 5 This is a structural diagram of an embodiment of a data classification device in an embodiment of the present application;

[0075] Figure 6 This is a schematic block diagram of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0078] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0079] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0080] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0081] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0082] With the development of machine learning and artificial intelligence technology, adversarial attacks have become an increasingly serious security issue. Adversarial attacks can deceive deep neural networks through carefully designed small perturbations, causing them to make incorrect classifications during testing or applications. In order to deal with adversarial attacks, existing technologies usually use strategies such as adversarial training, defensive distillation, gradient masking, and input transformation to improve the robustness of deep neural networks. However, most of the existing strategies are only effective against specific types of adversarial attacks, but are difficult to deal with unknown adversarial attacks, resulting in low adversarial robustness in actual data classification applications.

[0083] In view of this, the embodiments of the present application provide a data classification method, apparatus, computer-readable storage medium and terminal device to solve the problem that the generalization ability of deep neural networks in the prior art is poor, resulting in low adversarial robustness of data classification.

[0084] It should be noted that the execution subject of the method of the present application is a terminal device, specifically, it can be a common computing device such as a desktop computer, workstation, notebook, PDA, mobile phone, etc., or it can be other computing devices.

[0085] In the embodiment of the present application, a pre-trained multi-center random deep neural network can be used to classify data, where the data can be images, text, audio, etc.; specifically, please refer to Figure 1 The multi-center random deep neural network of the embodiment of the present application can extract features from the data, map the data into an embedding space (feature space), obtain the features of the data, and estimate the normal distribution of the features of the data; thereafter, the features of the data can be resampled from the normal distribution to extract richer and more robust features, and the data can be classified based on the resampled features, thereby better improving the adversarial robustness of the deep neural network under different attacks.

[0086] Before using the multi-center random deep neural network for actual data classification, the initial deep neural network can be trained to enable the initial deep neural network to learn parameters and centers to obtain the multi-center random deep neural network in the embodiment of the present application.

[0087] For the sake of ease of description, the learning process of the multi-center random deep neural network will be introduced first.

[0088] In an embodiment of the present application, a training sample set may be first established, and a multi-center random deep neural network may be learned based on the established training sample set.

[0089] Specifically, a preset number of data can be collected and the collected data can be used as training samples to obtain a preset number of training samples. The number of training samples can be set according to actual needs, and this application does not limit this. Afterwards, the category to which each training sample belongs can be marked to obtain the real category corresponding to each training sample; based on the preset number of training samples and the corresponding real category, a training sample set can be constructed.

[0090] It should be noted that the data here can be images, texts, audios and other types of data; different types of data can be used to establish different types of training sample sets, and the multi-center random deep neural network trained based on a certain type of training sample set can classify this type of data.

[0091] For example, if the training sample set includes various image samples, the multi-center random deep neural network trained based on the training sample set can classify the image data; for another example, if the training sample set includes various text samples, the multi-center random deep neural network trained based on the training sample set can classify the text data; for another example, if the training sample set includes various audio samples, the multi-center random deep neural network trained based on the training sample set can classify the audio data.

[0092] In a specific implementation of an embodiment of the present application, after the training samples are collected, the training samples can be preprocessed as needed; for example, if the training samples are images, the images can be cropped and aligned so that the included category subject is located in the middle of the image.

[0093] After the training sample set is constructed, the initial deep neural network can be learned using the training sample set to obtain the multi-center random deep neural network in the embodiment of the present application.

[0094] Here, the initial deep neural network may specifically include an initial feature extraction network, an initial fully connected layer, and initial centers of each category; wherein the initial feature extraction network may be used to perform feature extraction to obtain features of the data; the initial fully connected layer may estimate the distribution of features of the data to obtain an estimated feature distribution; based on the estimated feature distribution, the features may be resampled to obtain resampled features; thereafter, the data may be classified according to the degree of correlation between the resampled features and the initial centers of each category.

[0095] In an embodiment of the present application, the initial feature extraction network can be any common neural network for feature extraction in the prior art, including but not limited to residual neural network (Residual Network, ResNet), VGG (Visual Geometry Group) network, densely connected convolutional network (Densely Connected Convolutional Networks, DenseNet), vision Transformer (Vision Transformer, ViT) and other neural networks for feature extraction.

[0096] In an embodiment of the present application, the initial fully connected layer may include a parallel initial first fully connected layer and an initial second fully connected layer, the initial first fully connected layer can be used to estimate the mean of the distribution of data features, and the initial second fully connected layer can be used to estimate the variance of the distribution of data features.

[0097] In addition, you can also set various categories for the data, and you can set multiple centers for each category; if the correlation between the characteristics of the data and the center of a category is smaller, it can be considered that the possibility that the data belongs to this category is smaller; if the correlation between the characteristics of the data and the center of a category is greater, it can be considered that the possibility that the data belongs to this category is greater.

[0098] In an embodiment of the present application, the parameters of the initial feature extraction network, the parameters of the initial first fully connected layer, the parameters of the initial second fully connected layer, and the initial centers of each category can be randomly initialized to obtain an initial deep neural network, and the various parameters of the initial deep neural network can be continuously adjusted during the learning process.

[0099] Specifically, the initial deep neural network can be used to classify the data of each training sample in the training sample set to obtain the output category of each training sample. Afterwards, the initial deep neural network and the center can be learned based on the true category and output category corresponding to each training sample to obtain the multi-center random deep neural network in the embodiment of the present application.

[0100] Here, the initial feature extraction network can be used to extract features from the training samples to obtain initial sample features; then, the initial fully connected layer can be used to estimate the distribution of the initial sample features to obtain the sample feature distribution; wherein the sample feature distribution is the estimated distribution of the initial sample features.

[0101] Specifically, the initial first fully connected layer can be used to estimate the mean of the distribution of the initial sample features to estimate the feature mean; the initial second fully connected layer can also be used to estimate the variance of the distribution of the initial sample features to estimate the feature variance. Based on the feature mean and feature variance, the sample feature distribution can be determined.

[0102] Afterwards, the initial sample features may be resampled based on the sample feature distribution to obtain sample resampling features, and then classification may be performed based on the sample resampling features, and the category corresponding to the sample resampling features may be determined as the category of the training sample.

[0103] In the embodiment of the present application, a feature can be randomly collected from the sample feature distribution as the sample resampling feature. Specifically, it can be distributed from the standard state Determine the random perturbation amount of the initial sample characteristics , and can be based on the random perturbation , in the sample feature distribution Determine the sample resampling characteristics in ;in, is the characteristic mean of the sample characteristic distribution, is the output of the initial first fully connected layer, is the characteristic variance of the sample feature distribution, is the output of the initial second fully connected layer, is the initial sample feature, is the training sample.

[0104] Here, the sample perturbation amount can be multiplied by the feature variance, and the multiplication result can be added to the feature mean to obtain the sample resampling feature ,Right now, .

[0105] Afterwards, the output of the initial sample features can be determined based on the sample resampling features and the multi-center classification mechanism; specifically, the correlation between the sample resampling features and the initial centers corresponding to the categories can be calculated to determine the output of the initial sample features; here, the category corresponding to the maximum value of each correlation degree can be output, and the category is determined as the output category of the training sample.

[0106] By resampling features, the complexity and diversity of deep neural networks can be increased, allowing deep neural networks to automatically learn more robust feature representations during training, which helps improve the ability of deep neural networks to resist adversarial attacks.

[0107] Specifically, the sample resampling feature With Category Initial Center The degree of correlation between The calculation formula is as follows:

[0108] ,

[0109] in, is the calculation function of the preset correlation degree, is the vector calculated using this calculation function With vector The degree of correlation between , is the total number of categories; , The number of initial centers to include for a class.

[0110] In a specific implementation, .

[0111] In another specific implementation, .

[0112] Afterwards, the category corresponding to the maximum value in each correlation degree can be determined as the output category of the training sample. Through the multi-center classification mechanism of the embodiment of the present application, it is possible to adapt to the sample data at a fine-grained level, reduce the probability that the sample is located at the classification boundary, and thus reduce the risk of the sample being successfully attacked, which helps to improve the classification ability of the deep neural network.

[0113] For example, see Figure 2 , there are categories 1, category 2, ..., here, we can first calculate the correlation between the sample resampling features and the initial centers of category 1; specifically, the initial centers of category 1 are center 11, center 12, ..., center 1M, then we can calculate the correlation between the sample resampling features and center 11 (correlation 11), the correlation between the sample resampling features and center 12 (correlation 12), ..., the correlation between the sample resampling features and center 1M (correlation 1M), respectively; then, we can calculate the correlation between the sample resampling features and the initial centers of category 2; specifically, the initial centers of category 2 are If they are center 21, center 22, ..., center 2M respectively, then the correlation between the sample resampling feature and center 21 (correlation 21), the correlation between the sample resampling feature and center 22 (correlation 22), ..., the correlation between the sample resampling feature and center 2M (correlation 2M) can be calculated; according to the above method, the correlation between the sample resampling feature and each initial center corresponding to each category can be calculated in turn; if the maximum value of the above correlation degrees is correlation 22, the category corresponding to correlation 22 can be determined as the output category of the training sample, that is, category 2 can be determined as the output category of the training sample.

[0114] Here, the maximum value among the various correlation degrees can be specifically expressed as:

[0115] ,

[0116] in, is the output category of the training sample, For the The maximum value of the correlation between the initial centers of the categories and the sample resampling features, It can be expressed as:

[0117] ,

[0118] in, Represents the supremum.

[0119] After obtaining the output category of each training sample, the training loss of the initial deep neural network can be calculated based on the true category and output category corresponding to the training sample; based on the training loss, the initial deep neural network and the center can be learned; when the learning is completed, the multi-center random deep neural network in the embodiment of the present application can be obtained.

[0120] Since the initial deep neural network learning process specifically includes the process of feature extraction, the process of feature resampling, the process of distribution estimation and the process of data classification, the embodiment of the present application can comprehensively consider each processing process to calculate the training loss, so as to more comprehensively and accurately evaluate the loss of the learning process.

[0121] In the embodiment of the present application, the training loss may specifically include classification loss, distribution estimation loss and multi-center loss; see Figure 3 , classification loss is a kind of loss in the process of data classification based on multi-centers. Classification loss can be used to characterize the degree of difference between the output category of data and the true category; distribution estimation loss is the loss in the process of distribution estimation (specifically including the process of mean estimation and the process of variance estimation). Distribution estimation loss can be used to characterize the degree of difference between the estimated feature distribution and the standard normal distribution. Multi-center loss is another kind of loss in the process of data classification based on multi-centers. Multi-center loss can be used to characterize the degree of difference between centers of different categories.

[0122] When learning the initial deep neural network, the classification loss, distribution estimation loss and multi-center loss can be calculated separately to obtain the training loss of the initial deep neural network, and the initial deep neural network and the center can be learned based on the training loss.

[0123] Specifically, the classification loss can be calculated according to a preset classification loss function, which can be any function in the prior art for calculating the difference between the predicted category and the actual category; here, the classification loss function can be specifically a cross entropy loss function. Based on the cross entropy loss function, the classification loss is calculated. The formula is as follows:

[0124] ,

[0125] in, for:

[0126] ,

[0127] in, .

[0128] In an embodiment of the present application, if the distribution of the estimated data features is closer to the standard normal distribution, the distribution of the data in the embedding space can be made more compact, which is helpful in extracting richer and more robust data features during resampling, thereby helping to improve the robustness of the deep neural network. Therefore, the distribution estimation loss can also be calculated based on the degree of difference between the estimated normal distribution and the standard normal distribution, and the distribution estimation process can be guided and optimized based on the distribution estimation loss in the subsequent learning process to make the estimated distribution closer to the standard normal distribution.

[0129] Specifically, the distribution estimation loss can be calculated based on the sample feature distribution of the training sample and the standard normal distribution. , the specific calculation formula is as follows:

[0130] ,

[0131] in, is the preset hyperparameter, The value of can be set according to actual needs, and this application does not limit this.

[0132] In an embodiment of the present application, if the degree of difference between the centers of each category is large, it is easier and more accurate to distinguish the category to which the data belongs. Therefore, the multi-center loss can also be calculated based on the degree of difference between the initial centers of each category, and the centers of each category can be guided and optimized based on the multi-center loss in the subsequent learning process so that the centers of different categories can maintain a large difference; through the above-mentioned multi-center optimization strategy, information sharing and collaborative work among multiple centers can be achieved.

[0133] Specifically, the multi-center loss The calculation formula is as follows:

[0134] ,

[0135] in, For the The initial centers of the classes are The minimum value of the difference between the initial centers of the class; specifically, , Represents the infimum.

[0136] Based on the classification loss, distribution estimation loss and multi-center loss, the training loss can be calculated; specifically, the classification loss, distribution estimation loss and multi-center loss can be weighted and summed according to preset weights to obtain the training loss; wherein, the weights corresponding to each loss can be set according to actual needs, and this application does not limit this.

[0137] In a specific way, the weight corresponding to the classification loss can be set to 1, and the weight corresponding to the distribution estimation loss can be set to , set the weight corresponding to the multi-center loss to , and The value of can be set according to actual needs, and this application does not limit it; the training loss can be specifically expressed as:

[0138] .

[0139] Afterwards, the model parameters of the initial deep neural network and the initial center of the category can be learned based on the training loss. Specifically, the model parameters of the initial deep neural network and the initial center of the category can be learned by a stochastic gradient descent strategy; here, the parameters of the feature extraction network after learning are for:

[0140] ,

[0141] in, Initialize the parameters of the initial feature extraction network, is the learning rate, The value of can be set according to actual needs, and this application does not limit it; for right gradient.

[0142] Parameters of the initial first fully connected layer after learning for:

[0143] ,

[0144] in, Initialize the parameters of the first fully connected layer. for right gradient.

[0145] Parameters of the initial second fully connected layer after learning for:

[0146] ,

[0147] in, Initialize the parameters of the initial second fully connected layer, for right gradient.

[0148] After-school center for:

[0149] ,

[0150] in, for right gradient.

[0151] In an embodiment of the present application, when a preset stop learning condition is met, the above learning process can be stopped, and the obtained deep neural network can be determined as a multi-center random deep neural network in an embodiment of the present application.

[0152] In a specific implementation of the embodiment of the present application, the training sample set can be divided into a training set and a validation set, and the training set can be used for model learning, and the validation set can be used for model evaluation; or, the validation set can be established according to the above method for establishing a training sample set for model evaluation; after learning the initial deep neural network and the center, the validation set can be used to evaluate the learned deep neural network to obtain the accuracy of the learned deep neural network on the validation set; if the accuracy is less than the preset accuracy threshold, it can be considered that the stop learning condition is not met temporarily, and the model learning can be continued at this time; if the accuracy is greater than or equal to the preset accuracy threshold, it can be determined that the stop learning condition is met, at this time, the above learning process can be stopped, and the obtained deep neural network can be determined as a multi-center random deep neural network in the embodiment of the present application. Among them, the accuracy threshold can be set according to actual needs, and this application does not limit this.

[0153] In another specific implementation of the embodiment of the present application, if the number of iterations of model learning is less than a preset iteration number threshold, it can be considered that the stop learning condition is not met temporarily, and the model learning can be continued at this time; if the number of iterations of model learning is greater than or equal to the iteration number threshold, it can be determined that the stop learning condition is met, and at this time, the above learning process can be stopped, and the obtained deep neural network can be determined as the multi-center random deep neural network in the embodiment of the present application. Among them, the iteration number threshold can be set according to actual needs, and this application does not limit this.

[0154] When learning stops, the initial feature extraction network after learning can be determined as the feature extraction network of the multi-center random deep neural network, the initial first fully connected layer after learning can be determined as the first fully connected layer of the multi-center random deep neural network, the initial second fully connected layer after learning can be determined as the second fully connected layer of the multi-center random deep neural network, and the initial centers of each category after learning can be determined as the category centers of each category of the multi-center random deep neural network.

[0155] After obtaining the multi-center random deep neural network in the embodiment of the present application, it can be applied to actual data classification. Figure 4In an embodiment of the present application, an embodiment of a data classification method may include the following steps S401 to S405:

[0156] Step S401: Acquire target data.

[0157] In the embodiment of the present application, the data to be classified can be referred to as target data, and the multi-center random deep neural network can be used to classify the target data. Here, the target data can be of the type of image, text, or audio, etc., which is not limited in the embodiment of the present application.

[0158] In a specific implementation of the embodiment of the present application, the target data may be data collected or received in real time, for example, the target data may be images, text or audio collected in real time.

[0159] In another specific implementation of the embodiment of the present application, the target data may be data collected in advance and stored in a preset storage module. For example, the target data may be an image, text, or audio stored in the storage module.

[0160] Here, the target data may be data without adding disturbance (ie, clean data), or, when the performance of the multi-center random deep neural network needs to be tested and verified, the target data may also be data with adding disturbance.

[0161] In a specific implementation, the target data is the data to which disturbance is added; specifically, multiple rounds of attacks (white box attacks or black box attacks) may be performed on the initial data to obtain the data to which disturbance is added, and the data to which disturbance is added may be used as the target data; here, the intensity of the attack and the distance norm may be set as required, which is not limited in the embodiment of the present application; for example, the initial data may be attacked with an intensity of 8 / 255 and a distance norm of (Chebyshev distance) attack.

[0162] Step S402: extract features from the target data using a feature extraction network to obtain target data features.

[0163] In an embodiment of the present application, a feature extraction network of a multi-center random deep neural network can be used to extract features from target data to obtain target data features.

[0164] Specifically, the feature extraction network of the multi-center random deep neural network can map the target data into the embedded space (feature space) to obtain the features of the target data, which can be referred to as target data features here.

[0165] Step S403: Use the fully connected layer to estimate the distribution of target data features to obtain target feature distribution.

[0166] Afterwards, the fully connected layer of the multi-center random deep neural network can be used to estimate the distribution of target data features and obtain the target feature distribution.

[0167] Specifically, the fully connected layer includes a first fully connected layer and a second fully connected layer. The first fully connected layer can be used to estimate the mean of the distribution of the target data feature to obtain the estimated mean of the target data feature; the second fully connected layer can also be used to estimate the variance of the distribution of the target data feature to obtain the estimated variance of the target data feature. Based on the estimated mean and the estimated variance, the target feature distribution can be determined.

[0168] The first fully connected layer and the second fully connected layer are two parallel branches of the multi-center random deep neural network. Therefore, the above mean estimation process and variance estimation process are two parallel processes.

[0169] Step S404: Based on the target feature distribution, feature resampling is performed on the target data features to obtain target resampling features.

[0170] In the embodiment of the present application, random sampling can be performed in the target feature distribution to obtain resampled features, which can be referred to as target resampled features. Afterwards, data classification can be performed based on the target resampled features.

[0171] Step S405: classify the target data based on the target resampling characteristics and the multi-center classification mechanism to obtain the target data category.

[0172] Afterwards, the category of the target resampling feature may be determined from among the multiple categories, and the category may be determined as the category to which the target data belongs. Here, the category to which the target data belongs may be referred to as the target data category.

[0173] In an embodiment of the present application, each category may correspond to multiple centers. When performing data classification, the correlation between the target resampling features and the category centers corresponding to each category may be calculated to obtain each correlation degree, and the category corresponding to the maximum value of each correlation degree may be determined as the target data category; the calculation process of the above correlation degree may refer to the detailed description above and will not be repeated here.

[0174] In summary, the embodiment of the present application obtains target data; uses a feature extraction network to extract features from the target data to obtain target data features; uses a fully connected layer to estimate the distribution of the target data features to obtain target feature distribution; based on the target feature distribution, performs feature resampling on the target data features to obtain target resampling features; based on the target resampling features and a multi-center classification mechanism, performs data classification on the target data to obtain a target data category. In the embodiment of the present application, resampling can extract richer robust features; the multi-center classification mechanism adapts to the target data at a fine-grained level to reduce the probability that the sample is at the classification boundary. Accordingly, accurate data classification can be maintained in the face of unknown and diverse adversarial attacks.

[0175] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0176] Corresponding to a data classification method described in the above embodiment, Figure 5 A structural diagram of an embodiment of a data classification device provided in an embodiment of the present application is shown.

[0177] In an embodiment of the present application, a data classification device may include:

[0178] An acquisition module 501 is used to acquire target data;

[0179] An extraction module 502 is used to extract features from the target data using a feature extraction network to obtain target data features;

[0180] An estimation module 503 is used to estimate the distribution of the target data feature using a fully connected layer to obtain a target feature distribution;

[0181] A sampling module 504 is used to perform feature resampling on the target data feature based on the target feature distribution to obtain a target resampled feature;

[0182] The classification module 505 is used to classify the target data based on the target resampling characteristics and the multi-center classification mechanism to obtain the target data category.

[0183] In a specific implementation of the embodiment of the present application, the estimation module includes:

[0184] A mean estimation submodule, used to perform mean estimation on the distribution of the target data feature using the first fully connected layer to obtain an estimated mean of the target data feature;

[0185] A variance estimation submodule, configured to perform variance estimation on the distribution of the target data feature using the second fully connected layer to obtain an estimated variance of the target data feature;

[0186] A distribution determination submodule is used to determine the target feature distribution based on the estimated mean and the estimated variance.

[0187] In a specific implementation of the embodiment of the present application, the classification module includes:

[0188] A correlation degree calculation submodule is used to respectively calculate the correlation degree between the target resampling feature and each category center corresponding to each category, so as to obtain each correlation degree;

[0189] The category determination submodule is used to determine the category corresponding to the maximum value of each of the correlation degrees as the target data category.

[0190] In a specific implementation of the embodiment of the present application, the data classification device further includes:

[0191] A sample set establishment module, used to establish a training sample set; wherein the training sample set includes a preset number of training samples and corresponding real categories;

[0192] A data classification module, used to perform data classification on each of the training samples in the training sample set using an initial deep neural network to obtain an output category of each of the training samples;

[0193] The parameter learning module is used to learn the initial deep neural network and the center based on the real category and the output category corresponding to each of the training samples to obtain the multi-center random deep neural network.

[0194] In a specific implementation of the embodiment of the present application, the data classification module includes:

[0195] A feature extraction module, used to extract features from the training samples using an initial feature extraction network to obtain initial sample features;

[0196] A distribution estimation module, used to use an initial fully connected layer to perform distribution estimation on the normal distribution to which the initial sample features belong, to obtain a sample feature distribution;

[0197] A resampling module, used for performing feature resampling on the initial sample features based on the sample feature distribution to obtain sample resampling features;

[0198] The category determination module is used to determine the output category of the training sample based on the correlation between the sample resampling feature and the initial center of each category.

[0199] In a specific implementation of the embodiment of the present application, the resampling module includes:

[0200] A disturbance amount determination submodule, used for determining the random disturbance amount of the initial sample characteristics from a standard normal distribution;

[0201] The resampling submodule is used to determine the sample resampling feature in the sample feature distribution based on the random perturbation amount of the initial sample feature.

[0202] In a specific implementation of the embodiment of the present application, the parameter learning module includes:

[0203] A classification loss calculation submodule, used to calculate the classification loss based on the true category and output category of the training sample;

[0204] A distribution estimation loss calculation submodule, used to calculate the distribution estimation loss based on the sample feature distribution of the training sample and the standard normal distribution;

[0205] A multi-center loss calculation submodule is used to calculate the multi-center loss based on the degree of difference between the initial centers of each category;

[0206] A parameter learning submodule is used to learn the initial deep neural network and the center based on the classification loss, the distribution estimation loss and the multi-center loss to obtain the multi-center random deep neural network.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0209] Figure 6 A schematic block diagram of a terminal device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0210] like Figure 6 As shown, the terminal device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the above-mentioned data classification method embodiments are implemented, for example Figure 4Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 The functions of the acquisition module 501 to the classification module 505 are shown.

[0211] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the terminal device 6.

[0212] Those skilled in the art will understand that Figure 6 It is only an example of the terminal device 6 and does not constitute a limitation of the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 6 may also include input and output devices, network access devices, buses, etc.

[0213] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0214] The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 61 may also include both an internal storage unit of the terminal device 6 and an external storage device. The memory 61 is used to store the computer program and other programs and data required by the terminal device 6. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0215] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0216] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0217] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0218] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0219] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0221] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0222] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data classification method, characterized in that: include: Acquire target data, where the target data is an image, text or audio; Extracting features from the target data using a feature extraction network to obtain target data features; Using a fully connected layer to estimate the distribution of the target data features, to obtain a target feature distribution; Based on the target feature distribution, feature resampling is performed on the target data feature to obtain a target resampling feature; Classifying the target data based on the target resampling feature and the multi-center classification mechanism to obtain a target data category; The target data is classified based on the target resampling feature and the multi-center classification mechanism to obtain the target data category, including: Respectively calculating the correlation between the target resampling feature and each category center corresponding to each category to obtain each correlation degree; Determine the category corresponding to the maximum value of each of the correlation degrees as the target data category; The process of determining the target resampling feature includes: Determine the random perturbation amount of the initial sample characteristics , and based on the random perturbation , in the sample feature distribution In the above method, the random perturbation is multiplied by the feature variance, and the result of multiplying the random perturbation by the feature variance is added to the feature mean to obtain the target resampling feature ,Right now, ;in, is the characteristic mean of the sample characteristic distribution, is the output of the initial first fully connected layer, is the characteristic variance of the sample feature distribution, is the output of the initial second fully connected layer, is the initial sample feature, is the training sample.

2. The data classification method according to claim 1, characterized in that: The method of using a fully connected layer to estimate the distribution of the target data feature to obtain the target feature distribution includes: Using the first fully connected layer to estimate the mean of the distribution of the target data feature, to obtain an estimated mean of the target data feature; Using the second fully connected layer to estimate the variance of the distribution of the target data feature, to obtain the estimated variance of the target data feature; The target feature distribution is determined based on the estimated mean and the estimated variance.

3. The data classification method according to claim 1, characterized in that: The process of feature extraction, the process of distribution estimation, the process of feature resampling and the process of data classification are the reasoning process of the multi-center random deep neural network. The learning process of the multi-center random deep neural network includes: Establishing a training sample set; wherein the training sample set includes a preset number of training samples and corresponding real categories; Using an initial deep neural network to perform data classification on each of the training samples in the training sample set to obtain an output category of each of the training samples; Based on the true categories and output categories corresponding to each of the training samples, the initial deep neural network and the center are learned to obtain the multi-center random deep neural network.

4. The data classification method according to claim 3, characterized in that: The initial deep neural network includes an initial feature extraction network, an initial fully connected layer, and initial centers of each category; The using of the initial deep neural network to perform data classification on each of the training samples in the training sample set to obtain an output category of each of the training samples includes: Extracting features from the training samples using an initial feature extraction network to obtain initial sample features; Using the initial fully connected layer to estimate the distribution of the initial sample features, to obtain the sample feature distribution; Based on the sample feature distribution, feature resampling is performed on the initial sample feature to obtain a sample resampling feature; Based on the correlation between the sample resampling features and the initial centers of the various categories, the output category of the training sample is determined.

5. The data classification method according to claim 4, characterized in that: The initial deep neural network and the center are learned based on the real category and the output category corresponding to each of the training samples to obtain the multi-center random deep neural network, including: Calculating classification loss based on the true category and output category of the training sample; Calculating distribution estimation loss based on the sample feature distribution of the training sample and a standard normal distribution; Calculate the multi-center loss based on the degree of difference between the initial centers of each category; Based on the classification loss, the distribution estimation loss and the multi-center loss, the initial deep neural network and the center are learned to obtain the multi-center random deep neural network.

6. A data classification device, characterized in that: include: An acquisition module, used to acquire target data, wherein the target data is an image, text or audio; An extraction module, used to extract features from the target data using a feature extraction network to obtain target data features; An estimation module, used for performing distribution estimation on the target data feature using a fully connected layer to obtain a target feature distribution; A resampling module, used for performing feature resampling on the target data feature based on the target feature distribution to obtain a target resampling feature; A classification module, used for classifying the target data based on the target resampling feature and the multi-center classification mechanism to obtain a target data category; The classification module comprises: A correlation degree calculation submodule is used to respectively calculate the correlation degree between the target resampling feature and each category center corresponding to each category, so as to obtain each correlation degree; A category determination submodule, used to determine the category corresponding to the maximum value of each of the correlation degrees as the target data category; The resampling module includes: a disturbance amount determination submodule for distributing the disturbance amount from the standard state Determine the random perturbation amount of the initial sample characteristics ; A resampling submodule for random perturbations based on the initial sample characteristics , in the sample feature distribution In the above method, the random perturbation is multiplied by the feature variance, and the result of multiplying the random perturbation by the feature variance is added to the feature mean to obtain the target resampling feature ,Right now, ;in, is the characteristic mean of the sample characteristic distribution, is the output of the initial first fully connected layer, is the characteristic variance of the sample feature distribution, is the output of the initial second fully connected layer, is the initial sample feature, is the training sample.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the data classification method according to any one of claims 1 to 5 are implemented.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data classification method according to any one of claims 1 to 5 are implemented.

Images