Multi-label information processing method based on multi-view disturbance

Abstract

The invention relates to the technical field of machine learning and data mining, and discloses a multi-view perturbation-based multi-label information processing method, which comprises the following steps: step 1, data collection and preprocessing, step 2, label specific feature generation based on LIFT, step 3, multi-view data perturbation and feature selection, and step 4, multi-view perturbation and feature selection. And step 4, constructing a dichotomous learning device and voting integration. According to the method, appropriate distance measurement is selected for different types of data to determine the sample similarity, and a basis is provided for subsequent operation. The samples are disturbed by calculating the neighbor sample characteristic value standard deviation, the neighbor is determined according to the characteristic correlation, and the relevant statistical information standard deviation is calculated to disturb the characteristics, so that the diversity of the data and the characteristics is increased. The ReliefF algorithm is used for carrying out feature selection, importance is evaluated according to differences between features and similar and heterogeneous samples, a threshold value is set for screening the features, the calculation complexity is reduced, key features are reserved, and the classification accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning and data mining, and in particular to a multi-label information processing method based on multi-view disturbance. Background Art

[0002] With the rapid development of information technology, labeled data has shown an exponential growth trend, which contains rich and diverse information. In the field of machine learning and data mining, researchers are committed to mining valuable information from these labeled data, and have carried out a series of research related to label classification.

[0003] Traditional single-label classification mainly covers binary classification and multi-classification. Its core is to assign a single label in the label set to an instance. There are many mature algorithms in this field. These algorithms have excellent performance and are widely used in many fields. However, in actual scenarios, the limitations of single-label classification are gradually emerging. In fact, an instance is often associated with multiple labels. For example, in text classification, an electronic medical record may involve multiple disease labels such as diabetes, hypertension, and coronary heart disease; in the field of image classification, a tongue diagnosis image can simultaneously express tongue color, tongue coating, shape and other features, and then infer the patient's multiple physical sign labels; in music classification, a piece of audio can convey a variety of information such as piano performance, classical music style, and Mozart's works.

[0004] This classification problem in which one instance corresponds to multiple labels is defined as multi-label classification, and its essence is to associate one instance with a set of labels. Compared with single-label classification, multi-label classification faces many challenges. First, the number of labels corresponding to each instance is not fixed and there is more than one; second, there is semantic correlation between labels, which makes the classification process more complicated; in addition, there is label imbalance in data sets in some fields, which further increases the difficulty of multi-label classification tasks. The existence of these problems has prompted researchers to seek more effective multi-label information processing methods to cope with complex data classification needs in the real world. The present invention is based on such a background and proposes a multi-label information processing method based on multi-view perturbation. Summary of the invention

[0005] The present invention mainly solves the technical problems existing in the above-mentioned prior art and provides a multi-label information processing method based on multi-view disturbance.

[0006] In order to achieve the above object, the present invention adopts the following technical solution, a multi-label information processing method based on multi-view disturbance, comprising the following steps:

[0007] Step 1: Data collection and preprocessing:

[0008] Collect multi-label datasets from multiple fields, including disease diagnosis datasets in the medical field, scene classification datasets in the image field, and news classification datasets in the text field;

[0009] The collected data was cleaned, and the missing values ​​were filled with statistical methods, the numerical data were filled with the mean, and the categorical label data were filled with the mode; the outliers were detected and corrected using the Z-Score standardization method, and the median was used to replace the outliers;

[0010] Normalize different types of data. Use the minimum-maximum normalization formula to normalize the original pixel values ​​of image data to [0,1].

[0011] Step 2: LIFT-based label-specific feature generation:

[0012] The positive and negative sample sets are divided according to the association between samples and labels. Samples related to the labels belong to the positive sample set, otherwise they belong to the negative sample set.

[0013] The k-means clustering algorithm is used to cluster the positive and negative sample sets of each label, and the distance from the sample to the cluster center is calculated. The cluster center is determined by minimizing the objective function. After setting a reasonable number of clusters k, the distance from the sample to the cluster center is used as a new feature to construct a label-specific feature space.

[0014] Step 3: Multi-view data perturbation and feature selection:

[0015] Distance metric selection: For image data, the Euclidean distance based on pixel features is used, and for text data, the cosine distance in the word vector space is used; the number of neighbors of each sample is set according to the size of the data set and the feature distribution, and the standard deviation of the eigenvalues ​​of each sample's neighbor set is calculated. The sample is perturbed according to the standard deviation, which can be achieved by adding random noise or eigenvalue scaling;

[0016] Feature perspective perturbation: Calculate the correlation or similarity between features. Image features use the cosine similarity between feature vectors. Text vocabulary features use the co-occurrence frequency of words. Determine the neighbors of each feature based on the correlation or similarity, calculate the standard deviation of the neighbor feature statistics, and perturb the features based on the standard deviation. For text data, adjust the vocabulary feature weights, and for image data, scale the feature values.

[0017] Feature selection: Use the ReliefF algorithm to select features, calculate the difference between features and similar samples and heterogeneous samples to evaluate feature importance, set a threshold to filter features, and set the threshold based on the mean and standard deviation of all feature weights. Select features with weights higher than the threshold as the final feature subset;

[0018] Step 4: Build a binary classification learner and voting ensemble:

[0019] Use the selected feature subsets to build multiple binary classification learners, select appropriate classification algorithms and set parameters based on the characteristics of the feature subsets and data distribution, such as support vector machines, decision trees, and neural networks, and use the training data to train the binary classification learners using the gradient descent method;

[0020] For the test samples, multiple learners are used to predict and vote on the majority principle to combine the prediction results of multiple learners.

[0021] As a preference, in the data collection and preprocessing step, the data set is assumed to be D = {x1, x2, ..., x n}, where each sample x i Contains m features, if feature x ij If there are missing values, they are replaced by feature x j The mean Among them, x ij Not equal to missing value, here N j Indicates the number of samples with non-missing values ​​in feature j to ensure the integrity of the data.

[0022] Preferably, in the data collection and preprocessing step, for the sample x ij The eigenvalue of , its Z-Score is expressed as where μj is the mean of feature j, σ j is the standard deviation of feature j, when |z ij When |>3, it is determined as an outlier. When it is determined as an outlier, the median is used to replace the outlier.

[0023] Preferably, in the step of generating label-specific features based on LIFT, the positive sample set is The cluster centers are c1, c2, …, c k , then the distance d(p j ,c t ), by minimizing the objective function To determine the cluster centers.

[0024] Preferably, in the multi-view data perturbation and feature selection step, for two samples x of the image data i =(x i1 ,x i2 ,...,x im ) and x j =(x j1 ,x j2 ,...,x jm ), whose Euclidean distance Where m is the pixel feature dimension.

[0025] Preferably, in the multi-view data perturbation and feature selection steps, the word vectors of two samples of text data are represented as v i =(v i1 ,v i2 ,...,v in ) and v j =(v j1 ,v j2 ,...,v jn ), the cosine distance Where n is the word vector dimension.

[0026] Preferably, in the multi-view data perturbation and feature selection steps, for the image feature vector f i =(f i1 ,f i2 ,...,f im ) and f j =(f j1 ,f j2 ,...,f jm ), the cosine similarity Where m is the feature dimension.

[0027] Preferably, in the multi-view data perturbation and feature selection steps, the total number of words in the text dataset is V, and the number of words w is i and w j The number of times it appears in all texts is n ij , then their co-occurrence frequency εc ij for:

[0028]

[0029] As a preference, in the multi-view data perturbation and feature selection steps, the ReliefF algorithm calculates the feature f l In the sample x i The weight on where diff(f l ,x i ,x j ) represents the sample x i and x j In the feature f l The difference measure on i For sample x i The nearest neighbor sample set of the same type, M i For sample x i The nearest heterogeneous sample set is the absolute value of the difference for numerical features and the indicator function of whether the categorical features are the same.

[0030] As a preferred method, in the step of constructing a binary classification learner and voting integration, the parameter update formula of the binary classification learner based on the mean square error loss function is: Where θ is the parameter and α is the learning rate.

[0031] Beneficial Effects

[0032] The present invention provides a multi-label information processing method based on multi-view disturbance. It has the following beneficial effects:

[0033] 1. This multi-label information processing method based on multi-view perturbation collects multi-label data sets from multiple fields, ensuring the richness and diversity of the data and providing comprehensive information for the model. When cleaning the data, appropriate processing methods are used for missing values ​​and outliers to ensure data integrity and reduce interference from extreme values. At the same time, different types of data are normalized to eliminate scale effects, laying a good foundation for subsequent operations.

[0034] 2. This multi-label information processing method based on multi-view perturbation can construct an exclusive feature space for each label by dividing the positive and negative sample sets according to the association between samples and labels, so that the model can focus on learning label-specific information. The k-means clustering algorithm is used to further process the positive and negative sample sets, effectively cluster similar samples, and highlight label-related feature patterns. Converting the distance from the sample to the cluster center into a new feature to construct a feature space helps the model better grasp the label features and improve classification accuracy.

[0035] 3. This multi-label information processing method based on multi-perspective perturbation selects appropriate distance metrics for different types of data to determine sample similarity, providing a basis for subsequent operations. The sample is perturbed by calculating the standard deviation of the characteristic values ​​of the nearest sample, and the nearest neighbors are determined based on the feature correlation and the standard deviation of the relevant statistical information is calculated to perturb the features, thereby increasing the diversity of data and features. The ReliefF algorithm is used for feature selection, and the importance of features is evaluated based on the differences between features and similar and heterogeneous samples. Thresholds are set to filter features, reduce computational complexity, retain key features, and improve classification accuracy and efficiency.

[0036] 4. This multi-label information processing method based on multi-view perturbation selects appropriate classification algorithms and sets parameters according to the characteristics of feature subsets, which can adapt to various types of data and improve classification results. The gradient descent method is used to train the binary classification learner using training data, and the parameters are optimized to achieve accurate prediction. The test samples are predicted by multiple learners and voted to integrate the results, which reduces the impact of the error of a single learner and enhances the prediction stability and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the implementation of the present invention or the technical solution in the prior art, the following is a brief introduction to the drawings required for the implementation or the prior art description. Obviously, the drawings described below are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0038] Figure 1 It is a flow chart of the steps of the present invention;

[0039] Figure 2 This is a flow chart of data collection and preprocessing of the present invention;

[0040] Figure 3 Generate a flow chart for specific features of the label of the present invention;

[0041] Figure 4 This is a flow chart of multi-view disturbance and feature selection of the present invention;

[0042] Figure 5 This is a flow chart of the binary classification learner and voting integration of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Embodiment: A multi-label information processing method based on multi-view disturbance comprises the following steps:

[0045] Step 1: Data collection and preprocessing

[0046] We collect multi-label datasets from a wide range of fields, such as disease diagnosis datasets in the medical field (including multi-label information such as patient symptoms and test results), scene classification datasets in the image field (covering multi-label images such as natural scenes and character scenes), and news classification datasets in the text field (involving news articles with multiple topic labels such as economy and culture), to ensure the richness and diversity of the datasets.

[0047] The collected data is cleaned comprehensively. In the data cleaning operation, missing values ​​and outliers are first processed. For missing values, statistical filling methods are used. For numerical data, the mean is used for filling, and for categorical label data, the mode is used for filling.

[0048] Assume that the data set D = {x1, x2, ..., x n}, where each sample x i Contains m features, if feature x ij If there are missing values, they are replaced by feature x j The mean of , that is:

[0049]

[0050] Among them, x ij Not equal to missing value, here N j Indicates the number of samples with non-missing values ​​in feature j to ensure the integrity of the data.

[0051] For outliers, the Z-Score standardization method is used to detect and correct them. ij The eigenvalue of , its Z-Score is expressed as:

[0052]

[0053] where μj is the mean of feature j, σ j is the standard deviation of feature j. ij When |>τ, it is determined as an outlier, and the threshold τ is usually set to 3. For outliers, the median can be used instead to reduce the impact of extreme values ​​on subsequent analysis.

[0054] In the normalization process, adaptive conversion is performed for different data types. For image data, the original pixel value p is normalized to a specific range [0,1], using the minimum-maximum normalization formula:

[0055]

[0056] Where P min and P max are the minimum and maximum values ​​of all pixel values ​​in the image, respectively. This method can eliminate the influence of different pixel scales in the image, ensure the consistency of color and texture features, and facilitate subsequent label-specific feature generation and clustering operations.

[0057] Step 2: LIFT-based label-specific feature generation

[0058] For each label, the positive and negative sample sets are divided according to the association between the sample and the label. Let the data set D = {x1, x2, ..., x n}, the label set is L = {l1,l2,…,l m}. For label l i , if the sample x j With l i Related (such as in image scene classification, the image contains the label scene), then x j Belongs to the positive sample set Otherwise, it belongs to the negative sample set The purpose of this division is to subsequently construct a space that can highlight the characteristics of each label, so that the model can learn label-specific information. For example, in a medical diagnosis dataset, for the "diabetes" label, samples with abnormal blood sugar values, insulin levels and other related indicators can be divided into positive samples, and normal samples are negative samples.

[0059] For each label's positive and negative sample sets, the k-means clustering algorithm is used for clustering. The k-means algorithm aims to divide the sample set into k clusters so that the sum of the distances from each sample in each cluster to its cluster center is minimized. (n1 is the number of positive samples), the cluster centers are c1, c2, ..., c k , then sample p j To cluster center c t The distance d(p j ,c t ), by minimizing the objective function The effect of this formula is to make the clustering results reflect the distribution of samples in the feature space, clustering similar samples together, thereby highlighting the feature patterns related to the label. For example, in image scene classification, for the positive sample set of the "beach" label, clustering can be used to cluster beach images with similar colors and textures to form a representative feature space.

[0060] After setting a reasonable number of clusters k, calculate the distance from the sample to the cluster center to construct a label-specific feature space. j To its cluster center c t The distance d(x j ,c t ) as the new feature f jt For each label l i , whose characteristic indicates Where n is the total number of samples. This formula converts the distribution information of the original samples in the high-dimensional feature space into new features related to the label, so that each label has its own unique feature space, which helps the subsequent model to better learn the label features. For example, for the "beach" label, in the feature space obtained by clustering, the distance feature can highlight the beach-related feature patterns such as sand color, ocean color, and wave texture in the beach image. These feature patterns help to accurately identify images containing beach scenes in subsequent steps.

[0061] In the sample perspective perturbation and feature perspective perturbation in step 3, the samples and features used are all based on the samples and features in the label-specific feature space constructed in this step. When building a binary classification learner in step 4, the feature representation of each label obtained in this step is used to select a feature subset. These feature subsets are more representative and discriminative features after the label-specific features are generated, which can provide more valuable information for the classifier, thereby improving the accuracy of the classifier for each label classification.

[0062] Step 3: Multi-view data perturbation and feature selection

[0063] First, we select the distance metric. For different types of data, we need to select the appropriate distance calculation method to measure the similarity between samples. In image data, since its pixel features are continuous and numerical, it is appropriate to use the Euclidean distance based on pixel features. Suppose two image samples x i =(x i1 ,x i2 ,...,x im ) and x j =(x j1 ,x j2 ,…,x jm ), where m is the pixel feature dimension, the Euclidean distance d between them is calculated according to the following formula ij :

[0064]

[0065] This formula can accurately reflect the distance between two image samples in the pixel space. The smaller the distance, the more similar the samples are. By calculating the Euclidean distance between samples, the distribution relationship of the samples in the feature space can be determined, providing a basis for the subsequent determination of neighboring samples.

[0066] For text data, due to its sparsity and semantics, it is more appropriate to convert the text into word vector representation and use the cosine distance in the word vector space to measure the similarity between samples. Suppose the word vectors of two text samples are represented as v i =(v i1 ,v i2 ,…,v in ) and v j =(v j1 ,v j2 ,…,v jn ), where n is the word vector dimension, then the cosine distance d between them i ' j The calculation formula is:

[0067]

[0068] The cosine distance can measure the angle between two text samples in the semantic space. The closer the value is to 1, the more similar the samples are. In this way, the relationship between samples can be determined based on the semantic similarity of the text, which is helpful for subsequent perturbation operations.

[0069] After determining the distance metric, the number of neighbors for each sample is reasonably set according to the size of the dataset and the distribution of features. Taking a medium-sized image dataset as an example, the number of neighbors is set to 10-15. The method for determining the neighbors can be to calculate the distance between each sample and other samples, and then select several samples with the closest distance as neighbors. For a sample, calculate its distance to all other samples in the dataset, then sort the distances from small to large, and select the first 10-15 samples as the neighbor set. The determination of neighbor samples can reflect the distribution of samples in the local space, and the subsequent calculation of the standard deviation of the sample eigenvalue is based on the eigenvalue distribution of the neighbor samples.

[0070] For each sample's neighbor set, calculate the standard deviation of its eigenvalues. For image samples, their eigenvalues ​​include color, texture, and other eigenvalues. Let the neighbor set N(x i ) where the lth eigenvalue of the kth sample is x kl (k=1,2,…,|N(x i )|, l = 1, 2, ..., m), then the mean of the feature is:

[0071]

[0072] The standard deviation is:

[0073]

[0074] The standard deviation reflects the dispersion of neighbor samples on this feature.

[0075] The sample is disturbed according to the calculated standard deviation. One way is to add random noise based on the standard deviation to the sample feature value. For the color feature value x of the image sample il , i represents the sample number, l represents the color feature dimension, and the original value is added with a mean of 0 and a standard deviation of σ l Normally distributed random noise il ( il ~N(0,σ l )), the sample characteristic value after disturbance is

[0076] x′ il =x il +∈ il

[0077] In this way, multiple perturbed sample sets can be generated, each with a different eigenvalue distribution, thereby increasing the diversity of the data.

[0078] Another way to perturb is to perform eigenvalue scaling, multiplying the eigenvalue of the sample by a scaling factor based on the standard deviation:

[0079] α l =1+λσ l

[0080] Among them, λ is the scaling factor, and the eigenvalue after perturbation is obtained

[0081] x′ il =α l x il

[0082] Calculate the correlation or similarity between features and use corresponding methods for different types of data.

[0083] For image features, the cosine similarity between feature vectors is used to measure the correlation. Suppose the two feature vectors of the image are f i =(f i1 ,f i2 ,…,f im ) and f j =(f j1 ,f j2 ,…,f jm ), where m is the feature dimension, the cosine similarity calculation formula between them is:

[0084]

[0085] The purpose of this formula is to accurately measure the consistency of the direction of two feature vectors. The closer the value is to 1, the more related the two features are. By calculating the cosine similarity, the similarity relationship between image features can be determined, providing a basis for the subsequent determination of feature neighbors.

[0086] For text vocabulary features, the correlation is measured by counting the co-occurrence frequency of words. Suppose the total number of words in the text dataset is V, and the vocabulary w i and w j The number of times it appears in all texts is n ij , then their co-occurrence frequency εc ij for:

[0087]

[0088] The higher the co-occurrence frequency, the more semantically related the two words are. This method can reflect the correlation between text features at the lexical level, which is helpful for subsequent feature perturbation operations.

[0089] Determine the nearest neighbors of each feature based on the correlation or similarity between features. For a text dataset with 50-100 features, set the number of nearest neighbors for each feature to 3-5. The method for determining the nearest neighbors is to sort according to the calculated correlation or similarity measure values, and select the most relevant or similar features as the nearest neighbors. For a certain lexical feature in the text data, calculate its co-occurrence frequency with all other lexical features, then sort the co-occurrence frequencies from large to small, and select the top 3-5 lexical features as the nearest neighbor set. The determination of feature neighbors can reflect the local relationship of features in semantic or feature space, which is of great significance for the subsequent calculation of the standard deviation of neighbor feature statistical information.

[0090] For each feature’s neighbor set, calculate its mean and standard deviation. For the vocabulary feature weights in the text data, let the neighbor set N(w i ) has a weight of w for the kth feature k (k=1,2,…,|N(w i )|), then the mean value of the feature weight is:

[0091]

[0092] The standard deviation is:

[0093]

[0094] The standard deviation reflects the dispersion of the weights of neighbor features.

[0095] The features are perturbed according to the standard deviation. For the weights of vocabulary features in text data, the adjustment range is determined based on the standard deviation. Let the adjusted vocabulary feature weight be w i ′, the adjustment formula is w i ′=w i +λσ w Where λ is the adjustment coefficient. This can generate a perturbed feature set and increase the diversity of features. For the eigenvalues ​​of the image data, a scaling operation is performed, and the scaling factor is determined based on the standard deviation. Suppose the image eigenvalue is f il , where i represents the sample number, l represents the feature dimension, and the calculation formula of the scaling factor is:

[0096] α l =1+λσ l

[0097] Among them, σ l is the standard deviation of the corresponding feature, λ is the scaling factor, then the eigenvalue after disturbance is f′ il =α l f il .

[0098] Finally, feature selection is performed. For each label, the multiple feature spaces after perturbation are selected using the ReliefF algorithm. The ReliefF algorithm evaluates the importance of features by calculating the difference between features and samples of the same type and samples of different types. Suppose the sample set S = {x1, x2, …, x n}, where each sample x i Contains m features, for a certain feature f l (l=1,2,...,m), calculate its i The difference between the sample x and the samples of the same type and the samples of different types. First, find the sample x i The nearest neighbor sample set H i and the nearest heterogeneous sample set M i , the neighbor determination method is the same as the neighbor determination method in the previous sample view and feature view perturbation. Then, calculate the feature f l In the sample x i The weight w on l , the calculation formula is:

[0099]

[0100] where diff(f l ,x i ,x j ) represents the sample x i and x j In the feature f l The difference measure on , for numerical features, can be the absolute value of the difference, for categorical features, can be the indicator function of whether they are the same. The purpose of this formula is to highlight the features that can effectively distinguish different categories by comparing the differences between the samples and the same and different samples in terms of features, so that the feature weight can accurately reflect the importance of the feature to the classification.

[0101] The weight w of each feature is calculated according to the ReliefF algorithm l After that, we need to set a reasonable threshold θ to filter features. The threshold setting step is to first calculate the mean of all feature weights and standard deviation Then set the threshold to θ = μ w +kσ w , where k is the adjustment coefficient. Features with weights above the threshold θ are selected as relevant and necessary features, which are considered to have important contributions to label classification. Features with weights below the threshold are considered redundant or irrelevant features and are removed, resulting in the final feature subset for each label. The effect of this is to reduce the dimensionality of the feature space and the computational complexity of the model, while retaining the most valuable features for classification, improving the classification accuracy and efficiency of the model.

[0102] Step 4: Build a binary classification learner and voting ensemble

[0103] For each label, multiple binary classification learners are constructed using the selected feature subsets. When multiple binary classification learners are constructed based on the selected feature subsets, different classification algorithms have their own characteristics and applicable scenarios. Support vector machine (SVM) is suitable for linearly separable or nonlinearly separable data, and its kernel function parameters determine the mapping method of data in high-dimensional space. Let the kernel function be K(x i ,x j ), by adjusting the kernel function parameters, the shape of the classification boundary of the data can be changed to improve the classification accuracy. Decision trees perform classification by constructing a tree structure, and its depth parameter affects the complexity and generalization ability of the tree. Shallow decision trees may lead to underfitting, while too deep decision trees may overfit the data. Neural networks learn data features by constructing multiple layers of neurons, and the number of layers and nodes determines the expressive power of the model. For example, a simple neural network can have an input layer, a hidden layer, and an output layer. Suppose the hidden layer has h nodes, the input layer has m nodes (corresponding to the feature subset dimension), and the output layer has 1 node for binary classification. You can set appropriate values ​​according to the complexity of the data and the characteristics of the feature subset.

[0104] These parameters are set according to the statistical characteristics of the feature subset, including mean and variance, and data distribution, where data distribution includes whether it is normally distributed and data sparsity. When the feature subset dimension is high and the data distribution is complex, choose a neural network and increase the number of layers and nodes appropriately; when the data presents linearly separable features, use SVM with a linear kernel function and adjust its regularization parameters.

[0105] Each binary classification learner is trained using training data to optimize the algorithm and adjust the learner parameters. In this embodiment, the gradient descent method is used for a binary classification learner based on a mean square error (MSE) loss function with the following formula:

[0106]

[0107] where y i is the true label, To predict the label, the parameter update formula is:

[0108]

[0109] Among them, θ tis the current parameter, and α is the learning rate. The purpose of this formula is to gradually reduce the loss function value by continuously iteratively updating the parameters, so that the learner can more accurately predict the sample label. During the training process, the gradient of the loss function to the parameter is calculated based on the feature value and the true label of the sample, and then the parameter is updated according to the above formula until the stop condition is met.

[0110] For the test sample, it is input into the binary classification learner corresponding to each label to obtain the prediction result. Each learner judges whether the sample belongs to the label based on its own learned model and outputs the prediction result of 0 (negative class) or 1 (positive class).

[0111] For the prediction results of each label, the principle of minority obeys majority is used for voting integration. For a label of a certain sample, k binary classification learners participate in the voting, among which n1 learners predict the positive class, n0 learners predict the negative class, and n1+n0=k. If n1>n0, it is finally determined that the sample belongs to the label, otherwise it does not belong. The voting integration method can integrate the prediction results of multiple learners, reduce the error impact of a single learner, and improve the stability and accuracy of the prediction.

[0112] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-label information processing method based on multi-view perturbation, characterized in that: The following steps are involved: Step 1: Data collection and preprocessing: Collect multi-label datasets from multiple fields, including disease diagnosis datasets in the medical field, scene classification datasets in the image field, and news classification datasets in the text field; The collected data was cleaned, and the missing values ​​were filled with statistical methods, the numerical data were filled with the mean, and the categorical label data were filled with the mode; the outliers were detected and corrected using the Z-Score standardization method, and the median was used to replace the outliers; Normalize different types of data. Use the minimum-maximum normalization formula to normalize the original pixel values ​​of image data to [0,1]. Step 2: LIFT-based label-specific feature generation: The positive and negative sample sets are divided according to the association between samples and labels. Samples related to labels belong to the positive sample set, otherwise they belong to the negative sample set. The k-means clustering algorithm is used to cluster the positive and negative sample sets of each label, and the distance from the sample to the cluster center is calculated. The cluster center is determined by minimizing the objective function. After setting a reasonable number of clusters k, the distance from the sample to the cluster center is used as a new feature to construct a label-specific feature space. Step 3: Multi-view data perturbation and feature selection: Distance metric selection: For image data, the Euclidean distance based on pixel features is used, and for text data, the cosine distance in the word vector space is used; the number of neighbors of each sample is set according to the size of the data set and the feature distribution, and the standard deviation of the eigenvalues ​​of each sample's neighbor set is calculated. The sample is perturbed according to the standard deviation, which can be achieved by adding random noise or eigenvalue scaling; Feature perspective perturbation: Calculate the correlation or similarity between features. Image features use the cosine similarity between feature vectors. Text vocabulary features use the co-occurrence frequency of words. Determine the neighbors of each feature based on the correlation or similarity, calculate the standard deviation of the neighbor feature statistics, and perturb the features based on the standard deviation. For text data, adjust the vocabulary feature weights, and for image data, scale the feature values. Feature selection: Use the ReliefF algorithm to select features, calculate the difference between features and similar samples and heterogeneous samples to evaluate feature importance, set a threshold to filter features, and set the threshold based on the mean and standard deviation of all feature weights. Select features with weights higher than the threshold as the final feature subset; Step 4: Build a binary classification learner and voting ensemble: Use the selected feature subsets to build multiple binary classification learners, select appropriate classification algorithms and set parameters based on the characteristics of the feature subsets and data distribution, such as support vector machines, decision trees, and neural networks, and use the training data to train the binary classification learners using the gradient descent method; For the test samples, multiple learners are used to predict and vote on the majority principle to combine the prediction results of multiple learners.

2. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the data collection and preprocessing step, let the data set be D = {x1, x2, ..., x n }, where each sample x i Contains m features, if feature x ij If there are missing values, they are replaced by feature x j The mean Among them, x ij Not equal to missing value, here N j Represents the number of samples with non-missing values ​​in feature j to ensure the integrity of the data.

3. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the data collection and preprocessing steps, for the sample x ij The eigenvalue of , its Z-Score is expressed as where μj is the mean of feature j, σ j is the standard deviation of feature j, when |z ij When |>3, it is determined as an outlier. When it is determined as an outlier, the median is used to replace the outlier.

4. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the LIFT-based label-specific feature generation step, let the positive sample set be The cluster centers are c1, c2, …, c k , then the distance d(p j ,c t ), by minimizing the objective function To determine the cluster centers.

5. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the multi-view data perturbation and feature selection steps, for two samples x of image data i =(x i1 ,x i2 ,…,x im ) and x j =(x j1 ,x j2 ,…,x jm ), whose Euclidean distance Where m is the pixel feature dimension.

6. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the multi-view data perturbation and feature selection steps, the word vectors of two samples of text data are represented as v i =(v i1 ,v i2 ,…,v in ) and v j =(v j1 ,v j2 ,…,v jn ), the cosine distance Where n is the word vector dimension.

7. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the multi-view data perturbation and feature selection steps, for the image feature vector f i =(f i1 ,f i2 ,…,f im ) and f j =(f j1 ,f j2 ,…,f jm ), the cosine similarity Where m is the feature dimension.

8. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the multi-view data perturbation and feature selection steps, the total number of words in the text dataset is V, and the vocabulary w i and w j The number of times it appears in all texts is n ij , then their co-occurrence frequency εc ij for:

9. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the multi-view data perturbation and feature selection steps, the ReliefF algorithm calculates the feature f l In the sample x i The weight on where diff(f l ,x i ,x j ) represents the sample x i and x j In the feature f l The difference measure on i For sample x i The nearest neighbor sample set, M i For sample x i The nearest heterogeneous sample set is the absolute value of the difference for numerical features and the indicator function of whether the categorical features are the same.

10. The multi-label information processing method based on multi-view disturbance according to claim 1, characterized in that: In the steps of building a binary classification learner and voting integration, the parameter update formula of the binary classification learner based on the mean square error loss function is: Where θ is the parameter and α is the learning rate.

Images