Partial multi-mark feature selection method and device based on feature and mark collaboration
Through the method of feature and marking collaboration, a regular term and sparse coding model for feature marking collaborative cooperation are constructed, which solves the problems of noise marking and feature redundancy in partial multi-marking learning, realizes feature selection and dimensionality reduction, and improves the multi-marking classification model performance.
Patent Information
- Application Number
- CN202411433024.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing partially multi-label learning method ignores the influence of features when dealing with noise marking, resulting in increased risk of overfitting and computational costs, and fails to effectively select and reduce dimensionality effectively.
Using a method based on feature and marker collaboration, the relationship between feature space and marker distribution is constructed through a linear model, the marker distribution matrix is optimized using the norm loss function, and the feature marker collaborative cooperation regular terms are constructed based on feature similarity and semantic similarity, and sparse encoding is used. and normalize the feature selection to obtain the optimal feature subset.
Effectively reduce the impact of noise marking on model performance, realize feature selection and dimensionality reduction, improve the performance of multi-label classification models, and adapt to the multi-label environment of high-dimensional data.
Smart Images

Figure CN118968194A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for selecting partial multi-marker features based on feature and mark collaboration. Background Art
[0002] The premise of multi-label learning is that each data instance contains all possible labels associated with it, and these labels should be accurate. However, as the number of candidate labels for multi-label data increases, the task of labeling data becomes more difficult. The annotator may roughly assign a set of candidate labels to each sample, resulting in the sample's label set containing irrelevant noise labels. For example, Figure 1 Labeled by labelers as "mountain", "lake", "tree", "flower", "cloud", "bird", "house" and "road", but "bird", "house" and "road" are noise labels. These noise labels seriously affect the performance of multi-label classification models. In order to reduce the impact of noise labels on model performance, researchers have proposed a method for model learning from noise labels. Among them, the research on multi-label learning problems mainly focuses on learning with partial multi-labels.
[0003] However, existing partial multi-label learning methods mainly focus on how to identify noise labels and remove them to reduce their impact on model performance, while ignoring the impact of the features themselves on the model. In order to describe complex and diverse semantic information, the features of multi-label data are usually high-dimensional. In the feature space, there are many cases where there is a lack of correlation and redundancy between features, and only a few discriminative features can provide useful information when distinguishing different categories or predicting targets. Existing partial multi-label learning methods do not filter or eliminate the original features during the modeling process, but use all the information in the data space for modeling and prediction. Doing so not only has the risk of overfitting, but also increases the computational cost. How to perform feature selection and dimensionality reduction on multi-label data in the presence of noise labels has become one of the important challenges facing partial multi-label learning.
[0004] In view of this, the applicant filed this application after studying the existing technology. Summary of the invention
[0005] The present invention aims to provide a method and device for selecting multi-marker features based on the collaboration of features and markers to solve at least one of the above problems.
[0006] In order to solve the above technical problems, the present invention is implemented through the following technical solutions: A partial multi-marker feature selection method based on feature and marker collaboration, comprising: S1, obtaining a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and its corresponding candidate label set; S2, converting the candidate marker set into a logical marker matrix; S3, using a linear model to construct the relationship between the feature space of the over-labeled sample data and the label distribution, and obtaining a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid overfitting of the model; S4, using a preset metric to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical label matrix; then constructing a feature label collaborative regularization term based on the feature similarity and the semantic similarity to guide the learning of label distribution; S5, according to The norm loss function and the feature labeling cooperate with the regularization term to construct the objective function of the feature selection model, and sparsely constrain the weights of the model based on sparse coding and norm regularization, and optimize the feature selection model using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; according to the optimal weight matrix and the linear model, update the label distribution matrix of the sample data; S6, performing feature selection according to the importance of features based on the optimal weight matrix to obtain an optimal feature subset, and then obtaining a new sample feature matrix, that is, a result of the over-labeled feature selection.
[0007] Preferably, S2 is specifically: The multi-labeled dataset is defined as: ; in, For the samples, Indicates that the dimension in the range of real numbers is The matrix of express of dimensional features, T is the transposed matrix, is the characteristic dimension of the sample; express The candidate tag set, To have The output space of candidate class labels, is the total sample size, Represents the number of category labels, i is the index variable; Constructing samples using hard thresholding method Logical label vector of , the formula is: ; in, is a logical label vector The j-th element of .
[0008] Preferably, the relationship between the feature space and the label distribution of the overly labeled sample data constructed using a linear model is specifically: Assume that each column in the over-labeled sample data is a sample, then , represents the feature matrix composed of all samples; , represents the logical label matrix corresponding to all samples; The marker distribution matrix D obtained by linear model construction is: ; ; in, Representation sample The label distribution matrix is the weight matrix of the linear model; For the samples; represents the bias vector, Represents the matrix after the weight matrix is increased by one dimension, that is, an additional one-dimensional weight matrix ; Represents the sample matrix after adding one dimension, that is .
[0009] Preferably, by The formula for the norm loss function to optimize the error between the label distribution matrix and the logical label matrix is: ; in, , represents the feature matrix of all samples after adding one dimension; for Norm.
[0010] Preferably, the preset metric includes a Gaussian heat kernel function and a cosine function; The Gaussian heat kernel function is used to construct a feature similarity matrix to measure the feature similarity between samples. The calculation formula is: ; ; ; in, is the feature similarity matrix; express and The feature similarity of represents the Gaussian kernel function; represents the kernel size; and Respectively represent and Sample data; for norm; The cosine function is used to construct a semantic similarity matrix to measure the semantic relevance of each group of samples. The calculation formula is: ; ; in, is the semantic similarity matrix, and Respectively represent samples and The corresponding logical label vector; Representation sample and The semantic similarity of .
[0011] Preferably, the calculation process of the feature marker collaborative regularization term is: According to the feature similarity matrix and the semantic similarity matrix , construct the feature marker collaborative cooperation matrix, and the calculation formula is: ; ; in, is the feature labeling collaboration matrix, Represents Hadamard; Then, a feature label collaboration regularization term is constructed based on the feature label collaboration matrix to guide label distribution learning. The formula is: ; in, represents the feature marker collaborative regularization term, For the matrix The Laplace matrix of is a diagonal matrix whose diagonal elements are , , Respectively represent samples , The distribution of markers, Represents the trace of the matrix.
[0012] Preferably, S5 is specifically: Based on sparse coding technology and Norm regularization imposes sparse constraints on the one-dimensional weight matrix, and the formula is: ; in, express norm; The matrix after adding a one-dimensional bias vector to the weight matrix of the sample label distribution; According to the The norm loss function cooperates with the feature labeling regularization term to construct the objective function of the feature selection model. The objective function expression is: ; in, Marking collaborative regularization term weight parameters for the feature; For sparse coding and The norm regularizes the weight parameter of the sparsity constraint; Update and optimize the objective function to obtain the optimal weight matrix ; According to the linear model formula, the updated marker distribution matrix is obtained , the expression is: .
[0013] Preferably, the S6 is specifically: according to The optimal weight matrix is calculated by The size of the modulus of each row is used to obtain the importance of the feature. The calculation formula is: ; in, express No. OK, for norm, Indicates Importance of dimensional features; Then the importance of the feature Sort in descending order and select the top The importance of each feature and its corresponding index, and get the selected feature index set ,in, Indicates the number of selected features; Get feature subsets based on the selected feature index set , and then get the new sample feature matrix : ; ; in, Represents the new sample feature matrix The i-th sample instance in express The kth feature of .
[0014] Preferably, the new sample feature matrix and the updated label distribution matrix are also used to train a multi-label classifier, and the input multi-label samples are classified by the trained multi-label classifier to obtain a multi-label classification result; wherein the multi-label classifier adopts a multi-output regression model, and the multi-output regression model includes a multi-output support vector regression model, and the training process is: A multi-output support vector regression model is used as a multi-label classifier, according to the label distribution matrix And the new sample feature matrix The multi-label classifier is trained, and the objective function formula of the multi-label classifier is: ; in, , They represent the parameter matrices of the model, , represents the regularization parameter; for -Insensitive loss function; is the multi-label loss function of the sample; -Insensitive loss function The formula is: ; ; ; in, represents the error between the model output and the labeled distribution, Indicates that the sample Nonlinear kernel functions mapped to high-dimensional space; Indicates the set insensitive zone; Partially multi-label loss function The formula is: ; in, express exist The complement of Indicates a dimensional vector, if ,but No. elements are equal to 1, otherwise they are equal to 0; Indicates another dimensional vector, if ,but No. elements are equal to 1, otherwise they are equal to 0; The objective function is optimized by iterative weighted least squares method to obtain the optimal model parameters. and And the trained classification model, the classification model expression is: .
[0015] Define the input multi-labeled test sample as , then, the new sample feature matrix output by the multi-label feature selection model is ; The label distribution of the sample is calculated according to the trained classification model. The calculation formula is: ; Use hard threshold method to judge samples Whether it belongs to the category mark, the judgment formula is: ,like
[0016] in, express No. elements, and m is the preset threshold.
[0017] The present invention also provides a partial multi-marker feature selection device based on feature and mark collaboration, comprising: A data acquisition unit, used to acquire a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and a corresponding candidate label set; A logical tag generation unit, used for converting the candidate tag set into a logical tag matrix; The label distribution relationship unit is used to construct the relationship between the feature space of the partial multi-label sample data and the label distribution by using a linear model, and obtain a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid overfitting of the model; A feature labeling collaboration unit is used to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical labeling matrix using a preset metric; and then construct a feature labeling collaboration regularization term based on the feature similarity and the semantic similarity to guide the learning of labeling distribution; The model optimization unit is used to optimize the The norm loss function and the feature labeling cooperate with the regularization term to construct the objective function of the feature selection model, and sparsely constrain the weights of the model based on sparse coding and norm regularization, and optimize the feature selection model using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; according to the optimal weight matrix and the linear model, update the label distribution matrix of the sample data; The feature selection unit is used to perform feature selection according to the optimal weight matrix and the importance of the features to obtain the optimal feature subset, and then obtain a new sample feature matrix, that is, the result of the partial multi-label feature selection.
[0018] The present invention also provides a device for selecting multiple marker features based on the collaboration of features and markers, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be executed by the processor to implement a method for selecting multiple marker features based on the collaboration of features and markers as described above.
[0019] The present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, a method for selecting multi-marker features based on the collaboration of features and marks as described above is implemented.
[0020] In summary, compared with the prior art, the present invention has the following beneficial effects: The present invention first transforms the multi-label disambiguation problem into a label distribution learning task and establishes a The regression model of the norm loss function measures the error between the label distribution and the observed label, and uses the topological structure of the feature space of the sample data and the semantic information of the label space to construct a feature label collaborative regularization term to guide the learning of the label distribution, thereby effectively restoring the label distribution of the sample; then The norm constrains the model weights to achieve feature selection, and constructs a model update strategy to obtain the model weights and the label distribution of the training samples; finally, based on the model weights, the feature subset most relevant to the label distribution is selected to obtain the multi-label feature selection result. In addition, the feature subset and the updated label distribution can be used to train a multi-output regression model to obtain a classification model, so that the classification model can adapt to the prediction of partial multi-label data.
[0021] The present invention fully considers the feature information and label distribution information in partial multi-label learning, and proposes a new learning framework for the field of partial multi-label learning; and introduces a new label disambiguation mechanism based on label distribution learning, which aims to eliminate noisy candidate labels in the candidate label set. The present invention integrates the feature similarity and semantic similarity of training instances to guide label distribution learning, applies label enhancement and manifold learning to the algorithm of the present invention, and provides a high-performance classification method for partial multi-label environments for high-dimensional data. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 Background knowledge graph of the multi-label feature selection classification method provided as the background technology of the present invention.
[0024] Figure 2 A flowchart of a method for selecting multiple-marker features based on feature and mark collaboration provided in Example 1 of the present invention.
[0025] Figure 3 A schematic diagram of a framework of a method for selecting partial multi-marker features based on the collaboration of features and marks provided in Example 1 of the present invention.
[0026] Figure 4 A schematic diagram of a multi-marker feature selection device based on feature and mark collaboration provided in Example 2 of the present invention.
[0027] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0029] Embodiment 1 Embodiment 1 of the present invention provides a method for selecting multi-label features based on the collaboration of features and labels, which can be implemented by a multi-label feature selection device based on the collaboration of features and labels (hereinafter referred to as feature selection classification device), and in particular, executed by one or more processors in the feature selection classification device.
[0030] In this embodiment, the feature selection and classification device may be an electronic device equipped with a processor, the processor having a computer program of the multi-marker feature selection method based on the collaboration of features and marks, and the computer program can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., which is not limited here.
[0031] With the development of Internet technology, the amount of data has increased dramatically, and the amount of data that needs to be labeled is also growing rapidly. Labeling multi-label data is very time-consuming. The further increase in the amount of data means that the labeler can only roughly label the data, resulting in the candidate label set of the data containing a small number of irrelevant labels in addition to relevant labels, namely noise labels. Using this type of imprecisely labeled data to build an effective learning model is defined as partial multi-label learning.
[0032] The method of the present invention is mainly applied to the field of partial multi-label learning and is a new multi-label learning framework. In this learning framework, each example is assigned multiple candidate labels, and only some of these labels are correct. How to select the correct label from the numerous labels containing noise labels and combine the sample characteristics to improve the accuracy of label annotation is the focus of the present invention.
[0033] like Figures 2 to 3 As shown, a method for selecting partial multiple-marker features based on the collaboration of features and markers includes steps S1 to S6.
[0034] S1, obtaining a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and its corresponding candidate label set.
[0035] Specifically, the multi-labeled data set obtained is defined as: ; In the partially multi-labeled dataset to be processed, each sample contains sample features and candidate labels, and the candidate labels are divided into correct labels and irrelevant labels (i.e., noise labels). The partially multi-labeled dataset to be processed can be some labeled sample images, such as Figure 1 As shown in the figure, the image is labeled as "mountain", "lake", "tree", "flower", "cloud", "bird", "house" and "road" by the labeling worker, which are the candidate label sets of the image sample, but "bird", "house" and "road" in the candidate label sets are noise labels. Through the method of the present invention, combined with the sample features, a more accurate label is selected from the candidate label set.
[0036] in, For the samples, Indicates that the dimension in the range of real numbers is The matrix of express of dimensional features, T is the transposed matrix, is the characteristic dimension of the sample; Representation sample The candidate tag set, To have The output space of candidate class labels, is the total sample size, Represents the number of category labels, and i is the index variable.
[0037] S2, converting the candidate marker set into a logical marker matrix.
[0038] Specifically, the hard threshold method is used to construct the sample Logical label vector of , the formula is: ; in, is a logical label vector The j-th element of .
[0039] S3, using a linear model to construct the relationship between the feature space of the over-labeled sample data and the label distribution, and obtaining a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid model overfitting.
[0040] In this embodiment, a mapping relationship between the feature space of the partial multi-labeled data and the label distribution space is constructed by a general linear model. The relationship between the feature space of the partial multi-labeled sample data and the label distribution constructed by the linear model is specifically: Assume that each column in the over-labeled sample data is a sample, then , represents the feature matrix composed of all samples, Represents the dimension in the range of real numbers as Matrix of , represents the logical label matrix corresponding to all samples; Specifically, the label distribution matrix D is obtained by constructing the linear model: ; ; in, Representation sample The label distribution matrix is the weight matrix of the linear model; For the samples; represents the bias vector, Represents the matrix after the weight matrix is increased by one dimension, that is, an additional one-dimensional weight matrix ; Represents the sample matrix after one dimension is added, that is .
[0041] Specifically, in this embodiment, considering the sparsity of noise labels, The norm constructs a loss function to measure the error between the label distribution matrix and the logical label matrix. The formula of the norm loss function is: ; in, , represents the feature matrix of all samples after adding one dimension; for Norm.
[0042] S4, using a preset metric to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical label matrix; then constructing a feature label collaborative regularization term based on the feature similarity and the semantic similarity to guide the learning of label distribution; In this embodiment, the preset metric includes a Gaussian heat kernel function. In order to measure the feature similarity between samples, a feature similarity matrix is constructed using the Gaussian heat kernel function to measure the similarity between each group of samples. The calculation formula is: ; ; Based on experience Set to: ; in, is the feature similarity matrix; express and The feature similarity of represents the Gaussian kernel function; represents the kernel size; and Respectively represent and Sample data; for Norm.
[0043] In this embodiment, the Gaussian Heat Kernel function is a function widely used in many fields. It originates from the concept of Gaussian distribution (normal distribution) and is used to describe the diffusion process of various physical quantities such as heat, probability density, and signal.
[0044] In this embodiment, the semantic similarity matrix is calculated by considering the overlap of the candidate tag sets of the samples. If the overlap of the candidate tag sets of two samples is high (low), the semantic similarity of the two samples is high (low). Therefore, the preset metric includes a cosine function, and the semantic similarity matrix is constructed using the cosine function to measure the semantic relevance of each group of samples. The calculation formula is: ; ; in, is the semantic similarity matrix, and Respectively represent samples and The corresponding logical label vector; Representation sample and The semantic similarity of .
[0045] In this embodiment, the label distribution matrix is calculated by combining the feature similarity and semantic similarity of each sample through the smoothing hypothesis. Based on the assumption that similar samples have similar label distributions, and samples with high semantic similarity should also have similar label distributions, we try to use The characteristic information and The tag information of each instance is obtained The distribution of markers.
[0046] Specifically, according to the feature similarity matrix S and the semantic similarity matrix C, a feature tag collaborative cooperation matrix is constructed, and the calculation formula is: ; ; in, is the feature labeling collaboration matrix, Represents Hadamard; Then, a feature label collaboration regularization term is constructed based on the feature label collaboration matrix to guide label distribution learning. The formula is: ; in, represents the feature marker collaborative regularization term, For the matrix The Laplace matrix of is a diagonal matrix whose diagonal elements are , , Respectively represent samples , The distribution of markers, Represents the trace of the matrix.
[0047] In this embodiment, the trace of a matrix is a mathematical concept, which is the sum of all elements on the main diagonal of a square matrix (ie, a matrix with equal numbers of rows and columns).
[0048] S5, according to The norm loss function cooperates with the feature labeling regularization term to construct the objective function of the feature selection model, and sparse constraints are imposed on the weights of the model based on sparse coding and norm regularization. The feature selection model is optimized using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; based on the optimal weight matrix and the linear model, the label distribution matrix of the sample data is updated.
[0049] Specifically, in order to make the relevant features of the over-labeled data obtain greater weights and reduce the weights of redundant or less important features to zero, the sparse coding technology and Norm regularization imposes sparse constraints on the one-dimensional weight matrix, and the formula is: ; in, express norm; The matrix after adding a one-dimensional bias vector to the weight matrix of the sample label distribution; Integrate the strategies of S3-S4 to construct a feature selection model. The norm loss function cooperates with the feature labeling regularization term to construct the objective function of the feature selection model, and the expression of the objective function is: ; in, Marking collaborative regularization term weight parameters for the feature; For sparse coding and The norm regularizes the weight parameter of the sparsity constraint; It represents the minimization of the objective function.
[0050] In this embodiment, the weight matrix of the objective function is iteratively updated based on the alternating direction multiplier method until the objective function converges or reaches the maximum number of iterations to obtain the optimal solution, that is, the optimal weight matrix Finally, the tag distribution is calculated according to the weight matrix and the feature subset most relevant to the tag distribution is selected to complete tag disambiguation and feature selection.
[0051] According to the linear model formula, the updated marker distribution matrix is obtained , the expression is: .
[0052] S6, performing feature selection according to the importance of features based on the optimal weight matrix to obtain an optimal feature subset, and then obtaining a new sample feature matrix, that is, a result of the over-labeled feature selection.
[0053] Specifically, according to Norm calculation of the optimal weight matrix The size of the modulus of each row is used to obtain the importance of the feature. The calculation formula is: ; in, express No. OK, for norm, Indicates Importance of dimensional features; Then the importance of the feature Sort in descending order and select the top The importance of each feature and its corresponding index, and get the selected feature index set ,in, Indicates the number of selected features; Get feature subsets based on the selected feature index set , and then get the new sample feature matrix : ; ; in, Represents the new sample feature matrix The i-th sample instance in express The kth feature of .
[0054] In another preferred embodiment, it also includes: S7, the new sample feature matrix and the updated label distribution matrix are also used to train a multi-label classifier, and the input multi-label sample is classified by the trained multi-label classifier to obtain a multi-label classification result; wherein the multi-label classifier adopts a multi-output regression model, and the multi-output regression model includes a multi-output support vector regression model, and the training process is as follows: A multi-output regression model is used as a multi-label classifier, and the multi-label classifier is trained according to the feature subset and the updated label distribution matrix, so as to obtain a trained classification model.
[0055] In this embodiment, the multi-output regression model is a multi-output support vector regression model. The multi-output support vector regression model (Multi-output Support Vector Regression, M-SVR) is a regression method based on support vector machine (SVM) for processing regression problems with multiple output variables. The basic idea of M-SVR is to regard multiple output variables as a whole and simultaneously predict these output variables by building a unified regression model. Specifically: A multi-output support vector regression model is used as a multi-label classifier, according to the label distribution matrix And the new sample feature matrix The multi-label classifier is trained, and the objective function formula of the multi-label classifier is: ; in, , They represent the parameter matrices of the model, , represents the regularization parameter; for -Insensitive loss function; is the multi-label loss function of the sample; -Insensitive loss function The formula is: ; ; ; in, represents the error between the model output and the labeled distribution, Indicates that the sample Nonlinear kernel functions mapped to high-dimensional space; Indicates the set insensitive zone; Partially multi-label loss function The formula is: ; in, express exist The complement of Indicates a dimensional vector, if ,but No. elements are equal to 1, otherwise they are equal to 0; Indicates another dimensional vector, if ,but No. elements are equal to 1, otherwise they are equal to 0; The objective function is optimized by iterative weighted least squares method to obtain the optimal model parameters. and And the trained classification model, the classification model expression is: .
[0056] According to the trained classification model, the multi-label samples output by the multi-label feature selection model are predicted to obtain the multi-label feature classification results.
[0057] Define the input multi-labeled test sample as After feature selection, a new feature expression is obtained, that is, the new sample feature matrix .
[0058] Calculated according to the classification model function The distribution of markers ,based on Estimate the label of the sample to be tested, and set a threshold based on experience (in this embodiment, the threshold is set to 0.5 based on experience). When the set threshold is exceeded, the classification decision requirements are met, and the decision formula is: ,like ; in, express No. elements.
[0059] In summary, compared with the prior art, the present invention has the following beneficial effects: The present invention fully considers the feature information and label distribution information in partial multi-label learning, and proposes a new learning framework for the field of partial multi-label learning. In addition, the method of the present invention introduces a new label disambiguation mechanism based on label distribution learning, which aims to eliminate noisy candidate labels in the candidate label set, further integrate the feature similarity and semantic similarity of the training examples to guide the label distribution learning, and then adopt The norm constrains the model weights to achieve feature selection. Finally, by constructing a model update strategy, the model can adapt to the multi-label environment of high-dimensional data and correctly predict all possible labels to which the unknown labeled samples belong.
[0060] The present invention applies label enhancement and manifold learning to the algorithm of the present invention, providing a high-performance classification method for a multi-label environment facing high-dimensional data.
[0061] Embodiment 2 like Figure 4 As shown, the second embodiment of the present invention further provides a multi-marker feature selection device based on feature and mark collaboration, comprising: A data acquisition unit, used to acquire a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and a corresponding candidate label set; A logical tag generation unit, used for converting the candidate tag set into a logical tag matrix; The label distribution relationship unit is used to construct the relationship between the feature space of the partial multi-label sample data and the label distribution by using a linear model, and obtain a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid overfitting of the model; A feature labeling collaboration unit is used to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical labeling matrix using a preset metric; and then construct a feature labeling collaboration regularization term based on the feature similarity and the semantic similarity to guide the learning of labeling distribution; The model optimization unit is used to optimize the The norm loss function and the feature labeling cooperate with the regularization term to construct the objective function of the feature selection model, and sparsely constrain the weights of the model based on sparse coding and norm regularization, and optimize the feature selection model using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; according to the optimal weight matrix and the linear model, update the label distribution matrix of the sample data; The feature selection unit is used to perform feature selection according to the optimal weight matrix and the importance of the features to obtain the optimal feature subset, and then obtain a new sample feature matrix, that is, the result of the partial multi-label feature selection.
[0062] Embodiment 3 The third embodiment of the present invention also provides a multi-marker feature selection device based on the collaboration of features and markers, which includes a memory and a processor, wherein a computer program is stored in the memory, and the computer program can be executed by the processor to implement the multi-marker feature selection method based on the collaboration of features and markers as described above.
[0063] Embodiment 4 The fourth embodiment of the present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the method for selecting multi-marker features based on the collaboration of features and marks as described above is implemented.
[0064] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0065] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
[0066] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.
[0067] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0068] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0069] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0070] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.
[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for selecting partial multi-marker features based on the collaboration of features and markers, characterized in that: include: S1, obtaining a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and its corresponding candidate label set; S2, converting the candidate marker set into a logical marker matrix; S3, using a linear model to construct the relationship between the feature space of the over-labeled sample data and the label distribution, and obtaining a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid overfitting of the model; S4, using a preset metric to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical label matrix; then constructing a feature label collaborative regularization term based on the feature similarity and the semantic similarity to guide the learning of label distribution; S5, according to The norm loss function and the feature labeling cooperate with the regularization term to construct the objective function of the feature selection model, and sparsely constrain the weights of the model based on sparse coding and norm regularization, and optimize the feature selection model using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; according to the optimal weight matrix and the linear model, update the label distribution matrix of the sample data; S6, performing feature selection according to the importance of features based on the optimal weight matrix to obtain an optimal feature subset, and then obtaining a new sample feature matrix, that is, a result of the over-labeled feature selection.
2. The method for selecting multiple marker features based on feature and marker collaboration according to claim 1, characterized in that , the S2 is specifically: The multi-labeled dataset is defined as: ; in, For the samples, Indicates that the dimension in the range of real numbers is The matrix of express of dimensional features, T is the transposed matrix, is the characteristic dimension of the sample; express The candidate tag set, To have The output space of candidate class labels, is the total sample size, Represents the number of category labels, i is the index variable; Constructing samples using hard thresholding method Logical label vector of , the formula is: ; in, is a logical label vector The j-th element of .
3. The method for selecting multiple marker features based on feature and marker collaboration according to claim 2, characterized in that ,The relationship between the feature space and label distribution of the partial multi-label sample data constructed by using a linear model is specifically: Assume that each column in the over-labeled sample data is a sample, then , represents the feature matrix composed of all samples; , represents the logical label matrix corresponding to all samples; The marker distribution matrix D obtained by linear model construction is: ; ; in, Representation sample The label distribution matrix is the weight matrix of the linear model; For the samples; represents the bias vector, Represents the matrix after the weight matrix is increased by one dimension, that is, an additional one-dimensional weight matrix ; Represents the sample matrix after one dimension is added, that is .
4. The method for selecting multiple marker features based on feature and marker collaboration according to claim 3, characterized in that ,pass The formula for the norm loss function to optimize the error between the label distribution matrix and the logical label matrix is: ; in, , represents the feature matrix of all samples after adding one dimension; for Norm.
5. The method for selecting multiple marker features based on feature and marker collaboration according to claim 4, characterized in that ,The preset metric includes Gaussian heat kernel function and cosine function; The Gaussian heat kernel function is used to construct a feature similarity matrix to measure the feature similarity between samples. The calculation formula is: ; ; ; in, is the feature similarity matrix; express and The feature similarity of represents the Gaussian kernel function; represents the kernel size; and Respectively represent and Sample data; for norm; The cosine function is used to construct a semantic similarity matrix to measure the semantic relevance of each group of samples. The calculation formula is: ; ; in, is the semantic similarity matrix, and Respectively represent samples and The corresponding logical label vector; Representation sample and The semantic similarity of .
6. The method for selecting partial multi-marker features based on feature and marker collaboration according to claim 5, characterized in that ,The calculation process of the feature marker collaborative regularization term is: According to the feature similarity matrix S and the semantic similarity matrix C, a feature tag collaborative cooperation matrix is constructed, and the calculation formula is: ; ; in, is the feature labeling collaboration matrix, Represents Hadamard; Then, a feature label collaboration regularization term is constructed based on the feature label collaboration matrix to guide label distribution learning. The formula is: ; in, represents the feature marker collaborative regularization term, For the matrix The Laplace matrix of is a diagonal matrix whose diagonal elements are , , Respectively represent samples , The distribution of markers, Represents the trace of the matrix.
7. The method for selecting multiple marker features based on feature and marker collaboration according to claim 6, characterized in that , the S5 is specifically: Based on sparse coding technology and Norm regularization imposes sparse constraints on the one-dimensional weight matrix, and the formula is: ; in, express norm; According to the The norm loss function cooperates with the feature labeling regularization term to construct the objective function of the feature selection model. The objective function expression is: ; in, Marking collaborative regularization term weight parameters for the feature; For sparse coding and The norm regularizes the weight parameter of the sparsity constraint; Update and optimize the objective function to obtain the optimal weight matrix ; According to the linear model formula, the updated marker distribution matrix is obtained , the expression is: 。 8. The method for selecting partial multi-marker features based on feature and marker collaboration according to claim 7, characterized in that , the S6 is specifically: according to The optimal weight matrix is calculated by The size of the modulus of each row is used to obtain the importance of the feature. The calculation formula is: ; in, express No. OK, for norm, Indicates Importance of dimensional features; Then the importance of the feature Sort in descending order and select the top The importance of each feature and its corresponding index, and get the selected feature index set ,in, Indicates the number of selected features; Get feature subsets based on the selected feature index set , and then get the new sample feature matrix : ; ; in, Represents the new sample feature matrix The i-th sample instance in express The kth feature of .
9. The method for selecting multiple marker features based on feature and marker collaboration according to claim 8, characterized in that The new sample feature matrix and the updated label distribution matrix are also used to train a multi-label classifier, and the input multi-label samples are classified by the trained multi-label classifier to obtain a multi-label classification result; wherein, the multi-label classifier adopts a multi-output regression model, and the multi-output regression model includes a multi-output support vector regression model, and the training process is: A multi-output support vector regression model is used as a multi-label classifier, according to the label distribution matrix And the new sample feature matrix The multi-label classifier is trained, and the formula of the objective function of the multi-label classifier is: ; in, , They represent the parameter matrices of the model, , represents the regularization parameter; for -Insensitive loss function; is the multi-label loss function of the sample; -Insensitive loss function The formula is: ; ; ; in, represents the error between the model output and the labeled distribution, Indicates that the sample Nonlinear kernel functions mapped to high-dimensional space; Indicates the set insensitive zone; Partially multi-label loss function The formula is: ; in, express exist The complement of Indicates a dimensional vector, if ,but No. elements are equal to 1, otherwise they are equal to 0; Indicates another dimensional vector, if ,but No. The element is equal to 1, otherwise it is equal to 0, T represents the transposed matrix; The objective function is optimized by iterative weighted least squares method to obtain the optimal model parameters. and And the trained classification model, the classification model expression is: ; Define the input multi-labeled test sample as , then, the new sample feature matrix output by the multi-label feature selection model is ; The label distribution of the overly labeled sample to be tested is calculated according to the trained classification model, and the calculation formula is: ; Use hard threshold method to judge samples Whether it belongs to the category mark, the judgment formula is: ,like ; in, express No. elements, and m is the preset threshold.
10. A device for selecting partial multi-marker features based on the collaboration of features and marks, characterized in that: include: A data acquisition unit, used to acquire a partially multi-labeled data set to be processed; wherein the partially multi-labeled data set includes partially multi-labeled sample data and a corresponding candidate label set; A logical tag generation unit, used for converting the candidate tag set into a logical tag matrix; The label distribution relationship unit is used to construct the relationship between the feature space of the partial multi-label sample data and the label distribution by using a linear model, and obtain a label distribution matrix; and The norm loss function optimizes the error between the label distribution matrix and the logical label matrix to avoid overfitting of the model; A feature labeling collaboration unit is used to calculate the feature similarity between each sample in the partial multi-labeled data set and the semantic similarity of the logical labeling matrix using a preset metric; and then construct a feature labeling collaboration regularization term based on the feature similarity and the semantic similarity to guide the learning of labeling distribution; The model optimization unit is used to optimize the The norm loss function and the feature labeling cooperate with the regularization term to construct the objective function of the feature selection model, and sparsely constrain the weights of the model based on sparse coding and norm regularization, and optimize the feature selection model using the alternating direction multiplier method to obtain the optimal weight matrix of the feature selection model; according to the optimal weight matrix and the linear model, update the label distribution matrix of the sample data; The feature selection unit is used to perform feature selection according to the optimal weight matrix and the importance of the features to obtain the optimal feature subset, and then obtain a new sample feature matrix, that is, the result of the partial multi-label feature selection.
Citation Information
Patent Citations
Partial multi-mark learning method based on multi-subspace representation
CN111581469A
Partial multi-mark learning method based on global and local mark relations
CN111582506A
Partial multi-mark learning method for eliminating noise marks based on instance correlation
CN117668548A
Method and system for evaluating clinical efficacy of multi-label multi-class computational diagnostic models
US20240096492A1
Cited By
Multi-topic text data feature selection method with slack marks
CN119149982A
