Online multi-label feature selection method for label missing stream feature scene

Abstract

The invention discloses an online multi-label feature selection method for a label missing stream feature scene, and belongs to the technical field of feature selection. Aiming at a multi-label data environment in which features are dynamically reached and labels are missing, the method comprises the following steps: firstly, recovering missing labels in a multi-label decision system to obtain a complete label set; constructing a fuzzy similar class under the label set by using the fuzzy label co-occurrence similarity; when a new feature arrives, constructing a fuzzy similar class under the feature through a Gaussian kernel function, calculating the fuzzy relative correlation between the fuzzy similar class and the complete label set, and if the fuzzy relative correlation is lower than a threshold value, discarding the fuzzy relative correlation; otherwise, further analyzing the fuzzy redundancy between the features and the selected features, and determining to retain, replace or discard the features according to a redundancy comparison result. According to the method, efficient online feature selection can be realized under the condition that a complete feature space is unknown in advance and a label is incomplete, and the method has the advantages of high uncertainty processing capability, adaptability to a dynamic data environment and effective reduction of feature dimensions.

Description

Technical Field

[0001] This invention relates to feature selection methods, specifically an online multi-label stream feature selection method for scenarios with missing labels, which is particularly suitable for multi-label data environments where features arrive dynamically and labels are missing. Background Technology

[0002] With the advent of the big data era, the dimensionality of data continues to rise, bringing greater pressure to data mining and machine learning tasks. Multi-label feature selection, as an important preprocessing step in data mining, pattern recognition, and machine learning tasks, eliminates redundant and irrelevant attributes from a large number of features, reducing data dimensionality and improving algorithm efficiency.

[0003] In practical applications, the feature space is often unknowable in advance, arriving sequentially as features flow, meaning features are dynamically generated with a fixed set of instances. To address this challenge, multi-label streaming feature selection methods have emerged. Compared to traditional multi-label feature selection methods, these methods can select feature subsets online in real time from the dynamically arriving feature stream, enabling the model to flexibly adapt to the ever-changing feature set.

[0004] The acquisition of multi-label data is often constrained. Due to limitations such as high data annotation costs, missing instance labels are common, resulting in incomplete multi-label data. In such scenarios with missing labels, traditional multi-label streaming feature selection methods cannot directly address the problem. More importantly, in streaming feature scenarios, the complete feature space is unknown, rendering many traditional methods that rely on complete feature space information for label completion or recovery unsuitable. Therefore, current technology lacks a method that can effectively handle multi-label streaming feature selection under conditions of missing labels.

[0005] In the complex environment of multi-label learning, effectively handling data uncertainty is crucial for feature learning. To address this, fuzzy rough set theory constructs a mathematical framework for handling uncertainty and quantifying the fuzzy relationships between features and labels. Fuzzy similarity relations are used to measure the similarity between samples, while fuzzy similarity classes form a fuzzy partition of the universe of discourse; together, they constitute an important foundation for feature selection methods based on fuzzy rough sets. However, real-world multi-label data often exhibits sparsity: the vast majority of labels are negative, while positive labels, though scarce, often contain stronger discriminative information. In this context, traditional similarity measurement methods based on label consistency struggle to fully capture and utilize the key information conveyed by positive labels. Summary of the Invention

[0006] In view of the shortcomings of existing technologies, this invention proposes an online multi-label feature selection method for feature stream scenarios with missing labels. This method aims to effectively handle complex situations involving missing label data and dynamically arriving features, thereby dynamically reducing feature dimensionality.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0008] This invention proposes an online multi-label feature selection method for stream feature scenarios with missing labels, specifically including:

[0009] Step 1: Input a multi-label decision system with missing labels. ,in It is the domain of discourse. It is a set of features that are dynamically arrived. It is a collection of labels with missing values. It is the range. It is U and arrive The mapping; and the first threshold Second threshold ;

[0010] Step 2: Recover missing label data in the multi-label decision-making system to generate a complete label set. ;

[0011] Step 3: Based on the restored complete label set, construct a fuzzy label co-occurrence similarity model. The fuzzy similarity matrix between samples in the lower universe of discourse U; constructing fuzzy similarity classes under L using fuzzy similarity. ;

[0012] Step 4: Initialize the selected feature set It is an empty set;

[0013] Step 5: When new features Upon arrival, construct using a Gaussian kernel function. Similarity matrix between samples in the lower universe of discourse U; constructing using fuzzy similarity Fuzzy similarity class ;

[0014] Step Six: Calculation With a complete set of tags Fuzzy relative correlation between ,like Then discard permanently. Return to step five to process the next arriving feature; otherwise, proceed to the next step.

[0015] Step 7: For the currently selected feature set Various features Calculate separately Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy ;

[0016] Step 8: [Regarding...] With the currently selected feature set Various features Perform pairwise redundancy analysis, if throw away Return to step five to process the next arriving feature; if Then use replace And proceed to the next step;

[0017] Step 9: Return to Step 5 until no new features arrive;

[0018] Step 10: Output the final selected feature set .

[0019] In step two, a complete set of tags is required. Obtained through the following steps:

[0020] Step 1: For multi-label decision systems with missing labels Each sample in traverse its label vector Identify all tags as missing, i.e. The position of is denoted as sample. missing label set ;

[0021] Step 2: For the sample Each missing label Samples are generated using formula (1). Compared with other samples in the system In addition to the label Label consistency rate on all labels except those outside for

[0022] Formula (1)

[0023] in, For indicator functions, if Returns 1 if true, otherwise returns 0;

[0024] Step 3: Generate samples using formula (2) In the label The probability estimate of the value being 1 is:

[0025] Formula (2)

[0026] in, , ;

[0027] Step 4: If ,Will The value is restored to 1; otherwise, it is restored to 0.

[0028] The fuzzy label co-occurrence similarity in step three Obtained through formula (3):

[0029] Formula (3)

[0030] in, For samples in the universe of discourse.

[0031] In step three, the fuzzy similarity class under L Obtained through formula (4):

[0032] Formula (4)

[0033] Where n is Total number of samples.

[0034] In step five Similarity between samples in the lower domain U Obtained through formula (5):

[0035] Formula (5)

[0036] in, For samples in the universe of discourse, Let be the Euclidean distance between the two samples. The standard deviation is denoted as .

[0037] In step five Fuzzy similarity class Obtained through formula (6):

[0038] Formula (6)

[0039] Where n is Total number of samples.

[0040] In step six With a complete set of tags Fuzzy relative correlation between Obtained through formula (7):

[0041] Formula (7)

[0042] in, and Fuzzy mutual information and fuzzy entropy are generated by formulas (8) and (9), respectively:

[0043] Formula (8)

[0044] Formula (9)

[0045] Where n is Total number of samples for Fuzzy similarity classes, For fuzzy similarity classes under L, The cardinality of fuzzy similarity classes.

[0046] In step seven Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy Generate using formulas (10) and (11) respectively:

[0047] Formula (10)

[0048] Formula (11)

[0049] in, and The fuzzy conditional mutual information is generated using formulas (12) and (13) respectively:

[0050] Formula (12)

[0051] Formula (13)

[0052] Where n is Total number of samples for Fuzzy similarity classes, for Fuzzy similarity classes, for Fuzzy similarity classes, The cardinality of fuzzy similarity classes.

[0053] The beneficial effects of this invention are:

[0054] 1. This invention can restore missing labels in streaming feature scenarios where the feature space is unknown in advance. When the label data is incomplete, it can restore labels using only the existing label information without relying on the complete global feature space. This solves the problem that traditional methods cannot be directly applied in scenarios where labels are missing and features arrive dynamically.

[0055] 2. This invention uses fuzzy label co-occurrence similarity to calculate the fuzzy similarity between samples in a label set, which can more effectively capture the information contained in the data with label 1, and is more suitable for the sparse characteristics of label set data.

[0056] 3. This invention adopts a streaming processing mechanism to analyze and filter features one by one, supports dynamic real-time feature selection and discarding, and effectively realizes feature dimensionality reduction in high-dimensional and dynamic data environments where the complete feature space is unknown. Attached Figure Description

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2 This is a flowchart illustrating the implementation of the present invention. Detailed Implementation

[0059] The working principle and working steps of the present invention will be further explained below with reference to the accompanying drawings:

[0060] The method flow of this invention is described in [reference]. Figure 1 This invention provides an online multi-label feature selection method for stream feature scenarios with missing labels, which includes the following steps:

[0061] Step 1: Input a multi-label decision system with missing labels. ,in It is the domain of discourse. It is a set of features that are dynamically arrived. It is a collection of labels with missing values. It is the range. It is U and arrive The mapping; and the first threshold Second threshold ;

[0062] Step 2: Recover missing label data in the multi-label decision-making system to generate a complete label set. ;

[0063] Step 3: Based on the restored complete label set, construct a fuzzy label co-occurrence similarity model. The fuzzy similarity matrix between samples in the lower universe of discourse U; constructing fuzzy similarity classes under L using fuzzy similarity. ;

[0064] Step 4: Initialize the selected feature set It is an empty set;

[0065] Step 5: When new features Upon arrival, construct using a Gaussian kernel function. Similarity matrix between samples in the lower universe of discourse U; constructing using fuzzy similarity Fuzzy similarity class ;

[0066] Step Six: Calculation With a complete set of tags Fuzzy relative correlation between ,like Then discard permanently. Return to step five to process the next arriving feature; otherwise, proceed to the next step.

[0067] Step 7: For the currently selected feature set Various features Calculate separately Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy ;

[0068] Step 8: [Regarding...] With the currently selected feature set Various features Perform pairwise redundancy analysis, if throw away Return to step five to process the next arriving feature; if Then use replace And proceed to the next step;

[0069] Step 9: Return to Step 5 until no new features arrive;

[0070] Step 10: Output the final selected feature set .

[0071] In step two, a complete set of tags is required. Obtained through the following steps:

[0072] Step 1: For multi-label decision systems with missing labels Each sample in traverse its label vector Identify all tags as missing, i.e. The position of is denoted as sample. missing label set ;

[0073] Step 2: For the sample Each missing label Samples are generated using formula (1). Compared with other samples in the system In addition to the label Label consistency rate on all labels except those outside for

[0074] Formula (1)

[0075] in, For indicator functions, if Returns 1 if true, otherwise returns 0;

[0076] Step 3: Generate samples using formula (2) In the label The probability estimate of the value being 1 is:

[0077] Formula (2)

[0078] in, , ;

[0079] Step 4: If ,Will The value is restored to 1; otherwise, it is restored to 0.

[0080] The fuzzy label co-occurrence similarity in step three Obtained through formula (3):

[0081] Formula (3)

[0082] in, For samples in the universe of discourse.

[0083] In step three, the fuzzy similarity class under L Obtained through formula (4):

[0084] Formula (4)

[0085] Where n is Total number of samples.

[0086] In step five Similarity between samples in the lower domain U Obtained through formula (5):

[0087] Formula (5)

[0088] in, For samples in the universe of discourse, Let be the Euclidean distance between the two samples. The standard deviation is denoted as .

[0089] In step five Fuzzy similarity class Obtained through formula (6):

[0090] Formula (6)

[0091] Where n is Total number of samples.

[0092] In step six With a complete set of tags Fuzzy relative correlation between Obtained through formula (7):

[0093] Formula (7)

[0094] in, and Fuzzy mutual information and fuzzy entropy are generated by formulas (8) and (9), respectively:

[0095] Formula (8)

[0096] Formula (9)

[0097] Where n is Total number of samples for Fuzzy similarity classes, For fuzzy similarity classes under L, The cardinality of fuzzy similarity classes.

[0098] In step seven Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy Generate using formulas (10) and (11) respectively:

[0099] Formula (10)

[0100] Formula (11)

[0101] in, and The fuzzy conditional mutual information is generated using formulas (12) and (13) respectively:

[0102] Formula (12)

[0103] Formula (13)

[0104] Where n is Total number of samples for Fuzzy similarity classes, for Fuzzy similarity classes, for Fuzzy similarity classes, The cardinality of fuzzy similarity classes.

[0105] See implementation process Figure 2 Specifically:

[0106] (1) Multi-label decision system with input data containing missing labels

[0107] (2) Recovery of missing label data

[0108] (3) Dynamic selection of flow characteristics

[0109] (4) Output the selected feature set

[0110] Experimental example:

[0111] The effectiveness of the method presented in this invention is demonstrated by running it on the real-world dataset "Business". Table 1 shows the detailed information of the dataset. To verify the effectiveness of the missing label recovery method in this invention, the proposed label recovery strategy was run on a multi-label decision system containing missing labels. The recovered labels were compared with the original complete labels, and the accuracy of the label recovery was statistically analyzed. The results are shown in Table 2. Based on this, the online multi-label stream feature selection method proposed in this invention was used to filter dynamically arriving features, ultimately obtaining the feature-selected dataset. The original dataset and the feature-selected dataset were respectively input into the MLKNN classifier for training and testing, and Average Precision was used as the evaluation metric. The higher the value of this metric, the better the classification performance. The experimental results are shown in Table 3. After processing by the method of this invention, the classifier achieved better performance on the Average Precision metric, proving that the method of this invention can effectively recover missing labels and filter high-quality feature subsets online in complex scenarios with missing labels and dynamically arriving features, thereby improving the performance of multi-label learning tasks.

[0112] Table 1: Business Dataset Details

[0113] Dataset Total number of instances Number of training set instances Number of test set instances Feature set Tag set Business 5000 2000 3000 438 30

[0114] Table 2: Accuracy of Label Missing Recovery

[0115] Dataset Label missing rate Tag recovery accuracy Business 0.2 97.02%

[0116] Table 3: Comparison of the dataset after feature selection with the original dataset

[0117] Feature-selected dataset Original dataset Number of features 31 438 Average Precision 0.8735 0.8650

[0118] The above embodiments are not intended to limit the present invention, and the present invention is not limited to the above embodiments. Any embodiment that meets the requirements of the present invention is within the protection scope of the present invention.

Claims

1. An online multi-label feature selection method for stream feature scenarios with missing labels, characterized in that, Input: ① A multi-label decision system with missing labels ,in It is the domain of discourse. It is a set of features that are dynamically arrived. It is a collection of labels with missing values. It is the range. It is U and arrive Mapping; ② First threshold Second threshold ; Output: Selected feature set ; The method specifically includes the following steps: S101, recover the missing labels in the multi-label decision system to obtain a complete label set. ; S102, Calculate the complete label set using fuzzy label co-occurrence similarity. The fuzzy similarity matrix between samples in the lower universe of discourse U; constructing fuzzy similarity classes under L using fuzzy similarity. ; S103, Initialize the selected feature set It is an empty set; S104, when new features Upon arrival, construct using a Gaussian kernel function. Similarity matrix between samples in the lower universe of discourse U; constructing using fuzzy similarity Fuzzy similarity class ; S105, calculate its relationship with the complete tag set. Fuzzy relative correlation between ,like Then discard permanently. Return to step S104 to process the next arriving feature; otherwise proceed to the next step. S106, for the currently selected feature set Various features Calculate separately Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy ; S107, for With the currently selected feature set Various features Perform pairwise redundancy analysis, if throw away Return to step S104 to process the next arriving feature; if Then use replace And proceed to the next step; S108, return to step S104 until no new features arrive; S109, Output the final selected feature set .

2. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 1, characterized in that, Step S101 includes: S201, For multi-label decision systems with missing labels Each sample in traverse its label vector Identify all tags as missing, i.e. The location is denoted as sample. missing label set ; S202, for samples Each missing label Generate samples Compared with all other samples in the system In addition to the label Label consistency rate on all labels except those outside ; S203, Based on the label consistency ratio from step S202, generate samples. In the label Probability estimate of the value being 1 ; S204, recover the missing labels based on the probability estimate from step S203, if ,Will The value is restored to 1; otherwise, it is restored to 0.

3. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 2, characterized in that, Tag consistency ratio in step S202 Generate using formula (1): Official (1) in, For indicator functions, if Returns 1 if true, otherwise returns 0; In step S203, the sample In the label The probability estimate of the value of 1 is generated by formula (2): Official (2) in, , .

4. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 1, characterized in that, The co-occurrence similarity of fuzzy labels in step S102 Obtained through formula (3): Official (3) in, For samples in the universe of discourse; In step S102, the fuzzy similarity class under L Obtained through formula (4): Official (4) Where n is Total number of samples.

5. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 1, characterized in that, In step S104, the Gaussian kernel function is obtained through formula (5): Official (5) in, For samples in the universe of discourse, Let be the Euclidean distance between the two samples. Standard deviation; In step S104 Fuzzy similarity class Obtained through formula (6): Official (6) Where n is Total number of samples.

6. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 1, characterized in that, In step S105 With a complete set of tags Fuzzy relative correlation between Generate using formula (7): Official (7) in, and Fuzzy mutual information and fuzzy entropy are generated by formulas (8) and (9), respectively: Official (8) Official (9) Where n is Total number of samples for Fuzzy similarity classes, For fuzzy similarity classes under L, The cardinality of fuzzy similarity classes.

7. The online multi-label feature selection method for stream feature scenarios with missing labels as described in claim 1, characterized in that, In step S106 Given the following Fuzzy redundancy ,as well as Given the following Fuzzy redundancy Generate using formulas (10) and (11) respectively: Official (10) Official (11) in, and The fuzzy conditional mutual information is generated using formulas (12) and (13) respectively: Official (12) Official (13) Where n is Total number of samples for Fuzzy similarity classes, for Fuzzy similarity classes, for Fuzzy similarity classes, The cardinality of fuzzy similarity classes.

Images