Fuzzy multi-label inference learning method for heterogeneous data

Abstract

The application discloses a fuzzy multi-label inference learning method for heterogeneous data, comprising: label-aware enhanced feature construction, selecting a neighbor based on a hybrid difference metric and constructing a label-aware enhanced feature vector according to a difference relationship of positive and negative neighbor weight cumulative values; fusion enhanced antecedent construction, splicing the original feature and the enhanced feature and then obtaining a fuzzy feature vector through fuzzy rule activation; label structure guided consequent alignment, constructing a label difference matrix based on a statistical correlation coefficient between labels and taking the label difference matrix as a structured alignment constraint term acting on a consequent parameter matrix; joint optimization and prediction, integrating a data fitting term and the structured alignment constraint term into a unified objective function for optimization and solution to generate a multi-label prediction result. The application solves the problems of feature mapping modeling difficulty, insufficient learning of scarce labels and label dependency relationship interference in the heterogeneous data scene, and improves the accuracy and stability of multi-label classification.

Description

Technical Field

[0001] This application belongs to the field of heterogeneous data multi-label classification and intelligent reasoning learning technology, specifically involving a fuzzy multi-label reasoning learning method for heterogeneous data. Background Technology

[0002] Multi-label classification of heterogeneous data is a fundamental task in information processing, widely applied in image scene classification, audio content classification, and biological function category discrimination. Heterogeneous data refers to data of different forms that differ in representation and statistical distribution. For example, text, images, and audio data each possess different feature expressions and noise characteristics, thus requiring classification methods to have universal adaptability across data forms, enabling multi-label classification of various data types. In traditional classification settings, a single data sample typically corresponds to only one category label; however, in heterogeneous data scenarios, the same sample often involves multiple semantic categories, requiring the output of a result set composed of multiple labels, thus forming a multi-label classification problem. Compared to single-label classification, multi-label classification not only needs to learn the mapping rules from data features to the label set but also needs to ensure the overall consistency of the multi-label output. Furthermore, due to differences in feature expression, statistical distribution, and noise levels among different data forms, the model's discriminative ability and generalization performance are more prone to fluctuations across different data types, making its modeling and learning process more challenging.

[0003] Existing multi-label classification methods can be broadly categorized into two types: decompositional learning methods and joint learning methods. Decompositional learning methods typically break down the multi-label task into several manageable sub-tasks, such as transforming it into multiple binary discriminators, label combination classifiers, or chain prediction processes, and then calling existing learners to complete training and inference separately. Joint learning methods, on the other hand, directly construct a unified learning model and objective function within the multi-label output space, achieving synchronous prediction of multiple labels through overall optimization, and introducing output coordination mechanisms into the model structure, constraints, or inference strategies to better characterize the overall consistency and decision stability of multi-label results.

[0004] However, existing heterogeneous data multi-label classification methods struggle to simultaneously address the following core challenges in complex scenarios within the same framework: First, feature-to-label mapping modeling is difficult. In heterogeneous data classification, different data formats have significant differences in feature representation and noise levels, resulting in inconsistent discrimination boundaries for the same label under different data formats. This increases the difficulty of feature-to-label mapping modeling and affects the stability of multi-label classification.

[0005] Second, there is insufficient learning of scarce labels. In heterogeneous data, the label distribution often exhibits long-tail characteristics, and the number of samples for some labels in different data formats is relatively small. This causes the model training process to favor high-frequency labels, while the learning of scarce labels is insufficient, which in turn reduces the recognition accuracy of scarce labels.

[0006] Third, interference from inter-label dependencies. In multi-label joint prediction, complex relationships such as co-occurrence, dependency, or mutual exclusion commonly exist between labels. When the correlation patterns of labels under different data formats are inconsistent, the dependencies between labels are more likely to cause inference conflicts, thereby reducing the reliability of the overall inference and the application effect.

[0007] Therefore, there is an urgent need for a heterogeneous data multi-label classification method that can effectively address the three core challenges mentioned above within a single framework. This method needs to construct a feature-to-label mapping relationship while simultaneously considering the learning ability of scarce labels and the inference constraints of mutual influence between labels, thereby further improving the accuracy and stability of heterogeneous data multi-label classification. Summary of the Invention

[0008] To achieve the above objectives, this application provides a fuzzy multi-label inference learning method for heterogeneous data, the detailed technical solution of which is as follows: A fuzzy multi-label inference learning method for heterogeneous data includes the following steps: Label-aware enhanced feature construction: For each sample instance to be processed, based on a hybrid difference measure that integrates at least two similarities, a preset number of nearest neighbor samples with the smallest difference from the sample instance are selected from the training set to form a nearest neighbor sample set, and the sample instance is assigned a weight according to the similarity between each nearest neighbor sample and the sample instance; for each label dimension, the nearest neighbor sample set is divided into a positive neighbor set and a negative neighbor set according to whether each sample in the nearest neighbor sample set has the label, and based on the difference between the cumulative weight value of the positive neighbor set and the cumulative weight value of the negative neighbor set, a label-aware enhanced feature vector that can characterize its local label discrimination characteristics is constructed for the sample instance; Fusion enhancement antecedent construction: The original feature vector of the sample instance is concatenated with the label-aware enhancement feature vector to form a fusion enhancement feature vector. The fusion enhancement feature vector is used as the input space for antecedent learning. The fusion enhancement feature vector is activated by fuzzy rules to obtain the corresponding fuzzy feature vector, so as to realize fuzzy rule partitioning guided by label information. Label structure-guided consequent alignment: Based on the true label distribution of all labels in the training set, the statistical correlation coefficient between each pair of labels is calculated, and a label difference matrix reflecting the degree of difference between labels is constructed based on the statistical correlation coefficient. During the consequent parameter learning process, a structured alignment constraint term constructed based on the label difference matrix is ​​introduced. By applying the structured alignment constraint term to the consequent parameter matrix, the consequent parameters corresponding to labels with strong statistical correlation are made closer in the parameter space, while the consequent parameters corresponding to labels with weak statistical correlation or mutual exclusivity are kept distinguishable in the parameter space. Joint optimization and prediction: The data fitting term that minimizes the error between the predicted label and the true label after the fuzzy feature vector is mapped by the consequent parameter matrix, and the structured alignment constraint term are integrated into a unified objective function. The trained consequent parameter matrix is ​​obtained by optimizing and solving the objective function. For new samples, multi-label prediction results are generated based on their fused enhanced feature vector and the trained consequent parameter matrix.

[0009] The fuzzy multi-label inference learning method for heterogeneous data proposed in this application has the following technical effects: 1. By constructing a label-aware enhanced feature structure, the label distribution information of the nearest neighbors of the sample is explicitly encoded into an enhanced feature vector and concatenated with the original feature vector. This allows the input space of the prior learning to simultaneously contain the original feature information and the local label discrimination information, solving the problem of difficult feature-to-label mapping modeling in heterogeneous data scenarios and improving the discrimination ability of multi-label classification.

[0010] 2. By integrating and enhancing the antecedent construction steps, the enhanced feature vector is used as the input space for antecedent learning, so that the division of fuzzy rules is directly guided by the label information. The resulting fuzzy rule division is more discriminative and can better adapt to the differences in feature distribution under different data forms.

[0011] 3. Through the consequent alignment step guided by the label structure, a label difference matrix is ​​constructed based on the statistical correlation coefficient between labels, and it is used as a structured alignment constraint term to act on the consequent parameter matrix. This enables labels with strong statistical correlation to approach each other in the parameter space, while labels with weak statistical correlation or mutual exclusivity remain distinguishable in the parameter space. This effectively alleviates the inference conflict caused by the dependency relationship between labels and improves the overall consistency of multi-label joint prediction.

[0012] 4. By integrating the data fitting term that minimizes the prediction error with the structured alignment constraint term into a unified objective function and performing joint optimization, the learning of the feature-label mapping relationship and the utilization of label structure information are synergistic rather than independent, thereby improving the overall accuracy and stability of multi-label classification of heterogeneous data.

[0013] 5. Through the coordinated efforts of the above four steps, the learning effect of scarce labels under imbalanced label distribution conditions is improved without changing the overall learning framework. This allows samples related to low-frequency labels to receive clearer support in rule activation, thereby improving the overall classification performance under imbalanced conditions.

[0014] In some embodiments, the at least two similarities fused by the hybrid difference measure method include Pearson correlation coefficient and cosine similarity, and the construction process of the hybrid difference measure method is as follows: For sample instances and Centralization is performed to eliminate the effect of mean bias; Calculate the Pearson correlation coefficient between the two respectively. Similarity to cosine ; Introduce a balance weight to adjust the contributions of both. The range is calculated using the following formula: Mixed differences : .

[0015] By specifically defining the hybrid difference measure as a weighted fusion of Pearson correlation coefficient and cosine similarity, it considers both the strength of linear correlation between samples and the consistency of vector direction, thus providing a more comprehensive measure of the differences in heterogeneous data. The degree of difference between samples with the same distribution characteristics provides a more robust metric for subsequent nearest neighbor selection.

[0016] In some embodiments, the label-aware enhanced feature vector The construction process is as follows: Based on the aforementioned hybrid differences , for sample instances The nearest neighbors with the smallest differences are selected to form the nearest neighbor sample set. ; For any nearest neighbor Assign weights that are positively correlated with similarity according to the following formula. : ; For the Each label dimension will be used to define the sample instances. The set of positive neighbors is denoted as The set of negative neighbors is denoted as ; The cumulative weights of the positive neighbor set and the cumulative weights of the negative neighbor set are calculated using the following formulas: , ; Definition of the first The enhanced feature values ​​on each label dimension are: ; Combine the enhanced feature values ​​of all L label dimensions to form a sample instance. The label-aware enhanced feature vector: .

[0017] By specifying the complete construction process of the enhanced feature vector, the nearest neighbor size, weight assignment method, and difference calculation of the cumulative weights of positive and negative neighbors are clarified, enabling the label-aware enhanced features to accurately reflect the label dimensions of samples in the local neighborhood. The relative discriminative strength in degree provides a more discriminative input representation for the antecedent learning of fuzzy rules.

[0018] In some embodiments, the true label matrix Y of all training samples is obtained, and the value vector of each label is centered to obtain a centered label matrix. ; based on Calculate the Pearson correlation coefficient between each pair of tags, and use this as the statistical correlation coefficient to construct the tag correlation matrix. ; The label difference matrix is ​​constructed according to the statistical correlation coefficient using the following formula. ,in A column vector consisting entirely of 1s: .

[0019] By specifically defining the construction process of the label difference matrix, and using centering to calculate the Pearson correlation coefficient and convert it into a difference matrix, we can accurately capture the real statistical dependency structure between labels, providing a reliable relational constraint basis for the structured alignment of consequent parameters.

[0020] In some embodiments, the structured alignment constraint term acting on the consequent parameter matrix is ​​formally expressed as: , in, Let be the global consequent parameter matrix to be learned. Represents the trace operation of a matrix.

[0021] By formalizing the structured alignment constraint terms into trace operations, the constraints of the label difference matrix on the consequent parameters can be expressed in a concise matrix form, facilitating efficient gradient calculation and joint optimization within a unified objective function.

[0022] In some embodiments, the unified objective function formed in the joint optimization and prediction step is: , in, The fuzzy feature matrix is ​​obtained by activating the fuzzy rules in the prior learning of the fused and enhanced feature vectors of all samples. It is the Frobenius norm. It is an L1 norm. and Hyperparameters are used to balance the weights of each component.

[0023] By defining a unified objective function consisting of three parts—a data fitting term, a structured alignment constraint term, and an L1 norm sparse regularization term—label structure constraints and parameter sparsity are introduced while ensuring prediction accuracy. This effectively suppresses redundant rule parameters and improves the model's generalization ability and anti-overfitting performance.

[0024] In some embodiments, the unified objective function is optimized using a proximal gradient descent method combined with a momentum extrapolation strategy. Decompose the objective function into a differentiable smooth part. With the non-smooth part containing the L1 norm ; Utilizing the Lipschitz continuity of the smooth partial gradient, let its Lipschitz constant be... Construct a surrogate function at the current iteration point; In each iteration, a search point is constructed based on the momentum combination of the results of the previous two iterations: , Where the sequence satisfy , This is the result of the t-th iteration; Calculate the intermediate variables for proximal gradient descent: , in, The gradient of the smooth portion at the search point; By performing element-wise soft thresholding, the result of the (t+1)th iteration is obtained in closed-form solution. ; , in, for The corresponding element, The threshold for soft thresholding operations.

[0025] By employing a proximal gradient descent method combined with momentum extrapolation to optimize the objective function, a surrogate function is constructed using the Lipschitz continuity of the smooth portion, and momentum extrapolation is used to accelerate convergence and element-wise soft thresholding is used to achieve closed-form solution. This results in a faster convergence speed and lower solution complexity during model training, ensuring the computational feasibility of the method.

[0026] In some embodiments, the heterogeneous data includes feature representations from at least two data formats: text, images, and audio.

[0027] By specifically limiting the range of heterogeneous data types to at least two of the data formats of text, image, and audio, the application scenarios of this method are made clearer, demonstrating its general adaptability to cross-modal heterogeneous data.

[0028] In some embodiments, the antecedent learning is implemented using a TSK fuzzy system, and the fuzzy rules of the TSK fuzzy system are used to activate the fused enhanced feature vector to map the fused enhanced feature vector to a corresponding fuzzy feature vector for learning consequent parameters and final label prediction.

[0029] By employing the TSK fuzzy system to achieve antecedent learning, the fused enhanced feature vector is activated by fuzzy rules and mapped to a fuzzy feature vector. This allows the model to inherit the interpretability advantage of the TSK fuzzy system and gain stronger discriminative ability by enhancing the feature vector, thus achieving an effective balance between interpretability and classification performance.

[0030] In some embodiments, the initial values ​​of the consequent parameter matrix are determined before the first iteration. Determine as follows: , in, It's a hyperparameter. It is an identity matrix.

[0031] By using the regularized least squares analytical solution as the initial value of the consequent parameter matrix for iteration, the instability or slow convergence problems that may be caused by starting optimization from a random initial point are avoided, ensuring the numerical stability of the optimization process. At the same time, it provides a better search starting point for subsequent proximal gradient descent iterations, accelerating the convergence process of model training. Attached Figure Description

[0032] Figure 1 This is a general framework diagram of the method of this application.

[0033] Figure 2 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the AP index.

[0034] Figure 3 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the RL index.

[0035] Figure 4 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the OE index.

[0036] Figure 5 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the CV index.

[0037] Figure 6 This application demonstrates the neighborhood size on seven datasets. The parameter analysis chart.

[0038] Figure 7 This application demonstrates the fusion weights on seven datasets. The parameter analysis chart.

[0039] Figure 8 This application is about the number of rules on 7 datasets. The parameter analysis chart.

[0040] Figure 9 This application demonstrates its hyperparameter performance on seven datasets. The parameter analysis chart.

[0041] Figure 10 This application demonstrates its hyperparameter performance on seven datasets. The parameter analysis chart.

[0042] Figure 11 This is a convergence analysis plot of this application on the CAL500 dataset.

[0043] Figure 12 This is a convergence analysis graph of this application on the Emotions dataset.

[0044] Figure 13 This is a convergence analysis graph of this application on the Flags dataset.

[0045] Figure 14 This is a convergence analysis graph of this application on the Image dataset.

[0046] Figure 15 This is a convergence analysis plot of this application on the Rcv1s1 dataset.

[0047] Figure 16 This is a convergence analysis graph of this application on the Scene dataset.

[0048] Figure 17 This is a convergence analysis graph of this application on the Yeast dataset.

[0049] Figure 18 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the AP index.

[0050] Figure 19 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the RL index.

[0051] Figure 20 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the OE index.

[0052] Figure 21 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the CV index. Detailed Implementation

[0053] It should be noted that the following detailed descriptions are exemplary and intended to provide indicative explanations of the content of this application. It should be noted that all technical and scientific terms used in this application have the same meaning as commonly understood by a person skilled in the art to which this application pertains.

[0054] The system architecture and prior art solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only for explanation and illustration of this application, and not the entirety of the content. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without creative effort are within the protection scope of this application.

[0055] Example 1: like Figure 1 As shown, the fuzzy multi-label inference learning method for heterogeneous data provided in this application includes the following steps: S1. Label-aware enhanced feature construction: For each sample instance to be processed, based on a hybrid difference measurement method that integrates at least two similarities, a preset number of nearest neighbor samples with the smallest difference from the sample instance are selected in the training set to form a nearest neighbor sample set, and the sample instance is assigned a weight according to the similarity between each nearest neighbor sample and the sample instance; for each label dimension, the nearest neighbor sample set is divided into a positive neighbor set and a negative neighbor set according to whether each sample in the nearest neighbor sample set has the label, and based on the difference between the cumulative weight value of the positive neighbor set and the cumulative weight value of the negative neighbor set, a label-aware enhanced feature vector that can characterize its local label discrimination characteristics is constructed for the sample instance; S2. Construction of Fusion Enhancement Precondition: The original feature vector of the sample instance is concatenated with the label-aware enhancement feature vector to form a fusion enhancement feature vector. The fusion enhancement feature vector is used as the input space for precondition learning. The fusion enhancement feature vector is activated by fuzzy rules to obtain the corresponding fuzzy feature vector, so as to realize fuzzy rule partitioning guided by label information. S3. Label Structure-Guided Successor Alignment: Based on the true label distribution of all labels in the training set, calculate the statistical correlation coefficient between each pair of labels, and construct a label difference matrix reflecting the degree of difference between labels based on the statistical correlation coefficient. During the successor parameter learning process, introduce a structured alignment constraint term based on the label difference matrix. By applying the structured alignment constraint term to the successor parameter matrix, the successor parameters corresponding to labels with strong statistical correlation tend to converge in the parameter space, while the successor parameters corresponding to labels with weak statistical correlation or mutual exclusivity remain distinguishable in the parameter space. S4. Joint Optimization and Prediction: The data fitting term that minimizes the error between the predicted label and the true label after the fuzzy feature vector is mapped by the consequent parameter matrix, and the structured alignment constraint term are integrated into a unified objective function. The trained consequent parameter matrix is ​​obtained by optimizing and solving the objective function. For new samples, multi-label prediction results are generated based on their fused enhanced feature vector and the trained consequent parameter matrix.

[0056] The Label-Aware Fuzzy Enhanced Multi-label Learning Method (LAFEM) proposed in this application consists of antecedent learning and consequent learning. Antecedent learning takes the original feature space as input, constructs an enhanced antecedent representation through a label-aware mechanism, and completes antecedent parameter learning and fuzzy parameter determination within this enhanced space, aiming to guide fuzzy rules to form a more reasonable partition in the joint feature-label structure space. Consequence learning revolves around modeling the label output mapping relationship and aligning consequent relevance. While completing the basic mapping relationship learning, it introduces label data to further enhance the model's ability to model multi-label association information.

[0057] This application improves the co-processing of antecedent and consequent learning within the traditional TSK (Takagi-Sugeno-Kang) fuzzy system framework. This allows the model to complete the mapping modeling from features to label sets while also taking into account the learning requirements of scarce labels and the inference constraints of mutual influence between labels, thereby improving the overall classification performance of multi-label classification tasks for heterogeneous data.

[0058] The method described in this application has the following technical effects: First, by enhancing the feature construction step through label perception, the label distribution information of the nearest neighbors of the sample is explicitly encoded into an enhanced feature vector and concatenated with the original feature vector. This makes the input space of the previous learning simultaneously contain the original feature information and the local label discrimination information, which solves the problem of difficult feature-to-label mapping modeling in heterogeneous data scenarios and improves the discrimination ability of multi-label classification.

[0059] Second, by integrating and enhancing the antecedent construction steps, the enhanced feature vector is used as the input space for antecedent learning, so that the division of fuzzy rules is directly guided by the label information, making the resulting fuzzy rule division more discriminative, thereby better adapting to the differences in feature distribution under different data forms.

[0060] Third, through the consequent alignment step guided by the label structure, a label difference matrix is ​​constructed based on the statistical correlation coefficient between labels, and this matrix is ​​used as a structured alignment constraint term to act on the consequent parameter matrix. This allows labels with strong statistical correlation to converge in the parameter space, while labels with weak statistical correlation or mutual exclusivity remain distinguishable in the parameter space. Ultimately, this effectively alleviates inference conflicts caused by the dependency relationship between labels and improves the overall consistency of multi-label joint prediction.

[0061] Fourth, by integrating the data fitting term that minimizes the prediction error with the structured alignment constraint term into a unified objective function and performing joint optimization, the learning of the feature-label mapping relationship and the utilization of label structure information are mutually synergistic, thereby improving the overall accuracy and stability of multi-label classification of heterogeneous data.

[0062] Fifth, it improves the learning effect of scarce labels under imbalanced label distribution conditions, so that samples related to low-frequency labels can obtain clearer support in rule activation, thereby improving the overall classification performance under imbalanced conditions.

[0063] The fuzzy multi-label inference learning method for heterogeneous data proposed in this application is applicable to processing heterogeneous data of different data forms such as text, images, and audio. It is especially suitable for multi-label classification scenarios where the label distribution exhibits long-tail characteristics, there are complex co-occurrence or mutual exclusion relationships between labels, and there is a lack of scarce label samples.

[0064] Example 2: Based on Example 1, this example further defines the hybrid difference measurement method and the construction process of the label-aware enhanced feature vector in step S1.

[0065] In step S1, the at least two similarities fused by the hybrid difference measure include the Pearson correlation coefficient and cosine similarity. The construction process of this hybrid difference measure is as follows: S11, For sample instances and Centralization is performed to eliminate the effect of mean skew. Specifically, instances are centralized: , in , This represents the original feature dimension. Based on this, sample instances... and The Pearson correlation coefficient between them is defined as: .

[0066] S12. Simultaneously, calculate the sample instances respectively. and Cosine similarity: , in .

[0067] S13. Introduce a balancing weight to adjust the contributions of Pearson correlation coefficient and cosine similarity. The range is calculated using the following formula: Mixed differences : .

[0068] because and All values ​​are taken from Therefore, the above equations are normalized using the following mappings: and Mapping the difference between the two to Within the interval; further, in Under weighted fusion, The value of is also strictly limited to Within the range.

[0069] When there is a strong correlation between two instances and At the same time, obtaining a large value significantly reduces the corresponding difference term, thus leading to The similarity is relatively small. When the correlation between two instances is weak, the similarity decreases, and the normalized difference term increases accordingly, making the similarity smaller. The difference increases accordingly. Therefore, this difference function can measure the trend of change in the similarity between samples in a monotonically consistent manner.

[0070] It should be noted here that in practical applications, The optimal value can be determined by performing a small-scale grid search on the validation set (e.g., selecting from {0.1, 0.3, 0.5, 0.7, 0.9}) and based on the average precision metric. Alternatively, an adaptive strategy can be adopted based on the data characteristics: if the dimensions of the feature data are uniform and directionality is crucial for classification, then the weight of cosine similarity should be increased (i.e., Take the smaller value); if more attention is paid to the linear correlation trend between features, increase the weight of the Pearson correlation coefficient (i.e., (Take the larger value). Those skilled in the art should understand that the above parameter selection method is a conventional engineering practice in the field and does not affect the feasibility of the technical solution of this method.

[0071] In step S1, the mixture differences are calculated. Based on this, the label-aware enhanced feature vector The construction process is as follows: S14, Based on the aforementioned hybrid differences , for sample instances Select the M nearest neighbors with the smallest difference values ​​to form the nearest neighbor sample set. And exclude the sample itself. Specifically, let's assume... For the corresponding sorted index mapping, then the sample The M-nearest neighbor set is defined as: , To avoid the involvement of the sample's own information, the following conditions must be met when constructing the neighborhood: .

[0072] S15, for any nearest neighbor Assign weights that are positively correlated with similarity according to the following formula. : , This weight represents the ratio of neighbor samples to center samples. The smaller the mixture difference, the higher the similarity, and the greater the corresponding weight.

[0073] S16, Regarding the first Let each label dimension be represented by its set of positive neighbors. The set of negative neighbors is denoted as Specifically, regarding the first Each label is used to classify neighboring samples into two categories: positive and negative. , .

[0074] S17. Calculate the cumulative weight of the positive neighbor set and the cumulative weight of the negative neighbor set respectively using the following formulas: , ; The aforementioned scaling factor reflects the relative proportion of positive and negative neighbors in the overall neighborhood under a uniform weight scale, thus effectively measuring the distribution characteristics of label-related structures in the local sample space.

[0075] S18, Define Sample In the Label-aware enhancement features across the label dimension: ; This component represents the relative dominance of positive and negative neighbors in terms of weight, and can be regarded as the first... A description of the discriminative strength of each label in its local neighborhood.

[0076] S19. Combine the enhanced feature values ​​of all L label dimensions to form a sample. Label-aware enhanced feature vectors: .

[0077] Guided by label awareness, this vector will transform the sample Explicitly encoding the local neighborhood distribution information across each label dimension not only comprehensively measures the relative proportion of positive and negative neighbors in a weighted sense, but also incorporates the influence of label-related structures on instance discrimination characteristics. This enables the augmented representation to simultaneously possess sample similarity, local discriminability, and label association perception capabilities, providing a more discriminative input representation for subsequent antecedent modeling.

[0078] Example 3: Based on Embodiment 1 or 2, this embodiment further defines the specific implementation of step S2.

[0079] In step S2, the specific process of constructing the fusion enhancement antecedent input space is as follows: the original features are concatenated with the label-aware enhancement features to form a new feature vector. .

[0080] Furthermore, the new input vector Represented as: .

[0081] Further antecedent learning is performed in this enhanced feature space, thereby guiding the TSK rules to form a more discriminative fuzzy partition in a joint space that simultaneously contains feature information and label structure information. The antecedent learning is implemented using the TSK fuzzy system, and the fuzzy rules of the TSK fuzzy system are used to activate the fused enhanced feature vector, so as to map the fused enhanced feature vector to the corresponding fuzzy feature vector.

[0082] Based on this, it is possible to Mapped to the corresponding fuzzy feature vector , where K is the number of fuzzy rules. Correspondingly, it includes Enhancement matrix for each sample It can be mapped as a whole to a fuzzy feature matrix. ,in yes The fuzzy feature vector obtained through prior learning.

[0083] Based on the obtained fuzzy feature vector, for any antecedent input vector LAFEM in Predicted output on each label From its fuzzy feature vector With the corresponding consequent parameter vector Composed of linear combinations: , in Furthermore, for all labels, the preceding input vector Complete predicted output vector It can be represented as: , in, This is the global consequent parameter matrix to be learned.

[0084] Example 4: Based on Example 1, this example further defines the specific implementation of the label structure-guided successor alignment step in step S3.

[0085] In step S3, the specific process of consequent alignment guided by the tag structure is as follows: S31. Obtain the true label matrix of all training samples. Where N is the number of samples, and the value vector of each label is centered to obtain a centered label matrix. Specifically, let Indicates the first The tag is in The value vectors from each training sample are used to center each label vector: , , in This represents a column vector consisting entirely of 1s.

[0086] S32, based on Calculate the Pearson correlation coefficient between each pair of tags, and use this as the statistical correlation coefficient to construct the tag correlation matrix. : .

[0087] S33. Construct the label difference matrix based on the statistical correlation coefficient using the following formula. : , in For the whole vector, Indicates the space between tags Correlation matrix Let represent the label difference matrix. Therefore, we have: When two labels are highly correlated Smaller; when the label relevance is weak, Larger.

[0088] It should be noted here that, as a numerical stabilization strategy to prevent extreme cases, in actual calculations, the label difference matrix can be used... Add a very small positive perturbation term on the main diagonal ,like This perturbation term is only used to ensure the stability of numerical calculations and has no substantial impact on the model's learning performance.

[0089] S34. During training, a consequent relevance alignment mechanism based on label statistical structure is introduced. To improve the learning performance of consequent parameters, LAFEM utilizes label information to guide the learning of the global consequent parameter matrix R. Specifically, if two labels exhibit significantly different label distributions or weak co-occurrence statistical dependencies in the training data, their corresponding consequent parameters should maintain necessary distinctions in the parameter space to avoid unreasonable coupling of irrelevant labels during the mapping process; conversely, for label pairs with strong statistical correlation, their parameter similarity should not be excessively penalized. Based on the above ideas, the formalization of the structured alignment constraint term acting on the consequent parameter matrix is ​​as follows: , in , Let represent the consequent parameter vectors corresponding to the i-th and j-th columns of matrix R, i.e., the i-th and j-th labels, respectively. These are weighting coefficients determined by the label-side statistical structure, used to measure the strength of the difference between label i and label j. This represents the trace operation of a matrix. The objective term suppresses unreasonable coupling of irrelevant labels in the consequent parameter space as a whole, ensuring that the correlation structure of the consequent parameters is consistent with the statistical dependency structure of the label space. This improves the model's discriminative and generalization stability while maintaining data fitting ability.

[0090] Example 5: Based on Examples 1, 3, and 4, this example further defines the specific implementation of the joint optimization and prediction step described in step S4.

[0091] In step S4, the specific process of integrating the data fitting term that minimizes the error between the predicted label and the true label, along with the structured alignment constraint term, into a unified objective function is as follows: S41. First, to measure the overall mapping relationship between the fuzzy feature representation and the multi-label output, and to minimize the squared error between the predicted result and the true label, a data fitting term based on the Frobenius norm is introduced. For a single sample... Define the objective for optimizing the squared error of the predicted output: .

[0092] Furthermore, extending the single-sample optimization objective to the entire training set containing N samples, the predicted output for all samples can be represented as: The prediction error loss function of LAFEM can then be written as: .

[0093] This objective is equivalent to jointly minimizing the squared prediction errors across all samples and all label dimensions in the training set, thereby obtaining robust consequent mapping parameters in an overall sense.

[0094] S42. Simultaneously, to ensure that the correlation structure of the consequent parameters is consistent with the statistical correlation structure of the label space, a structural consistency alignment term is introduced, resulting in the basic optimization objective: , in To balance the hyperparameters, the weight of the label structure alignment term in the overall objective function is adjusted. The fuzzy feature matrix is ​​obtained by activating the fuzzy rules in the prior learning of the fused and enhanced feature vectors of all samples. It is the Frobenius norm. It is an L1 norm.

[0095] S43. Furthermore, to suppress redundant rule parameters and enhance the sparsity and generalization stability of the model, LAFEM introduces an L1 regularization term into the above objective function to perform sparse regularization on the global consequent parameter matrix R. Therefore, the final complete optimization objective of LAFEM can be formalized as: , in, To balance the hyperparameters.

[0096] S44. The unified objective function is optimized using a near-end gradient descent method combined with momentum extrapolation. Due to the existence of the L1 norm, the objective function is first decomposed into a differentiable smooth part and a non-smooth part containing the L1 norm: , The smooth portion is: .

[0097] because For a convex and continuously differentiable smooth function, taking its matrix derivative yields the gradient expression: .

[0098] Furthermore, there exists a constant. , making satisfy Continuity allows for the construction of locally quadratic surrogate functions that satisfy the upper bound property near the current iteration point: , , in To and An irrelevant constant. Therefore, the first... This update can be translated into a near-terminal issue: , , , in The above problem has a closed-form solution, and its corresponding proximal operator is an element-wise soft thresholding map: .

[0099] make The soft threshold operator is defined as: .

[0100] To further accelerate the convergence speed, a momentum extrapolation strategy is introduced to construct new search points. Specifically, in each iteration, intermediate variables are updated based on the solutions from the previous two iterations: , Where the sequence satisfy This is used to balance stability and convergence speed, and It is the first The result of the next iteration.

[0101] During the initialization phase, ignoring sparse terms, the objective function degenerates into a standard least squares problem, and its analytical solution can be used as the initial value: .

[0102] To ensure the stability of the solution, let: , in, It's a hyperparameter. It is an identity matrix.

[0103] After optimizing the consequent parameter matrix, for the new test sample The corresponding fuzzy feature representation can be obtained from the antecedent. ,Will The predicted value is obtained by multiplying it with the trained consequent parameter matrix as described in step S44, and then compared with the classification threshold. The predicted label vector. Example 6: Building upon Example 1, this example further defines the range of heterogeneous data types. The heterogeneous data includes feature representations from at least two data formats: text, images, and audio. This method uses a unified fuzzy multi-label inference learning framework at its core, employing a consistent modeling process and parameter learning strategy for sample representation, rule activation, and output inference under different data formats. This reduces dependence on specific data types, enabling the method to be applicable to multi-label classification tasks with a variety of heterogeneous data.

[0104] Experiment and Results Analysis: 1. Experimental details: 1.1 Dataset: This experiment was conducted on seven publicly available benchmark multi-label datasets to verify the effectiveness of the proposed method. Specific information about each benchmark dataset is shown in Table 1, covering data from multiple fields including music (CAL500, Emotions), images (Flags, Image, Scene), text (Rcv1s1), and biology (Yeast), fully demonstrating the characteristics of heterogeneous data.

[0105] Table 1: Detailed information on the seven benchmark multi-label datasets 1.2 Evaluation Indicators and Parameter Settings: This experiment uses four commonly used evaluation metrics to assess the method's performance: average accuracy (AP), ranking loss (RL), error rate (OE), and coverage (CV). In the experimental setup, the proposed method (LAFEM) is compared with eight other methods on seven publicly available benchmark multi-label datasets. The eight comparison methods are: CC, ML-TSKFS, 2SML, RLFSCL, SLOFS, EMC, MLBE-ICF, and SIDLE. All methods were trained and tested using five-fold cross-validation, and the relevant parameter configurations are shown in Table 2.

[0106] Table 2 Parameter Settings: 1.3 Comparison of experimental results: To evaluate the performance of LAFEM in multi-label learning, this experiment compared it with eight mainstream multi-label learning algorithms on seven public datasets. The experimental results are summarized in Table 3, where the bolded parts indicate the best performance under the corresponding metrics. Overall, LAFEM demonstrates strong competitiveness on most datasets, indicating that the proposed overall learning framework can achieve stable improvements across multiple datasets and evaluation metrics.

[0107]

[0108] 2. Ablation experiment analysis: To evaluate the contribution of label-aware augmented representations to the overall model, this section conducts ablation experiments on seven publicly available multi-label datasets. The experimental setup includes two comparative configurations: control group A and ablation group B. Control group A uses the complete LAFEM framework; while ablation group B removes the label-aware augmented feature construction module, performing antecedent modeling and parameter learning only based on the original feature space, with all other parameter configurations and training strategies remaining consistent. Experimental results are as follows: Figures 2 to 5 As shown, the bars represent the mean performance of group A and group B on different datasets, and the vertical line at the top of the bar represents the standard deviation of the performance.

[0109] Figure 2 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the AP index. Figure 3 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the RL index. Figure 4 This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the OE index. Figure 5This is an ablation experiment analysis diagram of the label-aware enhancement representation of this application on the CV index.

[0110] It can be observed that, across all datasets, control group A consistently outperforms ablation group B in all four evaluation metrics: AP, RL, OE, and CV. This indicates that relying solely on the original features is insufficient to fully express the implicit label-related structures between samples. However, by introducing label-aware augmented representations and integrating neighborhood relationships and label distribution information into the antecedent input space, it helps guide fuzzy rules to form more reasonable and discriminative divisions, thereby further improving multi-label prediction performance.

[0111] 3. Parameter sensitivity analysis: This application analyzes the sensitivity of LAFEM to neighborhood size M, fusion weight δ, number of rules K, and hyperparameters α and β on seven benchmark multi-label datasets. The parameter analysis ranges for M and δ are set to {5,15,25,35,45} and {0.1,0.3,0.5,0.7,0.9}, respectively, while the parameter analysis ranges for K, α, and β are {2,3,4,5,6,7,8,9,10}, {0.01,0.1,1,10,100}, and {0.01,0.1,1,10,100}, respectively. When analyzing one parameter, all other parameters are fixed to the configuration that achieves the optimal AP performance on the validation set.

[0112] Figure 6 This is a parametric analysis graph of the neighborhood size M on seven datasets in this application. Figure 7 This is a graph showing the parametric analysis of the fusion weight δ across seven datasets in this application. Figure 8 This is a graph showing the parametric analysis of the number of rules K on seven datasets in this application. Figure 9 This is a graph showing the parameter analysis of hyperparameter α on seven datasets in this application. Figure 10 This is a graph showing the parameter analysis of hyperparameter β on seven datasets in this application.

[0113] The results are as follows Figures 6 to 10As shown: (1) As M gradually increases from 5, the AP index on each dataset shows a steady increase and reaches its optimum in the range of {25, 35}; when the neighborhood size continues to increase, the performance improvement tends to saturate or even decreases slightly. (2) δ has a weak impact on LAFEM and the sensitivity curve is almost horizontal. Overall, LAFEM achieves the best performance when δ is in the range of {0.1, 0.3}. (3) With the change of K, the performance of LAFEM on the Image, Rcv1s1, Scene, and Yeast datasets is basically stable, while there are slight fluctuations on the CAL500, Emotions, and Flags datasets. Overall, LAFEM can achieve the best AP performance when K is in the range of 3 to 5. (4) As α increases, the AP performance on each dataset shows a significant downward trend. When α is in a small range of {0.01, 0.1}, it can maintain good and stable performance; while when α is further increased to {1, 10, 100}, the performance degrades significantly. (5) Experimental results show that when β is in the range of {0.01,0.1,1}, the model performance gradually improves and reaches a better level, while when β is in the range of {10,100}, the performance declines.

[0114] 4. Convergence analysis: To verify the convergence of the proposed method, this application conducted experimental analysis on the iterative process of LAFEM on seven publicly available multi-label datasets, including CAL500 and Emotions, with all parameters set to their optimal values ​​on the corresponding datasets.

[0115] Figure 11 This is a convergence analysis plot of this application on the CAL500 dataset. Figure 12 This is a convergence analysis graph of this application on the Emotions dataset. Figure 13 This is a convergence analysis graph of this application on the Flags dataset. Figure 14 This is a convergence analysis graph of this application on the Image dataset. Figure 15 This is a convergence analysis plot of this application on the Rcv1s1 dataset. Figure 16 This is a convergence analysis graph of this application on the Scene dataset. Figure 17 This is a convergence analysis graph of this application on the Yeast dataset.

[0116] Relevant results are as follows Figures 11 to 17 As shown in the figure, the vertical axis df represents the absolute value of the difference between two consecutive objective function values. The figure demonstrates that LAFEM converges within a finite number of rounds on all datasets, indicating that the model has good convergence performance.

[0117] 5. Statistical significance test: Based on the experimental results in Table 3, the Friedman test was used to statistically analyze the performance differences between LAFEM and eight other comparative methods on four evaluation indicators: AP, RL, OE, and CV. The null hypothesis was set as no significant differences among the methods on the above indicators. When the corresponding critical value is exceeded, the null hypothesis is rejected. Table 4 summarizes the test results of the nine methods on four indicators. It can be observed that each indicator corresponds to... The values ​​of all values ​​are greater than the critical value, indicating that the performance differences between LAFEM and the comparative method in terms of AP, RL, OE, and CV are statistically significant.

[0118]

[0119] Furthermore, to measure the significance level of the differences between LAFEM and other comparison methods, this section introduces the Bonferroni-Dunn follow-up test for analysis. Specifically, LAFEM is set as the control method, and the critical difference CD for its average ranking with other methods is defined as follows: ; Where p represents the number of methods and S represents the number of datasets. In this paper, we take... , At the significance level The corresponding critical value is Thus obtain .

[0120] Figure 18 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the AP index. Figure 19 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the RL index. Figure 20 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the OE index. Figure 21 This is the Bonferroni-Dunn test result of this application and eight comparative methods on the CV index.

[0121] Figures 18 to 21 The average ranking results of LAFEM and eight comparison methods on various datasets are presented, where a smaller average ranking value indicates better overall method performance. When the average ranking difference between LAFEM and a comparison method is less than one CD, it is considered that there is no statistically significant difference between the two; otherwise, the performance difference is considered statistically significant. Figures 18 to 21It is evident that LAFEM significantly outperforms CC, SLOFS, EMC, and MLBE-ICF in overall performance, while the differences with ML-TSKFS, RLFSCL, and SIDLE are not statistically significant. Further analysis of the average ranking results across the four evaluation metrics reveals that LAFEM has the lowest average ranking value, indicating its optimal overall performance.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A fuzzy multi-label inference learning method for heterogeneous data, characterized in that, It includes the following steps: Label-aware enhanced feature construction: For each sample instance to be processed, based on a hybrid difference measure that integrates at least two similarities, a preset number of nearest neighbor samples with the smallest difference from the sample instance are selected from the training set to form a nearest neighbor sample set, and the sample instance is assigned a weight according to the similarity between each nearest neighbor sample and the sample instance; for each label dimension, the nearest neighbor sample set is divided into a positive neighbor set and a negative neighbor set according to whether each sample in the nearest neighbor sample set has the label, and based on the difference between the cumulative weight value of the positive neighbor set and the cumulative weight value of the negative neighbor set, a label-aware enhanced feature vector that can characterize its local label discrimination characteristics is constructed for the sample instance; Fusion enhancement antecedent construction: The original feature vector of the sample instance is concatenated with the label-aware enhancement feature vector to form a fusion enhancement feature vector. The fusion enhancement feature vector is used as the input space for antecedent learning. The fusion enhancement feature vector is activated by fuzzy rules to obtain the corresponding fuzzy feature vector, so as to realize fuzzy rule partitioning guided by label information. Label structure-guided consequent alignment: Based on the true label distribution of all labels in the training set, the statistical correlation coefficient between each pair of labels is calculated, and a label difference matrix reflecting the degree of difference between labels is constructed based on the statistical correlation coefficient. During the consequent parameter learning process, a structured alignment constraint term constructed based on the label difference matrix is ​​introduced. By applying the structured alignment constraint term to the consequent parameter matrix, the consequent parameters corresponding to labels with strong statistical correlation are made closer in the parameter space, while the consequent parameters corresponding to labels with weak statistical correlation or mutual exclusivity are kept distinguishable in the parameter space. Joint optimization and prediction: The data fitting term that minimizes the error between the predicted label and the true label after the fuzzy feature vector is mapped by the consequent parameter matrix, and the structured alignment constraint term are integrated into a unified objective function. The trained consequent parameter matrix is ​​obtained by optimizing and solving this objective function. For a new sample, a multi-label prediction result is generated based on its fused enhanced feature vector and the trained consequent parameter matrix; the at least two similarities fused by the hybrid difference metric method include Pearson correlation coefficient and cosine similarity, and the construction process of the hybrid difference metric method is as follows: For sample instances and Centralization is performed to eliminate the effect of mean bias; Calculate the Pearson correlation coefficient between the two respectively. Similarity to cosine ; Introduce a balance weight to adjust the contributions of both. The range is calculated using the following formula: Mixed differences : 。 2. The fuzzy multi-label reasoning learning method for heterogeneous data as described in claim 1, characterized in that, The label-aware enhanced feature vector The construction process is as follows: Based on the aforementioned hybrid differences , for sample instances The nearest neighbors with the smallest differences are selected to form the nearest neighbor sample set. ; For any nearest neighbor Assign weights that are positively correlated with similarity according to the following formula. : ； For the Each label dimension will be used to define the sample instances. The set of positive neighbors is denoted as The set of negative neighbors is denoted as ; The cumulative weights of the positive neighbor set and the cumulative weights of the negative neighbor set are calculated using the following formulas: ， ； Definition of the first The enhanced feature values ​​on each label dimension are: ； Combine the enhanced feature values ​​of all L label dimensions to form a sample instance. The label-aware enhanced feature vector: 。 3. The fuzzy multi-label inference learning method for heterogeneous data as described in claim 1, characterized in that: Obtain the true label matrix Y of all training samples, and center the value vector of each label to obtain the centered label matrix. ; based on Calculate the Pearson correlation coefficient between each pair of tags, and use this as the statistical correlation coefficient to construct the tag correlation matrix. ; The label difference matrix is ​​constructed according to the statistical correlation coefficient using the following formula. ,in A column vector consisting entirely of 1s: 。 4. The fuzzy multi-label inference learning method for heterogeneous data as described in claim 3, characterized in that: The formalization of the structured alignment constraint term acting on the consequent parameter matrix is ​​as follows: ， in, Let be the global consequent parameter matrix to be learned. Represents the trace operation of a matrix.

5. The fuzzy multi-label inference learning method for heterogeneous data as described in claim 4, characterized in that: In the joint optimization and prediction steps, the unified objective function formed by integration is: ， in, The fuzzy feature matrix is ​​obtained by activating the fuzzy rules in the prior learning of the fused and enhanced feature vectors of all samples. It is the Frobenius norm. It is an L1 norm. and Hyperparameters are used to balance the weights of each component.

6. The fuzzy multi-label inference learning method for heterogeneous data according to claim 5, characterized in that, The unified objective function is optimized using a proximal gradient descent method combined with momentum extrapolation: Decompose the objective function into a differentiable smooth part. With the non-smooth part containing the L1 norm ; Utilizing the Lipschitz continuity of the smooth partial gradient, let its Lipschitz constant be... Construct a surrogate function at the current iteration point; In each iteration, a search point is constructed based on the momentum combination of the results of the previous two iterations: ， Where the sequence satisfy , This is the result of the t-th iteration; Calculate the intermediate variables for proximal gradient descent: ， in, The gradient of the smooth portion at the search point; By performing element-wise soft thresholding, the result of the (t+1)th iteration is obtained in closed-form solution. ; ， in, for The corresponding element, The threshold for soft thresholding operations.

7. The fuzzy multi-label reasoning learning method for heterogeneous data as described in claim 1, characterized in that, The heterogeneous data includes feature representations from at least two data formats: text, images, and audio.

8. The method as described in claim 1, characterized in that, The antecedent learning is implemented using the TSK fuzzy system. The fuzzy rules of the TSK fuzzy system are used to activate the fused enhanced feature vector, so as to map the fused enhanced feature vector into a corresponding fuzzy feature vector, which is used for the learning of consequent parameters and the final label prediction.

9. The fuzzy multi-label reasoning learning method for heterogeneous data as described in claim 6, characterized in that, The initial values ​​of the consequent parameter matrix before the first iteration. Determine as follows: ， in, It's a hyperparameter. It is an identity matrix.

Images