Multi-source data classification method and device based on multi-channel contrast, equipment and medium

Abstract

The application discloses a multi-source data classification method and device based on multi-channel contrast, equipment and medium, relate to multi-source data classification technical field.The method contains: obtaining multi-source data set.According to the attribute matrix of the multi-source data set, the sparse adjacency matrix of the similarity is constructed.The attribute matrix and the sparse adjacency matrix are encoded to obtain the initial embedding of each data source.Fusion network fuses each embedding to obtain consistent embedding, and then generates consensus label.Through the decoder, the initial embedding is reconstructed, and the reconstruction loss between the reconstructed features and the attribute matrix is minimized.A multi-channel contrast mechanism is introduced to compare and align the embedding representations output by the graph convolution layer of the same level of different convolution channels.Pseudo labels of each embedding representation are obtained, and global consistency alignment is performed using the consensus label and each pseudo label.According to the consensus label and the real label, the supervision optimization of the multi-source data classification task is completed through the cross entropy loss function, and a trained model is obtained.

Description

Technical Field

[0001] This invention relates to the field of multi-source data classification technology, and more specifically, to a multi-source data classification method, apparatus, device, and medium based on multi-channel comparison. Background Technology

[0002] Multi-source data can characterize object features from multiple complementary perspectives, providing more comprehensive and richer information than a single data source, thus mitigating the impact of missing features or biased single-perspective data to some extent. For example, in industrial fault detection, the operating status of mechanical equipment can be comprehensively characterized by multiple source data, such as vibration signals, acoustic signals, and thermal imaging data. In anomaly detection of financial accounts, transaction behavior often simultaneously includes different information such as transfer amounts, time series patterns, user social relationships, and device fingerprints.

[0003] The core value of multi-source data classification lies in enhancing the model's ability to distinguish sample categories by fusing information from multiple sources, thereby improving classification accuracy and robustness. Multi-source data classification methods have been proposed to alleviate the difficulty of obtaining large-scale, high-quality labeled samples. Existing methods attempt to improve classification accuracy by combining unlabeled samples with multi-source data structure information using a limited number of labeled samples.

[0004] However, existing technologies still face several challenges in implementation. First, semantic differences and structural inconsistencies exist among multi-source data, which hinders the discriminative power and robustness of the fusion process. Second, semi-supervised learning relies on pseudo-label generation, and in multi-source data scenarios, pseudo-labels generated from different sources may be biased. Without a unified alignment mechanism, the model's learning direction will shift, reducing classification accuracy.

[0005] Existing methods generally fall short in pseudo-label alignment. Some studies have attempted to introduce contrastive learning to enhance the consistency of representations across different source data, but these methods typically only focus on the consistency constraints of the last layer of features in the encoder, making it difficult to capture multi-level semantic information and resulting in insufficient cross-source data representation. Furthermore, in terms of pseudo-label alignment, existing methods often lack a globally unified alignment mechanism. Pseudo-labels generated between different source data are prone to bias and conflict, weakening the supervisory value of unlabeled data and even introducing noise, further limiting the model's discriminative ability. Summary of the Invention

[0006] The present invention provides a method, apparatus, device and medium for classifying multi-source data based on multi-channel comparison, in order to improve at least one of the above-mentioned technical problems.

[0007] Firstly, the present invention provides a multi-source data classification method based on multi-channel comparison, which includes steps S1 to S4.

[0008] S1. Obtain multi-source datasets.

[0009] S2. Based on the attribute matrix of the multi-source dataset, use the KNN algorithm to calculate the similarity between data points of each data source and construct a sparse adjacency matrix of similarity.

[0010] S3. Input the attribute matrix and sparse adjacency matrix into the graph neural network for encoding to obtain the initial embeddings of each data source. The graph neural network sets one convolutional channel for each data source, and each graph convolutional channel is connected in series with multiple graph convolutional layers.

[0011] S4. A consistent embedding is obtained by fusing various embeddings through a fusion network, and then a consensus label is generated through a classifier.

[0012] The training phase also includes steps S5 to S8.

[0013] S5. Reconstruct the initial embedding using a decoder and minimize the reconstruction loss between the reconstructed feature and attribute matrices.

[0014] S6. A multi-channel comparison mechanism is introduced to compare and align the embedded representations of graph convolutional layer outputs at the same level with different convolutional channels.

[0015] S7. Obtain the pseudo-labels of each embedded representation, and use the consensus label and each pseudo-label to perform global consistency alignment.

[0016] S8. Based on the consensus label and the true label, supervised optimization of the multi-source data classification task is performed using the cross-entropy loss function to obtain a trained model. The trained model includes the graph neural network, the fusion network, and the classifier.

[0017] Secondly, the present invention provides a multi-source data classification device based on multi-channel comparison, which includes a data acquisition module, a preprocessing module, an encoding module and a classification module.

[0018] The data acquisition module is used to acquire multi-source datasets.

[0019] The preprocessing module is used to calculate the similarity between data points from each data source using the KNN algorithm based on the attribute matrix of the multi-source dataset, and to construct a sparse adjacency matrix of similarity.

[0020] The encoding module is used to input the attribute matrix and sparse adjacency matrix into the graph neural network for encoding, thereby obtaining the initial embeddings of each data source. The graph neural network sets up a convolutional channel for each data source, and multiple graph convolutional layers are set in series for each graph convolutional channel.

[0021] The classification module is used to merge various embeddings through a fusion network to obtain a consistent embedding, and then generate a consensus label through a classifier.

[0022] The training phase also includes a first loss module, a second loss module, a third loss module, and a fourth loss module.

[0023] The first loss module is used to reconstruct the initial embedding through the decoder and minimize the reconstruction loss between the reconstructed feature and attribute matrices.

[0024] The second loss module is used to introduce a multi-channel contrast mechanism to compare and align the embedded representations of graph convolutional layer outputs at the same level with different convolutional channels.

[0025] The third loss module is used to obtain pseudo-labels for each embedded representation and to perform global consistency alignment using consensus labels and each pseudo-label.

[0026] The fourth loss module is used to perform supervised optimization of the multi-source data classification task based on the consensus label and the true label using the cross-entropy loss function, thereby obtaining a trained model. The trained model includes the graph neural network, the fusion network, and the classifier.

[0027] Thirdly, the present invention provides a multi-source data classification device based on multi-channel comparison, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a multi-source data classification method based on multi-channel comparison as described in any paragraph of the first aspect.

[0028] Fourthly, the present invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a multi-source data classification method based on multi-channel comparison as described in any paragraph of the first aspect.

[0029] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0030] The method of this invention extracts features from multi-source data based on graph neural networks, designs a hierarchical comparison mechanism to enhance cross-source data consistency at multiple semantic levels, and adopts a global pseudo-label alignment strategy to introduce pseudo-supervision information on unlabeled samples, aligning global pseudo-labels with pseudo-labels specific to the source data, enabling the model to obtain stronger generalization ability in semi-supervised learning scenarios and alleviating the performance bottleneck caused by label scarcity.

[0031] Furthermore, this invention improves the accuracy of multi-source data classification models by introducing a cross-entropy loss function to continuously optimize and update model parameters. This method enables accurate classification of unlabeled samples, thereby significantly improving overall classification performance while maintaining training efficiency. Attached Figure Description

[0032] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a network structure diagram of a multi-source data classification method.

[0034] Figure 2 This is a network structure diagram of the hierarchical comparison module.

[0035] Figure 3 This is a diagram of the network structure of a graph neural network. Detailed Implementation

[0036] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0037] Example 1, please refer to Figures 1 to 3 The first embodiment of this invention provides a multi-source data classification method based on multi-channel comparison to address the shortcomings of existing methods in terms of multi-level semantic alignment and global consistency of pseudo-labels. The method can be executed by a multi-source data classification device based on multi-channel comparison (hereinafter referred to as: classification device). Specifically, it is executed by one or more processors in the classification device to implement steps S1 to S4.

[0038] S1. Obtain multi-source datasets.

[0039] A multi-source dataset is a collection of data consisting of multiple sets of different features or descriptions, such as industrial fault detection data or abnormal financial account data. Each set of feature descriptions is called a source dataset. A multi-source dataset contains data from multiple sources, each providing an observation or understanding of a different aspect of the data object. These source datasets can be of different types, such as numerical data, text data, and image data, and may originate from different data sources or be obtained through different feature extraction methods. For example: Image feature extraction data: Multiple features can be extracted from an image, such as LBP (Local Binary Pattern), SIFT (Scale Invariant Feature Transform), HOG (Histogram of Oriented Gradients), and GIST (Global Feature Descriptor). These features can be considered as different source datasets for the image. Multilingual text data: An article can be described in multiple languages, such as English, Chinese, and Japanese. The text content in each language can be considered as a source dataset for the article.

[0040] In this embodiment, the multi-source dataset is industrial fault detection data. In industrial manufacturing and equipment operation and maintenance scenarios, the machine operating status often requires joint analysis of multi-source sensor data. For example, vibration data sources (frequency / time domain characteristics from accelerometers), acoustic data sources (sound spectrum characteristics generated during equipment operation), and infrared thermal imaging data sources (images of equipment surface temperature distribution). Temperature data sources and humidity data sources are also included.

[0041] The system parses node-level feature information from vibration data sources to obtain an attribute matrix. Specifically, it extracts statistical parameters (such as mean, variance, kurtosis, and crest factor) through time-domain analysis and extracts dominant frequency components and energy distribution characteristics through frequency-domain analysis (Fast Fourier Transform, FFT). Furthermore, it employs time-frequency analysis methods (such as Short-Time Fourier Transform, STFT, or wavelet transform) to capture instantaneous frequency changes in non-stationary signals, forming a multi-dimensional feature vector. Finally, the features of each sampling point or time window are combined into a high-dimensional attribute matrix, reflecting the local dynamic characteristics of the vibration signal.

[0042] To extract node-level feature information from acoustic data sources and obtain attribute matrices, the acoustic data needs to be automatically processed by deep learning models (such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) to extract temporal features. For example, audio signals can be converted into Mel-frequency cepstral coefficients (MFCCs) or spectrograms, which are then input into the model to extract hidden layer features and form acoustic feature vectors.

[0043] To extract node-level feature information from infrared thermal imaging data sources and obtain an attribute matrix, the grayscale image is converted into a temperature matrix using a calibration formula. Then, spatial feature extraction methods (such as gradient calculation, region segmentation, and thermal anomaly detection) are used to identify temperature distribution patterns. For example, node-level thermal features are extracted by combining thermophysical property analysis (such as temperature abrupt change zones and thermal anomaly regions) and correlated with spatial location information to construct a thermal imaging attribute matrix.

[0044] To extract node-level feature information from temperature data sources and obtain the attribute matrix, the temperature data needs to be analyzed in conjunction with time series analysis and spatial distribution feature extraction. The rate of temperature change, trend term, and periodic fluctuations (such as Fourier harmonic analysis) are calculated using a sliding window. Simultaneously, spatial interpolation methods (such as Kriging interpolation) are used to generate the spatial distribution matrix of the temperature field, extracting local temperature gradients and hotspot region features.

[0045] To extract node-level feature information from humidity data sources and obtain an attribute matrix, the following steps are employed: Humidity data processing methods are similar, requiring time series analysis (such as moving averages and outlier detection) to extract humidity change trends and outliers, and combining this with spatial distribution characteristics (such as humidity gradients and regional correlations) to construct the attribute matrix. For multi-source heterogeneous data (such as the coupling relationship between temperature and humidity), multivariate analysis methods (such as principal component analysis, PCA) can be introduced to extract comprehensive features, ensuring the comprehensiveness and consistency of node-level features.

[0046] In an optional embodiment, the multi-source dataset is abnormal financial account data. In a financial scenario, a user's transaction behavior can be characterized from multiple source data. For example, transaction record source data (numerical features such as transaction amount, time interval, and geographical location), relationship network source data (social relationship graph between the user and other accounts), device fingerprint source data (information such as the terminal device used, IP address, and operating system), and behavioral pattern source data (frequency of transaction operations, time period distribution, etc.).

[0047] The process involves parsing node-level feature information from transaction record source data to obtain an attribute matrix. Specifically, transaction record data includes fields such as time, amount, merchant information, and transaction location. First, the raw data is cleaned to remove outliers (such as negative amount transactions) and missing values. Then, features are extracted through the following steps: Statistical features: Calculating transaction frequency, average amount, maximum / minimum transaction amount, and transaction time interval distribution for nodes (such as users or merchants). Time series features: Extracting the time periodicity of transactions (such as daily transaction count), trend terms (such as transaction amount growth / decline rate), and outlier detection results based on a sliding window. Categorical feature encoding: One-hot encoding or embedding vectors is used for discrete attributes such as merchant category and transaction location to capture the correlation between merchant type and transaction risk. Association rule mining: Transaction patterns (such as "high-frequency small-amount transactions + nighttime transactions") are extracted through frequent itemset analysis (such as the Apriori algorithm) and converted into Boolean features. Finally, the above features are combined into node-level feature vectors to form the transaction record attribute matrix.

[0048] This process involves parsing node-level feature information from source data of a relational network to obtain an attribute matrix. Specifically, relational network data describes the interaction relationships between nodes (e.g., user-user transaction networks, device-user association graphs). Feature extraction combines network topology and graph neural network techniques: Topological features: calculating node degree centrality, proximity centrality, betweenness centrality, and community affiliation. Dynamic network analysis: extracting dynamic evolution features of node relationships from timestamp-annotated interaction data (e.g., frequency of new connections, relationship duration). Graph Neural Networks (GNNs): using Graph Convolutional Networks (GCNs) or Graph Attention Networks (GATs) to embed nodes in a low dimension, capturing implicit associations in the network (e.g., dense connection patterns in fraud rings). Path features: generating path sequences between nodes through random walks and statistically analyzing path lengths and high-frequency path patterns. These features are then integrated into node-level feature vectors to construct the relational network attribute matrix.

[0049] The device fingerprint source data is parsed to extract node-level feature information and obtain an attribute matrix. Specifically, device fingerprint data includes attributes such as device model, operating system, IP address, and geographical location. Feature extraction needs to consider both static attributes and dynamic behavior: Static attribute encoding: embedding or hashing categorical variables such as device model and operating system version, and mapping IP addresses to geographical locations (e.g., latitude and longitude coordinates). Behavioral pattern features: statistically analyzing device login frequency, login time period distribution (e.g., weekdays vs. non-weekdays), and geographical location jumps (e.g., logins across regions within a short period). Anomaly detection indicators: calculating the deviation of device behavior (e.g., KS test results with historical behavior) and device fingerprint consistency (e.g., the same device changing IP or geographical location multiple times). Temporal correlation features: identifying typical usage scenarios of the device (e.g., "office equipment" or "public WiFi device") through time series clustering (e.g., DBSCAN). Finally, the static and dynamic features are combined into a device fingerprint feature vector, forming a device fingerprint attribute matrix.

[0050] The behavioral pattern data is analyzed to extract node-level feature information and obtain an attribute matrix. Specifically, the behavioral pattern data covers user operation sequences (such as clickstream, page dwell time), access frequency, etc. Feature extraction requires combining time series analysis and pattern recognition: Time series features: Extract operation frequency, dwell time variance, and operation interval distribution (such as Poisson distribution fitting parameters) through a sliding window. Pattern recognition: Extract long-term dependency features of user behavior (such as the "search → browse → purchase" pattern) using LSTM or Transformer models, or identify common behavior categories (such as "high-frequency jump users") through clustering. Contextual features: Construct multi-dimensional behavioral features by combining contextual information such as timestamps, device type, and geographic location (such as "nighttime use + mobile device + high jump rate"). Abnormal behavior features: Detect behavioral deviation through statistical methods or isolated forests and convert it into risk scoring features. Integrate the above features into a behavioral pattern feature vector to construct a behavioral pattern attribute matrix.

[0051] It should be noted that during the inference phase, the multi-source dataset refers to the dataset from which the model is to be identified. During the training phase, the multi-source dataset contains multiple samples, each containing data from several different sources. After preprocessing, the dataset includes both labeled and unlabeled samples. The unlabeled samples are only used to calculate the reconstruction loss and during the comparison and alignment phases to train and fine-tune the model.

[0052] S2. Based on the attribute matrix of the multi-source dataset, use the KNN algorithm to calculate the similarity between data points of each data source and construct a sparse adjacency matrix of similarity.

[0053] This embodiment processes multi-source datasets using a graph construction method. First, it obtains the attribute matrices of each source data and the corresponding adjacency relationships, then generates a sparse adjacency matrix for a specific source data. Preferably, step S2 specifically includes steps S21 to S23.

[0054] S21. Parse the node-level feature information from each source data to obtain the attribute matrix.

[0055] An attribute matrix is ​​used to represent the attributes of data points in a multi-source dataset. The attribute matrix is ​​represented as follows: ,in, Represented as the first Attribute matrix of source data, Indicates the first The attribute vector of each data point This indicates the number of data points.

[0056] S21. Based on the attribute matrix, use the KNN algorithm to calculate the similarity between data points from each data source.

[0057] S22. Based on the preset nearest neighbor number k, select the k most similar data points for each data point to establish an undirected graph connection.

[0058] S23. By filtering with edge weight thresholds or retaining proportions, the undirected graph connections are sparsified to obtain the adjacency graph.

[0059] Pruning strategies are particularly important when constructing adjacency graphs, especially with large-scale datasets, as they can effectively reduce computational and storage burdens while preserving the core structural information of the graph. Common pruning methods include distance-based threshold filtering, edge removal based on importance, local connectivity adjustment, and pruning methods designed according to algorithm requirements.

[0060] S24. Perform a normalization transformation on the adjacency graph to obtain a sparse adjacency matrix that meets the input requirements of a graph neural network.

[0061] The multi-source dataset preprocessed in steps S21 to S24 contains attribute matrices obtained from multiple sources or using different feature extraction techniques, corresponding to multiple sparse adjacency matrices, as well as the class label of each sample, which facilitates the training and validation of the subsequent model.

[0062] Based on the above embodiments, in an optional embodiment of the present invention, the present invention performs classification tasks by constructing a multi-source data classification model based on multi-channel contrast and graph neural networks.

[0063] The multi-source data classification model in the inference phase includes: graph neural networks, fusion networks, and a classifier. Specifically, the graph neural network assigns a convolutional channel to each data source, and each graph convolutional channel is connected in series with multiple graph convolutional layers. In the training phase, the multi-source data classification model also includes: a decoder, a hierarchical comparison module, and a global pseudo-label alignment module.

[0064] The graph convolutional channel employs a three-layer graph convolutional neural network to extract node features layer by layer, using the ReLU function as the activation function and linear layers to transform the feature dimensions, making it suitable for multi-channel contrast mechanisms. Preferably, the graph convolutional channel includes a first convolutional layer, a first ReLU activation function layer, a first linear layer, a second ReLU activation function layer, a second convolutional layer, a third ReLU activation function layer, a second linear layer, a fourth ReLU activation function layer, a third convolutional layer, and a third linear layer connected in sequence.

[0065] The detailed parameters of the graph neural network are as follows: The input dimension of the first convolutional layer is the original feature dimension, and the output dimension is 512. The original feature dimension is determined by the number of features extracted in step S1. The input and output dimensions of the first linear layer are both 512. The input dimension of the second convolutional layer is 512, and the output dimension is 256. The input and output dimensions of the second linear layer are both 256. The input dimension of the third convolutional layer is 256, and the output dimension is 128. The input and output dimensions of the third linear layer are both 128. The original feature dimension is determined by the attribute matrix.

[0066] The graph neural network of this invention is constructed as a multi-layer feature extraction architecture. The first layer of graph convolution performs preliminary encoding and compression of the original features. Subsequently, multiple linear layers and activation functions are used to achieve nonlinear transformation and hierarchical feature extraction, ultimately generating a highly condensed 128-dimensional embedding representation. This dimensional design balances feature representation capability and computational efficiency, effectively controlling model complexity and overfitting risk while ensuring representational power.

[0067] S3. Input the attribute matrix and sparse adjacency matrix into the graph neural network for encoding to obtain the initial embeddings of each data source. Specifically, such as... Figures 1 to 3 As shown, the attribute matrix and sparse adjacency matrix of each source data are input into the graph neural network for feature extraction to obtain the initial embedding.

[0068] The first one obtained after preprocessing Attribute matrix of source data and sparse adjacency matrix The inputs are then fed into the graph neural network. and In a graph neural network, the signal goes through three convolutional layers, four ReLU activation functions, and three linear layers, ultimately extracting the signal specific to the first convolutional layer. Initial embedding of source data .

[0069] .

[0070] In the formula, Indicates the first The graph neural network with source data of the nth source data. Layer embedding representation, For activation function, For the normalized degree matrix, For identity matrix, For the first The sparse adjacency matrix of the source data, for Learnable weights for The bias value.

[0071] S4. A consistent embedding is obtained by fusing various embeddings through a fusion network, and then a consensus label is generated through a classifier. For example... Figure 1 As shown, step S4 specifically includes steps S41 to S42.

[0072] S41. The embedded representations of each source data after encoding are fused to obtain a consistent embedding. Preferably, the source data-specific embeddings obtained in step S2 are summed by mean to obtain a consistent embedding.

[0073] .

[0074] In the formula, For consistent embedding. For the first Initial embedding of source data. This represents the number of data sources.

[0075] S42. Use a classifier to process the consistent embedding. Classify and obtain consensus labels.

[0076] During training, a multi-source data classification method based on multi-channel contrast also includes steps S5 to S8.

[0077] S5. Reconstruct the initial embedding using a decoder and minimize the reconstruction loss between the reconstructed feature and attribute matrices.

[0078] S51. The initial embedded input is input to the decoder to obtain the reconstructed features. Specifically, the decoder uses conventional techniques in this field, which are not limited or further explained in this invention.

[0079] S52. Based on the attribute matrix and the reconstructed features, the mean squared error (MSE) loss function is used to optimize the difference between the reconstructed features and the initial features.

[0080] The formula for the mean squared error loss function is:

[0081] .

[0082] In the formula, Represents the mean squared error loss function, Represented as the first Attribute matrix of source data, Indicates the first Reconstruction characteristics of individual source data Indicates the total number of source data, Represents the norm.

[0083] S6. A multi-channel comparison mechanism is introduced to compare and align the embedded representations of graph convolutional layer outputs at the same level with different convolutional channels, thereby improving the representational power and hierarchical consistency of the embedding space.

[0084] like Figure 2 As shown, the hierarchical comparison module obtains the embedded representations of the outputs of each linear layer from the graph neural network and constructs the comparison relationship between cross-source data at the same layer. Although the loss function of the hierarchical comparison module is only calculated during the training phase, it indirectly enhances the consistency of the model's representation and its adaptability to downstream tasks during the inference phase by improving the consistency and robustness of cross-layer embeddings.

[0085] Preferred, such as Figures 1 to 2 As shown, step S6 specifically includes steps S61 to S63.

[0086] S61. For the embedded representations of the convolutional layer outputs of graph neural networks at the same level for different source data, calculate the cosine similarity to measure the similarity between layers across source data and construct a similarity matrix.

[0087] The formula for constructing the similarity matrix is:

[0088] .

[0089] In the formula, According to and The constructed similarity matrix, Indicates the first The graph neural network with source data of the nth source data. The output of the first layer Embedded representation of data points, Indicates the first The graph neural network with source data of the nth source data. The output of the first layer Embedded representation of data points, This represents the L2 norm.

[0090] S62. The embedded representations of the convolutional layer outputs of the same level of the graph neural network for different source data are used to obtain pseudo-labels through the classification head, and a pseudo-label matrix is ​​constructed for each source data. Here, the classification head is the classification performed using a classifier.

[0091] .

[0092] In the formula, Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The output of the first layer The data point and the The pseudo-label matrix elements constructed from the pseudo-labels corresponding to each data point. Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The first layer of cross-source data embedding in the output of the layer A predicted class probability vector for each data point. Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The first layer of cross-source data embedding in the output of the layer A predicted class probability vector for each data point. This is the similarity threshold.

[0093] The diagonal elements represent the class probabilities of the same data points from different source data. For other elements, connections are constructed in the pseudo-label matrix when the similarity is higher than a threshold τ.

[0094] S63. Align the similarity matrix and pseudo-label matrix through contrastive learning so that the embedded feature structure is constrained by class information.

[0095] .

[0096] In the formula, To compare the losses. Let be the number of combinations of any two source data. This is the sequence number of the combination. Indicates the first The graph neural network with combinations of source data. The output of the first layer The pseudo-label matrix elements of the data points. This represents the log function. This represents the natural exponential function. For the first The graph neural network with combinations of source data. The output of the first layer The data point and the Similarity matrix of data points, This indicates the number of data points. Indicates the first The graph neural network with combinations of source data. The output of the first layer The data point and the The pseudo-label matrix elements constructed from the pseudo-labels corresponding to each data point.

[0097] S7. Obtain the pseudo-labels of each embedded representation, and perform global consistency alignment using the consensus label and each pseudo-label. Preferably, step S7 includes steps S71 to S72.

[0098] S71. Use a classifier to classify the first... Initial embedding of source data Classify the data and obtain pseudo-labels for each source data.

[0099] S72. Global pseudo-label alignment is performed through comparative learning to correct potential biases in pseudo-labels generated from single-source data.

[0100] .

[0101] In the formula, Global pseudo-label alignment loss. This indicates the number of data points. Represents the cosine distance. This represents the global pseudo-label representation of the j-th data point. This indicates that the j-th data point is in the... The predicted class probability vector in the source data. Indicates the first The data point at the th th The predicted class probability vector in the source data. This refers to temperature hyperparameters.

[0102] S8. Based on the consensus label and the true label, supervised optimization of the multi-source data classification task is performed using the cross-entropy loss function to obtain a trained model. The trained model includes the graph neural network, the fusion network, and the classifier.

[0103] The formula for the cross-entropy loss function is:

[0104] .

[0105] In the formula, This is the cross-entropy loss function. This indicates the number of data points. Indicates the first The data point at the th th The true labels for each category. Indicates consistent embedding After passing through the classifier, at the... The predicted probability of each category. This indicates the total number of categories.

[0106] Specifically, during training, the preprocessed and labeled multi-source data training set is input into the constructed multi-source data classification model for model training. The parameters of the model are adjusted, and the model with adjusted parameters is saved to obtain the trained multi-source data classification model, thus obtaining a model with good classification performance.

[0107] In this embodiment, a publicly available multi-source dataset is used for model testing, and all experiments are conducted under the same hardware environment. The training dataset is a publicly available dataset containing 2688 samples and image data with four feature transformations, and four source datasets with dimensions of 512, 59, 864, and 254, respectively. This embodiment uses a graph construction method during training, generating edges for each node by calculating the nearest neighbor of each node using Euclidean distance. Three common evaluation metrics are used to evaluate the model testing results: classification accuracy (ACC), normalized mutual information (NMI), and adjusted Rand index (ARI). When the model evaluation result reaches a set threshold, the trained multi-source data classification model is obtained. Newly acquired multi-source datasets are preprocessed and input into the trained multi-source data classification model to obtain the multi-source data classification results.

[0108] The multi-source data classification method based on multi-channel comparison of the present invention includes a preprocessing stage, a pre-training stage, a training stage, and an inference / application stage. Step S6 is the inference / application stage.

[0109] In the preprocessing stage: The input is a multi-source raw dataset. Node-level feature information is extracted to generate an attribute matrix. Simultaneously, the similarity between data points is calculated using the KNN algorithm, and an adjacency graph is constructed using the nearest neighbor strategy. Then, the graph connections are pruned and sparsified by applying edge weight thresholds or retention ratios. Finally, after normalization, a standard adjacency matrix form conforming to the computational requirements of graph neural networks is output.

[0110] In the pre-training phase: a graph neural network is used to encode the attribute matrix and sparse adjacency matrix of each source data. Specifically, this is done through a graph neural network containing three convolutional layers and two ReLU activation functions to obtain an embedding representation specific to the source data. Subsequently, the features are reconstructed through a decoder, and the mean squared error (MSE) loss function is used to minimize the difference between the reconstructed features and the initial features, thus completing the initialization of the model parameters.

[0111] During the training phase: Hierarchical embedding comparison is performed between different layers of the graph neural network to improve representation ability and hierarchical consistency. First, the cosine similarity between the embeddings of each source data layer is calculated to construct a cross-source data similarity matrix. Simultaneously, a pseudo-label matrix is ​​generated for the embeddings of the same-layer source data using a classification head. Finally, the hierarchical consistency between the similarity matrix and the pseudo-label matrix is ​​compared to ensure that the predicted category information is consistent with the similarity relationship between samples, thereby improving the reliability of the embedding representation. Subsequently, the embeddings of each source data are fused to obtain a consistent embedding, and consensus labels are generated based on the consistent embedding. Finally, global consistency comparison learning is performed by minimizing the alignment loss between the global pseudo-label and each pseudo-label. Based on the consistent embedding, the classifier generates the predicted category probability, and cross-entropy loss is calculated in combination with the real labels for supervised optimization, ultimately achieving the multi-source data classification task.

[0112] In the inference / application phase: First, multi-source data to be classified is acquired and input into the trained model. The model outputs the predicted probabilities for each category and determines the target category, thus obtaining the final classification result.

[0113] For example, in industrial fault detection, the operating status of mechanical equipment can be comprehensively characterized by multiple source data, such as vibration signals, acoustic signals, and thermal imaging data. A trained model can be used to classify samples into categories such as "normal operation," "minor anomaly," and "serious fault," thereby achieving accurate classification of equipment status.

[0114] In the detection of abnormal financial accounts, transaction behavior often includes different information such as transfer amount, time series pattern, user social relationship and device fingerprint. A trained model can be used to classify transaction behavior into categories such as "normal transaction", "high-risk transaction" and "suspicious fraud", thereby preventing financial fraud.

[0115] When using a multi-source data classification method based on a multi-channel comparison mechanism to detect industrial faults, the fault categories are set according to the actual possible fault types, or directly set to two states: fault and healthy. When using the same multi-source data classification method to detect abnormal financial accounts, the detection categories can be set to two categories: normal and abnormal. Therefore, the classification results not only provide numerical predictions from the model, but also offer clear category labels at the application level that can directly support decision-making.

[0116] The method of this invention extracts features from multi-source data based on graph neural networks, designs a hierarchical comparison mechanism to enhance cross-source data consistency at multiple semantic levels, and adopts a global pseudo-label alignment strategy to introduce pseudo-supervision information on unlabeled samples, aligning global pseudo-labels with pseudo-labels specific to the source data, enabling the model to obtain stronger generalization ability in semi-supervised learning scenarios and alleviating the performance bottleneck caused by label scarcity.

[0117] Furthermore, this invention improves the accuracy of multi-source data classification models by introducing a cross-entropy loss function to continuously optimize and update model parameters. This method enables accurate classification of unlabeled samples, thereby significantly improving overall classification performance while maintaining training efficiency.

[0118] Example 2: The present invention provides a multi-source data classification device based on multi-channel comparison, which includes a data acquisition module, a preprocessing module, an encoding module and a classification module.

[0119] The data acquisition module is used to acquire multi-source datasets.

[0120] The preprocessing module is used to calculate the similarity between data points from each data source using the KNN algorithm based on the attribute matrix of the multi-source dataset, and to construct a sparse adjacency matrix of similarity.

[0121] The encoding module is used to input the attribute matrix and sparse adjacency matrix into the graph neural network for encoding, thereby obtaining the initial embeddings of each data source. The graph neural network sets up a convolutional channel for each data source, and multiple graph convolutional layers are set in series for each graph convolutional channel.

[0122] The classification module is used to merge various embeddings through a fusion network to obtain a consistent embedding, and then generate a consensus label through a classifier.

[0123] The training phase also includes a first loss module, a second loss module, a third loss module, and a fourth loss module.

[0124] The first loss module is used to reconstruct the initial embedding through the decoder and minimize the reconstruction loss between the reconstructed feature and attribute matrices.

[0125] The second loss module is used to introduce a multi-channel contrast mechanism to compare and align the embedded representations of graph convolutional layer outputs at the same level with different convolutional channels.

[0126] The third loss module is used to obtain pseudo-labels for each embedded representation and to perform global consistency alignment using consensus labels and each pseudo-label.

[0127] The fourth loss module is used to perform supervised optimization of the multi-source data classification task based on the consensus label and the true label using the cross-entropy loss function, thereby obtaining a trained model. The trained model includes the graph neural network, the fusion network, and the classifier.

[0128] Example 3: This invention provides a multi-source data classification device based on multi-channel comparison, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a multi-source data classification method based on multi-channel comparison as described in any paragraph of Example 1.

[0129] Example 4: This invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a multi-source data classification method based on multi-channel comparison as described in any paragraph of Example 1.

[0130] It is understood that the classified devices can be electronic devices with computing capabilities, such as portable laptops, desktop computers, servers, smartphones, or tablets.

[0131] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0132] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0134] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0136] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0137] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0138] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-source data classification method based on multi-channel comparison, characterized in that, Include: Obtain multi-source datasets; Based on the attribute matrix of the multi-source dataset, the KNN algorithm is used to calculate the similarity between data points from each data source, and a sparse adjacency matrix of similarity is constructed. The attribute matrix and sparse adjacency matrix are input into the graph neural network for encoding to obtain the initial embeddings of each data source; the graph neural network sets a convolutional channel for each data source, and each graph convolutional channel is connected in series with multiple graph convolutional layers; Consistent embeddings are obtained by fusing various embeddings through a fusion network, and then consensus labels are generated through a classifier; The training phase also includes: The initial embedding is reconstructed using a decoder, and the reconstruction loss between the reconstructed feature and attribute matrices is minimized. A multi-channel contrast mechanism is introduced to compare and align the embedded representations of graph convolutional layer outputs at the same level with different convolutional channels; Specifically, this includes: calculating cosine similarity between the embedded representations of the convolutional layer outputs of graph neural networks at the same level for different source data, in order to measure the similarity between layers across source data and construct a similarity matrix; In the formula, According to and The constructed similarity matrix, Indicates the first The graph neural network with source data of the nth source data. The output of the first layer Embedded representation of data points, Indicates the first The graph neural network with source data of the nth source data. The output of the first layer Embedded representation of data points, The L2 norm is represented; the embedding representation of the output of the convolutional layer of the graph neural network at the same level for different source data is obtained through the classification head to obtain pseudo-labels, and a pseudo-label matrix is ​​constructed for each source data. In the formula, Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The output of the first layer The data point and the The pseudo-label matrix elements constructed from the pseudo-labels corresponding to each data point; Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The first layer of cross-source data embedding in the output of the layer A predicted class probability vector for each data point; Indicates the first Individual source data and the first The source data is generated by the graph neural network's first... The first layer of cross-source data embedding in the output of the layer A predicted class probability vector for each data point; The similarity threshold is used; the similarity matrix and pseudo-label matrix are aligned through contrastive learning so that the embedded feature structure is constrained by class information. In the formula, To compare the losses; Let be the number of combinations of any two source data. The sequence number of the combination; Indicates the first The graph neural network with combinations of source data. The output of the first layer The pseudo-label matrix elements of each data point; Represents the log function; Represents the natural exponential function; For the first The graph neural network with combinations of source data. Similarity of layer outputs Indicates the number of data points; Indicates the first The graph neural network with combinations of source data. The output of the first layer The data point and the The pseudo-label matrix elements constructed from the pseudo-labels corresponding to each data point; Obtain the pseudo-labels of each embedded representation, and perform global consistency alignment using the consensus label and each pseudo-label; Based on the consensus label and the true label, supervised optimization of the multi-source data classification task is completed through the cross-entropy loss function to obtain a trained model; the trained model includes the graph neural network, the fusion network and the classifier.

2. The multi-source data classification method based on multi-channel comparison according to claim 1, characterized in that, The multi-source dataset consists of industrial fault detection data or abnormal financial account data; the industrial fault detection data includes vibration data source data, acoustic data source data, infrared thermal imaging data source data, temperature data source data, and humidity data source data; the abnormal financial account data includes transaction record source data, relationship network source data, equipment fingerprint source data, and behavioral pattern source data. During the inference phase, the multi-source dataset is the multi-source dataset to be identified; During the training phase, the multi-source dataset contains multiple samples, each containing data from several different sources. After preprocessing, the dataset includes labeled and unlabeled samples. The unlabeled samples are only used to calculate the reconstruction loss and in the comparison and alignment phases to train and adjust the model. The attribute matrix is ​​used to represent the attributes of data points in a multi-source dataset; the attribute matrix is ​​represented as follows: ,in, Represented as the first Attribute matrix of source data, Indicates the first The attribute vector of each data point This indicates the number of data points.

3. The multi-source data classification method based on multi-channel comparison according to claim 1, characterized in that, Based on the attribute matrix of the multi-source dataset, the KNN algorithm is used to calculate the similarity between data points from each data source, and a sparse adjacency matrix of similarity is constructed, specifically including: Analyze the node-level feature information from each source data to obtain the attribute matrix; Based on the attribute matrix, the KNN algorithm is used to calculate the similarity between data points from each data source. Based on the preset nearest neighbor number k, select the k most similar data points for each data point to establish an undirected graph connection; The undirected graph connections are sparsified by using edge weight threshold filtering or proportional retention to obtain the adjacency graph. Apply a normalization transformation to the adjacency graph to obtain a sparse adjacency matrix.

4. The multi-source data classification method based on multi-channel comparison according to claim 1, characterized in that, The attribute matrix and sparse adjacency matrix are encoded using a graph neural network to obtain the initial embeddings of each data source, specifically including: The attribute matrices and sparse adjacency matrices of each data source are input into a graph neural network for feature extraction to obtain the initial embedding. In the formula, For the first The graph neural network with source data of the nth source data. Layer embedding representation, For activation function, For the normalized degree matrix, For the first The sparse adjacency matrix of the source data, For identity matrix, for Learnable weights for The bias value; A consistent embedding is obtained by fusing various embedding representations through a fusion network, and then a consensus label is generated through a classifier, specifically including: The encoded embedding representations of each source data are fused to obtain a consistent embedding. In the formula, For consistent embedding; For the first Initial embedding of source data; The number of data sources; Use a classifier to classify consistent embeddings and obtain consensus labels.

5. The multi-source data classification method based on multi-channel comparison according to claim 1, characterized in that, The graph convolutional channel uses a three-layer graph convolutional neural network to extract node features layer by layer, uses the ReLU function as the activation function, and uses linear layers to complete the transformation of feature dimensions to suit multi-channel contrast mechanisms. The graph convolution channel includes a first graph convolutional layer, a first ReLU activation function layer, a first linear layer, a second ReLU activation function layer, a second graph convolutional layer, a third ReLU activation function layer, a second linear layer, a fourth ReLU activation function layer, a third graph convolutional layer, and a third linear layer connected in sequence. The input dimension of the convolutional layer in the first figure is the original feature dimension, and the output dimension is 512; where the original feature dimension is determined by the attribute matrix. The input and output dimensions of the first linear layer are both 512; The second convolutional layer has an input dimension of 512 and an output dimension of 256. The second linear layer has an input dimension of 256 and an output dimension of 256. The third convolutional layer has an input dimension of 256 and an output dimension of 128. The input and output dimensions of the third linear layer are both 128.

6. The multi-source data classification method based on multi-channel comparison according to claim 1, characterized in that, The initial embedding is reconstructed using a decoder, and the reconstruction loss between the reconstructed feature and attribute matrices is minimized. Specifically, this includes: The initial embedding representation is input into the decoder to obtain the reconstructed features; The mean squared error loss function is used to optimize the difference between the reconstructed features and the attribute matrix; In the formula, Represents the mean squared error loss function, Represented as the first Attribute matrix of source data, Indicates the first Reconstruction characteristics of individual source data Indicates the total number of source data, Represents the norm; Obtain the pseudo-labels for each embedded representation, and perform global consistency alignment using the consensus label and each pseudo-label. Specifically, this includes: Use a classifier to classify the first Initial embedding of source data Classify the data and obtain pseudo-labels for each source data; Global pseudo-label alignment is performed through contrastive learning to correct potential biases in pseudo-labels generated from single-source data. In the formula, Global pseudo-label alignment loss; Indicates the number of data points; Represents cosine distance; This represents the global pseudo-label representation of the j-th data point; This indicates that the j-th data point is in the... The predicted class probability vector in the source data; Indicates the first The data point at the th th The predicted class probability vector in the source data; This refers to temperature hyperparameters.

7. A multi-source data classification device based on multi-channel comparison, characterized in that, Suitable for performing a multi-source data classification method based on multi-channel comparison as described in any one of claims 1 to 6; the multi-source data classification device comprises: The data acquisition module is used to acquire multi-source datasets; The preprocessing module is used to calculate the similarity between data points from each data source using the KNN algorithm based on the attribute matrix of the multi-source dataset, and to construct a sparse adjacency matrix of similarity. The encoding module is used to input the attribute matrix and sparse adjacency matrix into the graph neural network for encoding to obtain the initial embedding of each data source; the graph neural network sets a convolutional channel for each data source, and each graph convolutional channel is connected in series with multiple graph convolutional layers; The classification module is used to fuse various embeddings through a fusion network to obtain a consistent embedding, and then generate consensus labels through a classifier; The training phase also includes: The first loss module is used to reconstruct the initial embedding through the decoder and minimize the reconstruction loss between the reconstructed feature and attribute matrices; The second loss module is used to introduce a multi-channel contrast mechanism to compare and align the embedded representations of the graph convolutional layer outputs at the same level with different convolutional channels; The third loss module is used to obtain pseudo-labels for each embedded representation and to perform global consistency alignment using consensus labels and each pseudo-label. The fourth loss module is used to perform supervised optimization of the multi-source data classification task based on consensus labels and true labels through the cross-entropy loss function, and obtain a trained model; the trained model includes the graph neural network, the fusion network and the classifier.

8. A multi-source data classification device based on multi-channel comparison, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a multi-source data classification method based on multi-channel comparison as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a multi-source data classification method based on multi-channel comparison as described in any one of claims 1 to 6.

Images