A Multidimensional Data Exploration Method Based on Joint Embedding of Samples and Labels

Abstract

This invention discloses a multidimensional data exploration method based on joint embedding of samples and labels. The method includes: encoding labels and samples to determine sample encoding and label encoding; performing self-supervised learning on the sample encoding and label encoding to determine sample embedding and label embedding; and calculating the reconstruction loss L. R , contrast loss L CL and cross-entropy loss L CE The overall objective function L is determined, and the embedded network is iteratively updated using the Adam algorithm; the clustering loss L is used to... kl and MSE loss L MSE The sample embeddings are reconstructed to obtain the reconstructed sample embeddings. Two-dimensional projection is performed on the reconstructed sample embeddings and label embeddings. Label copies are determined based on the sample coordinates and label coordinates, and the label copies and samples are correlated. The correlation between labels and samples is visualized through a multi-dimensional data exploration module. This invention enhances the manipulability of the embedding process and the interpretability of the projection results by establishing a joint embedding network model.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a multidimensional data exploration method based on joint embedding of samples and labels. Background Technology

[0002] Embedding techniques refer to learning low-dimensional numerical vector representations for high-dimensional data entities. The resulting vectors are called embeddings of that data entity, where the similarity between embeddings reflects the feature similarity of the high-dimensional data entities in the original space. Various embedding techniques have been proposed to meet the needs of different data types, covering numerical, word / text, graph / network, media data, and many others. These embedding techniques are widely used in various downstream tasks.

[0003] In the field of data visualization, embedding techniques can be used to visualize patterns in raw data by learning and projecting the embedded representations of samples. In a typical workflow, analysts need to perform the following operations: (1) First, use embedding techniques to learn the vector of each sample, which serves as the embedding of that sample. This step is the foundation of the entire analysis process because the sample embedding preserves the key information of the sample in the raw data. (2) Next, project the embeddings of all samples into a low-dimensional space. This operation can transform the originally complex and high-dimensional embeddings into a relatively low-dimensional space, thus facilitating subsequent analysis and visualization. (3) Finally, analysts need to explore each visual structure in the projection. In the low-dimensional projection space, each visual structure, such as clusters and outliers, can be regarded as a pattern. Analysts will select specific visual structures to determine the common features of the internal samples, which can explain what factors led to the formation of this visual structure.

[0004] However, the above workflow has flaws. First, most embedding networks learn the similarity of samples across all features when generating embeddings. While each feature contributes to the construction of the sample embedding, features with significant differences between samples contribute more and play a dominant role. We call the patterns associated with these significantly different features the dominant patterns. When projecting using these embeddings, the dominant features guide the visual structure to reflect more information from the dominant patterns. However, patterns also exist on features with less significant differences. These features have relatively low discriminative power between samples, but the associated patterns still have potential value for dataset analysis. When analysts are interested in these patterns, the dominant patterns will mask the patterns on these features, preventing them from forming clear visual structures, hindering further analysis and exploration, and limiting deeper exploration.

[0005] Secondly, embeddings are generally presented in vector form, which is still an abstract representation of the original data and lacks interpretability. To intuitively understand the relationships between the samples corresponding to the embeddings, we project the embeddings into a low-dimensional space for visualization. However, the visual structure presented by the projection only contains the sample element, making it difficult for analysts to directly determine which features the samples in the visual structure have that are the same or different. The interpretability of the projection remains insufficient. In this case, to obtain effective information about the visual structure, analysts must manually select each structure and analyze and compare all the features of each sample in it one by one. This process is very time-consuming. Summary of the Invention

[0006] In view of the above problems, the purpose of this invention is to provide a multidimensional data exploration method based on joint embedding of samples and labels. (1) This invention allows operators to predefine labels by arbitrarily dividing the range of attribute values, focusing the training of the embedding network on the target pattern, thereby guiding the network to output the corresponding embedding results and enhancing the manipulability of the embedding process. (2) This invention utilizes the embedding network to learn all three relationships between samples and labels and reflects them in the structure of low-dimensional projection, which can reduce the time cost for users in the analysis process and enhance the interpretability of the projection results. (3) The multidimensional data exploration system designed in this invention helps operators to complete the analysis process quickly and accurately. In the projection, samples are distributed in an orderly manner according to different labels. Samples with the same or similar labels are "grouped" together to form clusters; while samples with different labels have obvious boundaries, which is excellent in terms of visualization effect.

[0007] The first aspect of this invention provides a multidimensional data exploration method based on joint embedding of samples and labels, comprising:

[0008] Obtain labels and samples;

[0009] The label and the sample are encoded to obtain the sample code s. i and label encoding l j ;

[0010] The sample is encoded by an autoencoder. i and label encoding l j Perform self-supervised learning and output sample embeddings. and tag embedding

[0011] The sample is embedded by the decoder. Perform reconstruction and calculate the reconstruction loss L. R ;

[0012] The sample is embedded using a classifier. and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ;

[0013] Calculate the reconstruction loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L, and the Adam algorithm is used to iteratively update the embedded network.

[0014] Through clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings.

[0015] Embedding of the reconstructed sample and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample.

[0016] The multidimensional data exploration module visualizes the relationship between labels and samples.

[0017] In this solution, obtaining the tag includes:

[0018] By setting lower boundary, step size and upper boundary, the range of attribute values ​​is discretized to determine multiple attribute value intervals;

[0019] Set filters for each attribute value range and create tags that contain the corresponding filters.

[0020] In this scheme, the label and the sample are encoded to obtain the sample code s. i and label encoding l j ,include:

[0021] Determine the sample code s based on the original features of the sample. i ;

[0022] The tag is randomly initialized with an encoding to determine the tag code. j .

[0023] In this scheme, the sample embedding is performed through a decoder. Perform reconstruction and calculate the reconstruction loss L. R ,include:

[0024] Sample embedding via decoder Reconstruct the sample and embed it. Reconstructed into corresponding feature codes si Given vectors of the same length, determine the reconstructed sample embedding s' i ;

[0025] By reconstructing the loss L R Calculate feature encodings s i and the reconstructed sample embeddings s' i Differences;

[0026]

[0027] In this scheme, the sample embedding is performed using a classifier. and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ,include:

[0028] Computing sample embeddings and tag embedding The association degree is used to predict the label of the sample and compare it with the original data to determine whether the association between the sample and the label has been learned correctly.

[0029] By comparing the loss L CL Embedding of the sample and tag embedding The association is performed, and labels that are associated with the sample are identified as positive labels, while labels that are not associated are identified as negative labels.

[0030]

[0031] Where N is the number of samples, M is the number of labels; A is the set of labels for the i-th sample, and u represents a label in label set A. It is all tag embedding, It is the embedding of the tag u. It is about finding the inner product of two embeddings;

[0032] According to label l j With sample s i Analyze the correlation to determine the cross-entropy coefficient y. j ;

[0033] When label l j With sample s i When associated, y j =1; tag l j With sample s i When there is no association, y j =0;

[0034] According to the cross-entropy coefficient y j Calculate the cross-entropy loss LCE ;

[0035]

[0036] Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings, and s() is a software function.

[0037] In this scheme, the calculation of reconstruction loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L. The Adam algorithm is then used to iteratively update the embedded network, including:

[0038] Reconstructing loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these values ​​serves as the overall objective function L for the embedding generation process;

[0039] Before training begins, the parameters of each component are randomly initialized.

[0040] During training, the Adam algorithm and the overall objective function L are used to iteratively update the network. The update ends when the embedded network satisfies the minimum loss function.

[0041] In this scheme, the clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings. include:

[0042] Embedding samples Input to clustering loss L kl A fully connected network outputs the reconstructed sample embeddings.

[0043] The clustering loss L kl Expressed as a formula:

[0044]

[0045] Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings, and s() is the softmax function used for normalization;

[0046] Loss L through MSE MSE For the reconstructed sample embedding Process

[0047]

[0048] Where N is the number of samples, The sample embeddings are obtained from the joint embedding network. Represents the reconstructed sample embedding;

[0049] Embedding of the reconstructed samples Embedded with related tags Strengthen the association.

[0050] In this scheme, the embedding of the reconstructed sample and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample, including:

[0051] The label embedding and sample embedding are projected onto a two-dimensional plane using the t-SNE method, and the label coordinates and sample coordinates of the two-dimensional projection are obtained respectively.

[0052] The sample coordinates are divided by a clustering recognition algorithm to determine the number of clusters formed by the samples in the projection space;

[0053] Filter labels associated with multiple clusters;

[0054] The label is split into one or more label copies based on the number of associated clusters, and the one or more label copies are assigned to each associated cluster.

[0055] This plan also includes:

[0056] The weighted average of the coordinates of all samples in the cluster is used to determine the two-dimensional coordinates of the label copy corresponding to the cluster.

[0057] In this solution, the visualization of the relationship between labels and samples through the multidimensional data exploration module includes:

[0058] Set upper and lower limits and step parameters through the label creator and create labels. Display the label list in the form of a bar chart, showing the number of associated samples for each label.

[0059] The sample and label are displayed by embedding projection using an embedding projector;

[0060] The pattern interpreter displays the sample features in each visual structure.

[0061] This invention discloses a multidimensional data exploration method based on joint embedding of samples and labels. The method includes: encoding labels and samples to determine sample encoding and label encoding; performing self-supervised learning on the sample encoding and label encoding to determine sample embedding and label embedding; and calculating the reconstruction loss L. R , contrast loss L CL and cross-entropy loss L CE The overall objective function L is determined, and the embedded network is iteratively updated using the Adam algorithm; the clustering loss L is used to... kl and MSE loss L MSE The sample embeddings are reconstructed to obtain the reconstructed sample embeddings. Two-dimensional projection is performed on the reconstructed sample embeddings and label embeddings. Label copies are determined based on the sample coordinates and label coordinates, and the label copies and samples are correlated. The correlation between labels and samples is visualized through a multi-dimensional data exploration module. This invention enhances the manipulability of the embedding process and the interpretability of the projection results by establishing a joint embedding network model. Attached Figure Description

[0062] Figure 1 The flowchart of a multidimensional data exploration method based on joint embedding of samples and labels provided by the present invention is shown;

[0063] Figure 2 A flowchart of the tag acquisition method provided by the present invention is shown;

[0064] Figure 3 The sample code s provided by the present invention is shown. i and label encoding l j Flowchart of the acquisition method;

[0065] Figure 4 A schematic diagram of the embedded network structure provided by the present invention is shown;

[0066] Figure 5 A schematic diagram of the embedding conversion process and tag splitting process provided by the present invention is shown;

[0067] Figure 6 This diagram illustrates the tag splitting process when the tag provided by the present invention is associated with samples in multiple clusters;

[0068] Figure 7 A schematic diagram of the multidimensional data exploration module provided by the present invention is shown;

[0069] Figure 8 A schematic diagram of the projection layout based on attribute axis arrangement provided by the present invention is shown. Detailed Implementation

[0070] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0071] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0072] Figure 1 The flowchart illustrates a multidimensional data exploration method based on joint embedding of samples and labels provided by the present invention.

[0073] like Figure 1 As shown, this invention discloses a multidimensional data exploration method based on joint embedding of samples and labels, comprising:

[0074] S102, Obtain labels and samples;

[0075] S104, Encode the label and sample to obtain the sample code s i and label encoding l j ;

[0076] S106, the sample is encoded by an autoencoder. i and label encoding l j Perform self-supervised learning and output sample embeddings. and tag embedding

[0077] S108, sample embedding via decoder Perform reconstruction and calculate the reconstruction loss L. R ;

[0078] S110, embedding samples using a classifier and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ;

[0079] S112, Calculate the reconstruction loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L, and the Adam algorithm is used to iteratively update the embedded network.

[0080] S114, using clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings.

[0081] S116, embedding the reconstructed sample and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample.

[0082] S118 uses a multidimensional data exploration module to visualize the relationship between labels and samples.

[0083] According to an embodiment of the present invention, a joint embedding network model is established, and multi-dimensional data exploration of joint embedding of samples and labels is carried out through three aspects: generating usable labels, using the network to jointly learn the embedding of samples and preset labels, and enhancing the embedding mode.

[0084] First, label generation is performed by discretizing the attribute value range into attribute value intervals. A filter is set for each attribute value interval, and a label with that filter is created. Thus, each label corresponds to an attribute value interval. The samples were collected by those skilled in the art based on data exploration needs. Then, embedding network learning is performed, such as... Figure 4 As shown, the embedding network integrates two autoencoders, a sample decoder, and a classifier. Each component is implemented as a fully connected network with two hidden layers, where each neuron is connected to all neurons in the previous layer. After the labels and samples are encoded, the sample encodings are... i and label encoding l j The samples are processed separately by two autoencoders. The autoencoders learn and output the corresponding sample embeddings through self-supervised learning. and tag embedding The network fixes the learned embedding length at 2048 and uses a reconstruction loss L... R , contrast loss L CL and cross-entropy loss L CEThe sum of these parameters serves as the overall objective function L for the embedding generation process. Before training begins, the parameters of each component are randomly initialized, and the network is iteratively updated using Adam and the overall objective function L during training, gradually guiding the network towards minimizing the loss function. Subsequently, the sample embeddings and label embeddings output by the embedding network are adjusted through embedding pattern enhancement to make the visual structure in the projection clearer. A fully connected network with clustering loss is used to reconstruct the sample embeddings, narrowing the distance between related sample embeddings and label embeddings, making the boundaries of the visual structure more distinct in the dimensionality-reduced projection, forming highly independent clusters. Labels are assigned to associated clusters. When a label is associated with multiple clusters, the label is split into one or more label copies based on the number of associated clusters, and these one or more label copies are assigned to each associated cluster. Finally, the association management of labels and samples is visualized through the three components of the multidimensional data exploration module: the label creator, the embedding projector, and the pattern interpreter. In the projection, the samples are distributed in an orderly manner according to their different labels. Samples with the same or similar labels are "grouped" together to form clusters; while samples with different labels have clear boundaries, which is excellent in terms of visualization effect.

[0085] Figure 2 A flowchart of the tag acquisition method provided by the present invention is shown.

[0086] like Figure 2 As shown, according to an embodiment of the present invention, obtaining a tag includes:

[0087] S202, by setting the lower limit boundary, step size and upper limit boundary, the range of attribute values ​​is discretized to determine multiple attribute value intervals;

[0088] S204 sets filters for each attribute value range and creates tags containing the corresponding filters.

[0089] It should be noted that the attribute value range is discretized into attribute value intervals based on the system's preset interval division rules. Numerical attributes are continuous, and the value intervals are mainly divided according to natural division and statistical conventions to create multiple labels. Categorical attributes are inherently discrete and do not require segmentation; each category is directly used as an interval. By setting a filter for each interval and creating a label with that filter, each label corresponds to an attribute value interval. The attribute value range refers to which values ​​these attributes can take, or which values ​​are allowed. The specific values ​​within the attribute value range are set by the system.

[0090] Figure 3 The sample code s provided by the present invention is shown. i and label encoding l j A flowchart for obtaining the method.

[0091] like Figure 3 As shown, according to an embodiment of the present invention, the label and the sample are encoded to obtain the sample code s. i and label encoding l j ,include:

[0092] S302, Determine the sample code s based on the original features of the sample. i ;

[0093] S304, Perform random initialization encoding on the tag to determine the tag code l j .

[0094] It should be noted that, for each sample, the original features of each sample (i.e., the values ​​of the sample on each attribute) are used as its encoding, denoted as sample encoding s. i For each tag, the code for each tag is randomly initialized, denoted as tag code l. j .

[0095] According to an embodiment of the present invention, the sample embedding is performed by a decoder. Perform reconstruction and calculate the reconstruction loss L. R ,include:

[0096] Sample embedding via decoder Reconstruct the sample and embed it. Reconstructed into corresponding feature codes s i Given vectors of the same length, determine the reconstructed sample embedding s' i ;

[0097] By reconstructing the loss L R Calculate feature encodings s i and the reconstructed sample embeddings s' i Differences;

[0098]

[0099] It should be noted that the decoder will reconstruct the sample embeddings to ensure that the sample encoder can effectively capture the feature information of the original samples. The decoder will embed the samples... As input, embed the sample Reconstructed to its original feature encodings i For vectors of the same length, the reconstructed sample embeddings are represented by s' i This is expressed as, and the loss L is reconstructed. R Calculate the difference between the original feature encoding and the reconstructed sample embedding.

[0100] According to embodiments of the present invention, sample embedding is performed using a classifier. and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ,include:

[0101] Computing sample embeddings and tag embedding The association degree is used to predict the label of the sample and compare it with the original data to determine whether the association between the sample and the label has been learned correctly.

[0102] By comparing the loss L CL Sample embedding and tag embedding The association is performed, and labels that are associated with the sample are identified as positive labels, while labels that are not associated are identified as negative labels.

[0103]

[0104] Where N is the number of samples, M is the number of labels; A is the set of labels for the i-th sample, and u represents a label in label set A. It is all tag embedding, It is the embedding of the tag u. It is about finding the inner product of two embeddings;

[0105] According to label l j With sample s i Analyze the correlation to determine the cross-entropy coefficient y. j ;

[0106] When label l j With sample s i When associated, y j =1; tag l j With sample s i When there is no association, y j =0;

[0107] Based on the cross-entropy coefficient y j Calculate the cross-entropy loss L CE ;

[0108]

[0109] Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings, and s() is a software function.

[0110] It should be noted that the classifier predicts the label of a sample by calculating the correlation between the sample embedding and the label embedding, and compares this prediction with the original data to determine whether the association between the sample and the label has been correctly learned. This is achieved by using contrastive loss Li. CL To align sample embeddings and tag embedding The association is defined as follows: Each sample is considered an anchor point, and labels associated with that sample are considered positive labels (i.e., positive samples in contrastive learning), while unrelated labels are considered negative labels (i.e., negative samples in contrastive learning).

[0111] The embedded network further incorporates cross-entropy loss L CE Assume y j It represents the tag l j With sample s i The cross-entropy coefficient of association, when j∈A, label l j With sample s i Related, y j =1; otherwise, there is no correlation, y j =0.

[0112] According to an embodiment of the present invention, the reconstruction loss L is calculated. R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L. The Adam algorithm is then used to iteratively update the embedded network, including:

[0113] Reconstructing loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these values ​​serves as the overall objective function L for the embedding generation process;

[0114] Before training begins, the parameters of each component are randomly initialized.

[0115] During training, the Adam algorithm and the overall objective function L are used to iteratively update the network. The update ends when the embedded network satisfies the minimum loss function.

[0116] It should be noted that the embedding network uses the sum of these three losses as the overall objective function L for the embedding generation process. Before training begins, the parameters of each component are randomly initialized, and during training, Adam and L are used to iteratively update the network, causing the network to gradually evolve towards minimizing the loss function.

[0117] According to an embodiment of the present invention, through clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings. include:

[0118] Embedding samples Input to clustering loss L kl A fully connected network outputs the reconstructed sample embeddings.

[0119] Clustering loss L kl Expressed as a formula:

[0120]

[0121] Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings, and s() is the softmax function used for normalization;

[0122] Loss L through MSE MSE For the reconstructed sample embedding Process

[0123]

[0124] Where N is the number of samples, The sample embeddings are obtained from the joint embedding network. Represents the reconstructed sample embedding;

[0125] For the reconstructed sample embedding Embedded with related tags Strengthen the association.

[0126] It should be noted that, as Figure 5 As shown in Figure (ab), the embedding transformation process is enhanced by adjusting the sample embeddings and label embeddings output by the embedding network to make the visual structure in the projection clearer. The basic idea is to operate in the embedding space and strengthen the association between the sample embeddings and the associated label embeddings (i.e., increase the inner product of the two embeddings in the embedding space, making the embeddings closer), thereby forming a more compact cluster in the projection.

[0127] Construct a fully connected network with clustering loss. The input to this network is the sample embedding E. i s The output is the reconstructed sample embedding E. i s′ During this process, the label embeddings remain unchanged, providing a stable target for adjusting the sample embeddings. The network also includes a reconstruction loss function, the MSE loss function, which is calculated by... MSE This ensures that the reconstructed sample embeddings can still retain the original important features.

[0128] By embedding samples Reconstruction can shorten the distance between the interrelated sample embeddings and label embeddings, making the boundaries of their visual structures more obvious in the dimensionality-reduced projection, and forming highly independent clusters.

[0129] According to an embodiment of the present invention, the reconstructed sample embedding and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample, including:

[0130] The label embedding and sample embedding are projected onto a two-dimensional plane using the t-SNE method, and the label coordinates and sample coordinates of the two-dimensional projection are obtained respectively.

[0131] The sample coordinates are divided by a clustering recognition algorithm to determine the number of clusters formed by the samples in the projection space;

[0132] Filter labels associated with multiple clusters;

[0133] The label is split into one or more label copies based on the number of associated clusters, and the one or more label copies are assigned to each associated cluster.

[0134] It should be noted that because labels may be associated with samples in multiple clusters simultaneously, their positions during projection may appear outside the clusters, forming outliers and reducing projection interpretability. Therefore, this method introduces a label splitting process. The label splitting process is as follows: Figure 5 As shown in (cd), the basic idea is: (1) Find the label associated with multiple clusters. By finding all samples associated with the label and using a clustering recognition algorithm to divide the samples, it is determined whether the sample is divided into multiple clusters in the projection space, thereby determining whether the label is associated with multiple clusters; (2) Split each label into multiple copies; (3) Assign the copies to each cluster associated with the label. This process ensures that the projection of all label copies appears in the cluster, improving the interpretability of the projection.

[0135] like Figure 6 As shown, this process first uses the t-SNE method to project the embedding onto a two-dimensional plane, obtaining the label coordinates and sample coordinates in the two-dimensional projection. Then, for each label, the DBSCAN algorithm is used to identify the sample clusters associated with that label in the projection, and a label copy is created for each identified cluster. Next, the weighted average of the sample coordinates within the cluster is calculated as the two-dimensional coordinates of the corresponding label copy. Finally, the label copy replaces the original label and is added to the label set.

[0136] According to an embodiment of the present invention, it further includes:

[0137] The weighted average of the coordinates of all samples in the cluster is used to determine the two-dimensional coordinates of the label copy corresponding to the cluster.

[0138] It should be noted that the weight of each sample coordinate is determined using methods such as monotonic distance weighting and density weighting. For example, when determining the weight of each sample coordinate using monotonic distance weighting, the weight is inversely proportional to the distance from the sample coordinate to the cluster center; the closer the distance, the higher the weight. The two-dimensional coordinates of the label copy corresponding to the cluster are determined by multiplying each sample coordinate in the cluster with its corresponding weight, summing the results, and finally dividing by the sum of all weights.

[0139] According to an embodiment of the present invention, a multidimensional data exploration module is used to visualize the relationship between labels and samples, including:

[0140] Set upper and lower limits and step parameters through the label creator and create labels. Display the label list in the form of a bar chart, showing the number of associated samples for each label.

[0141] The sample and label are displayed by embedding projection using an embedding projector;

[0142] The pattern interpreter displays the sample features in each visual structure.

[0143] It should be noted that this method designs a multidimensional data exploration module to integrate the above models and complete the data exploration workflow. The workflow of the multidimensional data exploration module mainly consists of three steps: (1) creating labels for analysis; (2) learning the embeddings of labels and samples and jointly projecting them; (3) selecting a visualization structure from the projection to explore patterns. Figure 7 As shown, the multidimensional data exploration module implements its workflow through three components, including Figure 7 (a) tag creator, Figure 7 Embedded projectors in (b) and (c), and Figure 7 (d) The pattern interpreter.

[0144] The label creator displays a vertically arranged list of labels, presented as a bar chart. The color of the label bar chart indicates the attribute restricted by the corresponding filter; labels with the same attribute have the same color. The length of the bar chart indicates the number of samples with that label (i.e., those satisfying the corresponding attribute value range). Additionally, the label creator provides a control that allows those skilled in the art to set three parameters: lower bound, step size, and upper bound, to discretize the attribute value range and create multiple labels. The upper and lower bounds are divided into several labels based on the step size. Additionally, a label is generated for each of the attribute's minimum value to the lower bound and the maximum value to the upper bound in the original data.

[0145] The embedding projector displays embedded projections of samples and labels. In the projection, samples are represented by gray dots, and labels by stars, with the star color matching the corresponding attribute color in the label creator. The embedding projector has two layouts: one is a projection layout obtained by directly reducing the dimensionality of the embeddings, and the other is a layout based on attribute axes. For example... Figure 8 The image shows a projection layout based on an attribute axis. The operator can select an attribute from the list in the upper right corner of the embedded projector, and an attribute axis will appear in the center of the view. Labels related to that attribute are evenly arranged along the axis. Based on the labels contained within a cluster, clusters are arranged along both sides of the axis and connected to their associated labels.

[0146] The pattern interpreter helps operators identify and compare features of visual structures. The pattern interpreter contains multiple cards, each focusing on the interpretation of a specific visual structure. Each card consists of multiple horizontal heatmaps, with each bar representing the features of a sample within the visual structure on a particular attribute. Based on the discreteness of each attribute in the label creator, the heatmap is divided into several regions. The color of each heatmap corresponds to an attribute; the intensity of the color indicates the distribution of values ​​for a particular attribute among the samples in the visual structure, with darker colors indicating a greater number of samples matching that value range.

[0147] All information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, terms such as "labels," "samples," and "attribute value ranges" mentioned in this disclosure were obtained with full authorization.

[0148] This invention discloses a multidimensional data exploration method based on joint embedding of samples and labels. The method includes: encoding labels and samples to determine sample encoding and label encoding; performing self-supervised learning on the sample encoding and label encoding to determine sample embedding and label embedding; and calculating the reconstruction loss L. R , contrast loss L CL and cross-entropy loss L CE The overall objective function L is determined, and the embedded network is iteratively updated using the Adam algorithm; the clustering loss L is used to... kl and MSE loss L MSEThe sample embeddings are reconstructed to obtain the reconstructed sample embeddings. Two-dimensional projection is performed on the reconstructed sample embeddings and label embeddings. Label copies are determined based on the sample coordinates and label coordinates, and the label copies and samples are correlated. The correlation between labels and samples is visualized through a multi-dimensional data exploration module. This invention enhances the manipulability of the embedding process and the interpretability of the projection results by establishing a joint embedding network model.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0150] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0152] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A multidimensional data exploration method based on joint embedding of samples and labels, characterized in that, include: Obtain labels and samples; The label and the sample are encoded to obtain the sample code s. i and label encoding l j ; The sample is encoded by an autoencoder. i and label encoding l j Perform self-supervised learning and output sample embeddings. and tag embedding The sample is embedded by the decoder. Perform reconstruction and calculate the reconstruction loss L. R ; The sample is embedded using a classifier. and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ; Calculate the reconstruction loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L, and the Adam algorithm is used to iteratively update the embedded network. Through clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings. Embedding of the reconstructed sample and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample. The multidimensional data exploration module visualizes the relationship between labels and samples. The sample embedding is performed using a classifier. and tag embedding Perform association learning and calculate the contrastive loss L. CL and cross-entropy loss L CE ,include: Computing sample embeddings and tag embedding The association degree is used to predict the label of the sample and compare it with the original data to determine whether the association between the sample and the label has been learned correctly. By comparing the loss L CL Embedding of the sample and tag embedding The association is performed, and labels that are associated with the sample are identified as positive labels, while labels that are not associated are identified as negative labels. Where N is the number of samples, M is the number of labels; A is the set of labels for the i-th sample, and u represents a label in label set A. It is all tag embedding, It is the embedding of the tag u. It is about finding the inner product of two embeddings; According to label l j With sample s i Analyze the correlation to determine the cross-entropy coefficient y. j ; When label l j With sample s i When associated, y j =1; tag l j With sample s i When there is no association, y j =0; According to the cross-entropy coefficient y j Calculate the cross-entropy loss L CE ; Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings; s() is a software function. The clustering loss L kl and MSE loss L MSE Sample embedding Reconstruction is performed to obtain the reconstructed sample embeddings. include: Embedding samples Input to clustering loss L kl A fully connected network outputs the reconstructed sample embeddings. The clustering loss L kl Expressed as a formula: Where N is the number of samples and M is the number of labels. It calculates the inner product of two embeddings, and s() is the softmax function used for normalization; Loss L through MSE MSE Embedding of reconstructed samples Process Where N is the number of samples, The sample embeddings are obtained from the joint embedding network. Represents the reconstructed sample embedding; Embedding of the reconstructed samples Embedded with related tags Strengthen associations. include: Set upper and lower limits and step parameters through the label creator and create labels. Display the label list in the form of a bar chart, showing the number of associated samples for each label. The sample and label are displayed by embedding projection using an embedding projector; The pattern interpreter displays the sample features in each visual structure.

2. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 1, characterized in that, The acquisition of tags includes: By setting lower boundary, step size and upper boundary, the range of attribute values ​​is discretized to determine multiple attribute value intervals; Set filters for each attribute value range and create tags containing the corresponding filters.

3. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 1, characterized in that, The label and the sample are encoded to obtain the sample code s. i and label encoding l j ,include: Determine the sample code s based on the original features of the sample. i ; The tag is randomly initialized with an encoding to determine the tag code. j .

4. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 1, characterized in that, The sample embedding is performed through a decoder. Perform reconstruction and calculate the reconstruction loss L. R ,include: Sample embedding via decoder Reconstruct the sample and embed it. Reconstructed into corresponding feature codes s i Given vectors of the same length, determine the reconstructed sample embedding s' i ; By reconstructing the loss L R Calculate feature codes s i and the reconstructed sample embeddings s' i Differences; 5. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 1, characterized in that, The calculation of reconstruction loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these factors determines the overall objective function L. The Adam algorithm is then used to iteratively update the embedded network, including: Reconstructing loss L R , contrast loss L CL and cross-entropy loss L CE The sum of these values ​​serves as the overall objective function L for the embedding generation process; Before training begins, the parameters of each component are randomly initialized. During training, the Adam algorithm and the overall objective function L are used to iteratively update the network. The update ends when the embedded network satisfies the minimum loss function.

6. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 1, characterized in that, The embedded data of the reconstructed sample and tag embedding Perform two-dimensional projection, determine the label copy based on the projected sample coordinates and label coordinates, and correlate the label copy and the sample, including: The label embedding and sample embedding are projected onto a two-dimensional plane using the t-SNE method, and the label coordinates and sample coordinates of the two-dimensional projection are obtained respectively. The sample coordinates are divided by a clustering recognition algorithm to determine the number of clusters formed by the samples in the projection space; Filter labels associated with multiple clusters; The label is split into one or more label copies based on the number of associated clusters, and the one or more label copies are assigned to each associated cluster.

7. The multidimensional data exploration method based on joint embedding of samples and labels according to claim 6, characterized in that, Also includes: The weighted average of the coordinates of all samples in the cluster is used to determine the two-dimensional coordinates of the label copy corresponding to the cluster.

Images