Depth metric learning method for multi-dimensional semantic object
Through the two-stage deep measurement learning method, combined with attention module and 5 nearest neighbor classification, the problem of failure to effectively utilize class variable dependencies in multi-dimensional semantic object classification is solved, the classification performance and generalization ability are improved, and the multi-dimensionality is achieved An effective measure of semantic object similarity.
Patent Information
- Application Number
- CN202411005624.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-19
AI Technical Summary
The existing multidimensional semantic object classification method fails to effectively utilize the potential dependencies between class variables, resulting in poor generalization performance, and the commonly used smoothing assumptions are corrupted in the multidimensional semantic space, making it difficult to measure the similarity of samples.
The two-stage deep metric learning method is adopted. The first stage calculates the unique similarity measure in each dimension and the common similarity measure across dimensions at the same time. The second stage uses the attention module to combine the hidden feature vector to learn the unique similarity measure of each dimension. And classify it through 5 nearest neighbor classification criteria.
The classification performance and generalization ability of the multi-dimensional semantic object classification model are improved, the similarity of marking spaces in different dimensions is effectively measured, and the discriminant ability is enhanced.
Smart Images

Figure CN118981690A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of pattern recognition and machine learning, and mainly relates to a deep metric learning method for multi-dimensional semantic objects. Background Art
[0002] Multidimensional semantic objects contain rich semantics in different dimensions. Each object is represented by a single sample and is associated with multiple class variables corresponding to different semantic spaces. For example, in animal classification, animals can be classified from the species dimension (which may be labeled as dog, cat, rabbit, etc.), the tail dimension (which may be labeled as long, short, tailless, etc.), and the fur dimension (which may be labeled as curly, straight, etc.). In fact, the development of multidimensional semantic object classification is closely related to various real-world applications, including computer vision, bioinformatics, text mining, etc.
[0003] An intuitive solution to the multi-dimensional semantic object classification problem is to transform the prediction task on all dimensions into independent multi-classification problems on each dimension's label space. However, this strategy fails to exploit the potential dependencies between class variables and may lead to suboptimal generalization performance. Another simple strategy is to treat each different label combination as a new class label and transform the multi-dimensional semantic object classification task into a single multi-classification problem with new class variables on the output space. However, this strategy may lead to severe overfitting due to the significant reduction in the number of samples per class and the fact that label combinations that do not appear in the training set will be ignored. In order to exploit class dependencies and overcome the shortcomings of the above strategies, existing multi-dimensional semantic object classification methods aim to transform the original classification problem through implicit methods or explicitly rely on the assumption that there are certain specific relational structures on the label space. Implicit transformation methods usually manipulate the feature space through feature augmentation or manipulate the label space through carefully designed label encodings. On the other hand, methods based on explicit relational structures assume specific structures, including pairwise relations, chain orders, and directed acyclic graphs.
[0004] Despite partial progress in multidimensional semantic object classification, a fundamental challenge remains. Specifically, the commonly used smoothness assumption suggests that similar samples should produce similar labels. However, this assumption is often violated by the properties of multidimensional semantic spaces. For example, consider the animal classification problem mentioned earlier: two dogs share the same category in the species dimension, which means that they should be similar to each other in the feature space. On the other hand, although both belong to dogs in the species dimension, these dogs may have different tail sizes due to belonging to different breeds, which indicates that they are different in the tail dimension. This dichotomy is a common problem in multidimensional semantic object classification and highlights a key need for further research: how to measure the similarity of multidimensional semantic object samples. Obviously, a unified measure for all dimensions is not sufficient for this purpose. Summary of the invention
[0005] The present invention is aimed at the problems existing in the prior art and provides a deep metric learning method for multi-dimensional semantic objects, which includes at least two learning stages; in the first learning stage: two groups of latent feature vectors are calculated simultaneously, one of which is used to learn the unique similarity metric in each dimension, and the other is used to learn the shared similarity metric across dimensions; in the second learning stage: the attention module is used to combine the two groups of latent feature vectors learned in the first learning stage to further learn the similarity metric specific to each dimension, consider the dimensional correlation information through the shared metric, output the latent feature representation exclusive to each dimension, and finally use the 5 nearest neighbor classification criterion to classify the multi-dimensional semantic objects. The method of the present invention further improves the classification performance and generalization ability of the multi-dimensional semantic object classification model.
[0006] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is: a deep metric learning method for multi-dimensional semantic objects, comprising at least two learning stages;
[0007] In the first learning phase: two sets of latent feature vectors are calculated simultaneously, one set is used to learn a unique similarity measure within each dimension, and the other set is used to learn a common similarity measure across dimensions;
[0008] In the second learning stage: the attention module is used to combine the two sets of latent feature vectors learned in the first learning stage to further learn the similarity measurement unique to each dimension. The dimension correlation information is considered through the common measurement, and the latent feature representation exclusive to each dimension is output. Finally, the 5 nearest neighbor classification criterion is used to classify multi-dimensional semantic objects.
[0009] As an improvement of the present invention, a deep metric learning method for multi-dimensional semantic objects includes the following steps:
[0010] S1: For each dimension of the label space, two sets of nonlinear mappings are calculated for all samples based on the multidimensional semantic object training set and two sets of multilayer perceptrons with a single hidden layer;
[0011] S2: For each dimension of the label space, construct the inter-class and intra-class similarity sets corresponding to all samples based on the multi-dimensional semantic object training set and the first set of nonlinear mappings of all samples obtained in step S1;
[0012] S3: For each power set space, construct the inter-class and intra-class similarity sets corresponding to all samples based on the dimensional semantic object training set and the second set of nonlinear mappings of all samples obtained in step S1;
[0013] S4: For each dimension of the label space, based on the two sets of nonlinear mappings obtained in step S1, a dot product attention module and a third set of multilayer perceptrons with a single hidden layer are used to calculate the latent feature representation of each sample;
[0014] S5: constructing inter-class and intra-class similarity sets corresponding to all samples in each dimension based on the latent feature representation of each sample obtained in step S4;
[0015] S6: Based on the inter-class and intra-class similarity sets of all samples obtained in steps S2, S3, and S5, a multi-dimensional metric loss objective function is constructed and the overall network is trained;
[0016] S7: Obtain latent feature representation for the multi-dimensional semantic test sample in each dimensional semantic space based on the network parameters obtained in step S6, and determine its semantic label based on the 5-nearest neighbor criterion to achieve classification.
[0017] As an improvement of the present invention, the specific method of calculating the two sets of nonlinear mappings of all samples in each dimension in step S1 is as follows: each multidimensional semantic sample in the multidimensional semantic object training set is represented by a d-dimensional feature vector Represents, corresponding to the label vector Let {φ j |1≤j≤q} is the first group of multi-layer perceptrons, {ψ j |1≤j≤q} is the second set of multilayer perceptrons. For the j-th dimension label space (1≤j≤q), the two sets of nonlinear mapping results for all samples are {φ j (x i )|1≤i≤m} and {ψ j (x i )|1≤i≤m}.
[0018] As another improvement of the present invention, the specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in step S2 is as follows: for the j-th dimension label space (1≤j≤q), sample x iThe first group of nonlinear mappings corresponding to (1≤i≤m) is φ(x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is:
[0019]
[0020]
[0021] in, express and The Euclidean distance of .
[0022] As another improvement of the present invention, the specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in each power set space in step S3 is: is a complete set of dimension indexes. For any non-empty set Note The power set space formed by the Cartesian product of the label space corresponding to the dimension of the index is Yizhiquan multi-dimensional labeling space For the j-th dimension label space (1≤j≤q), the corresponding power set space is defined as Consider the sample x i (1≤i≤m), the corresponding positive sample set And negative sample set The calculation formula is as follows:
[0023]
[0024] Sample x i The corresponding nonlinear mapping is ψ(x i ), which is in the power set space The intra-class similarity set on and the inter-class similarity set The calculation formula is as follows:
[0025]
[0026] As another improvement of the present invention, the specific implementation method of calculating the latent feature representation of each sample in step S4 is as follows: for the j-th dimension label space (1≤j≤q), consider the sample x i , let {δ j |1≤j≤q} is the third group of multi-layer perceptrons, then x i The latent feature representation g j (x i ) is calculated as:
[0027]
[0028] in, represents the dot product attention formula, h is the dimension of latent feature representation; then the latent feature representation set {g j (x i )|1≤i≤m}.
[0029] As another improvement of the present invention, the specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in each dimension in step S5 is as follows:
[0030] For the j-th dimension label space (1≤j≤q), the sample x i The hidden feature corresponding to (1≤i≤m) is represented as g j (x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is as follows:
[0031]
[0032] As a further improvement of the present invention, the specific implementation method of constructing a multi-dimensional metric loss objective function and training the overall network in step S6 is as follows:
[0033] Defining a metric function is the contrast loss function, for the intra-class similarity set S and the inter-class similarity set The calculation formula is:
[0034]
[0035] Among them, α represents a fixed interval;
[0036] For all samples, the first set of nonlinear mappings {φ j (x i )|1≤j≤q, 1≤i≤m}, the second group of nonlinear mappings {ψ j (x i )|1≤j≤q, 1≤i≤m} and the latent feature representation set {g j (x i )|1≤j≤q, 1≤i≤m}, and its corresponding metric loss calculation formula and They are:
[0037]
[0038] Total metric loss objective function for:
[0039]
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] 1. The present invention takes into account the similarity changes in different dimensional labeling spaces caused by the properties of the multidimensional semantic space in the multidimensional semantic object classification problem, and uses a two-stage metric learning method to learn a specific metric for each dimension.
[0042] 2. Compared with the existing multidimensional semantic object classification methods, the present invention is the first to learn effective metrics for multidimensional semantic objects. While considering the spatial dependencies between different dimensional labels of multidimensional semantic objects, it effectively measures the similarities between multidimensional semantic objects, thereby enhancing the discrimination ability and generalization performance of the multidimensional semantic object classification model. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a flowchart of the steps of the deep metric learning method for multi-dimensional semantic objects of the present invention;
[0044] Figure 2 is a schematic flow chart of step S2 of the method of the present invention;
[0045] Figure 3 is the power set space in step S3 of embodiment 1 of the present invention Calculate x i Flowchart of the corresponding positive sample set;
[0046] Figure 4 In step S3 of embodiment 1 of the present invention, in the power set space Calculate x i Flowchart of the corresponding negative sample set;
[0047] Figure 5 Schematic diagram of the process of step S7 of the method of the present invention. DETAILED DESCRIPTION
[0048] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.
[0049] Example 1
[0050] A deep metric learning method for multi-dimensional semantic objects includes at least two learning stages. First, in the first learning stage, two sets of latent feature vectors are calculated simultaneously, one set is used to learn the unique similarity metric in each dimension, and the other set is used to learn the shared similarity metric across dimensions. Then, in the second learning stage, an attention module is used to combine the two sets of latent feature vectors learned in the previous stage to further learn the similarity metric unique to each dimension. At the same time, the dimension correlation information is considered through the shared metric to output the latent feature representation exclusive to each dimension. Finally, the 5 nearest neighbor classification criterion is used to classify the multi-dimensional semantic objects.
[0051] A deep metric learning method for multi-dimensional semantic objects, such as Figure 1 As shown, it includes the following steps:
[0052] Step S1: For each dimension of the label space, two sets of nonlinear mappings of all samples are calculated based on the multi-dimensional semantic object training set and two sets of multi-layer perceptrons with a single hidden layer. The specific implementation method is as follows:
[0053] remember is the d-dimensional feature space, is a multidimensional labeling space with q dimensions, where the labeling space By K j The tags describe the semantic information of the jth dimension of the sample (1≤j≤q), and the multiplication sign indicates the Cartesian product. is a multi-dimensional semantic object training set, where each multi-dimensional semantic sample consists of a d-dimensional feature vector Represents, corresponding to the label vector
[0054] Let {φ j |1≤j≤q} is the first group of multi-layer perceptrons, {ψ j |1≤j≤q} is the second set of multilayer perceptrons. For the j-th dimension label space (1≤j≤q), the two sets of nonlinear mapping results for all samples are {φ j (x i )|1≤i≤m} and {ψ j (xi)|1≤i≤m}.
[0055] Step S2: For each dimension of the label space, based on the multi-dimensional semantic object training set and the first set of nonlinear mappings of all samples obtained in step S1, the inter-class and intra-class similarity sets corresponding to all samples are constructed. The specific construction method is as follows:
[0056] Figure 2 is a flow chart of step S2 of this embodiment. Considering the sample x i and its j-th dimension label Divide the positive sample set by examining whether the training sample has this label And negative sample set The calculation formula is shown in formula (1)-(2);
[0057]
[0058] For the j-th dimension label space (1≤j≤q), the sample x i The first group of nonlinear mappings corresponding to (1≤i≤m) is φ(x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is as shown in formula (3)-(4),
[0059]
[0060] in, express and The Euclidean distance of .
[0061] Thus, the set of positive samples corresponding to all samples in the j-th dimension label space is calculated. And negative sample set
[0062] Step S3: For each power set space, based on the dimensional semantic object training set and the second set of nonlinear mappings of all samples obtained in step S1, the inter-class and intra-class similarity sets corresponding to all samples are constructed. The specific construction method is as follows:
[0063] make is a complete set of dimension indexes. For any non-empty set The power set space formed by the Cartesian product of the labeled space of the corresponding dimension of the index in C is Yizhiquan multi-dimensional labeling space For the j-th dimension label space (1≤j≤q), the corresponding power set space is defined as Consider the sample x i (1≤i≤m), the corresponding positive sample set And negative sample set The calculation formula is shown in formula (5)-(6): Figure 3 and Figure 4 For this embodiment, in the power set space Calculate x i Flowchart of the corresponding positive (negative) sample set;
[0064]
[0065] Sample x iThe corresponding nonlinear mapping is ψ(x i ), which is in the power set space The intra-class similarity set on and the inter-class similarity set The calculation formula is shown in formula (7)-(8);
[0066]
[0067] Step S4: For each dimension of the label space, the two groups of nonlinear mappings obtained in step S1 are used to calculate the latent feature representation of each sample using the dot product attention module and the third group of multilayer perceptrons with a single hidden layer. The specific implementation method is as follows:
[0068] For the j-th dimension label space (1≤j≤q), consider the sample x i , let {δ j |1≤j≤q} is the third group of multi-layer perceptrons, then x i The latent feature representation g j (x i ) The calculation formula is shown in formula (9);
[0069]
[0070] in, represents the dot product attention formula, h is the dimension of latent feature representation;
[0071] Step S5: Based on the latent feature representation of each sample obtained in step S4, a set of inter-class and intra-class similarities corresponding to all samples in each dimension is constructed. The specific implementation method is as follows:
[0072] For the j-th dimension label space (1≤j≤q), the sample x i The hidden feature corresponding to (1≤i≤m) is represented as g j (x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is shown in formula (10)-(11);
[0073]
[0074] Thus, the set of positive samples corresponding to all samples in the j-th dimension label space is calculated. And negative sample set
[0075] Step S6: Based on the inter-class and intra-class similarity sets of all samples obtained in steps S2, S3, and S5, a multi-dimensional metric loss objective function is constructed and the overall network is trained. The specific construction method is as follows:
[0076] Defining a metric function is the contrast loss function, for the intra-class similarity set and the inter-class similarity set The calculation formula is shown in formula (12);
[0077]
[0078] Wherein, α represents a fixed interval, which is set to 1.0 in this embodiment, and max represents a maximum value function;
[0079] For all samples, the first set of nonlinear mappings {φ j (x i )|1≤j≤q, 1≤i≤m}, the second group of nonlinear mappings {ψ j (x i )|1≤j≤q, 1≤i≤m} and the latent feature representation set {g j (x i )|1≤j≤q, 1≤i≤m}, and its corresponding metric loss calculation formula and As shown in formulas (13)-(15) respectively;
[0080]
[0081] Total metric loss objective function The calculation formula is shown in formula (16);
[0082]
[0083] The network training method uses the Adam optimization algorithm. In this embodiment, the learning rate is set to 0.001 and the weight decay rate is set to 10 -5 .
[0084] Step S7: Based on the network parameters obtained in step S6, the multi-dimensional semantic test sample is represented by latent features in each dimensional semantic space, and its semantic label is judged based on the 5 nearest neighbor criterion, so as to achieve classification. The specific implementation method is as follows:
[0085] Figure 5 The flowchart of step S7 of this embodiment is as follows. For the test sample x* and the j-th dimension label space (1≤j≤q), based on the trained network g j The hidden feature representation on the j-dimensional label space is g j (x * )(see formula 9);
[0086] This calculates the latent feature representation of all samples in the multi-dimensional semantic object training set and g j (x *), select the 5 closest training samples and take the one with the largest number of marks As x * The predicted label of the jth dimension. Traverse j from 1 to q to achieve multi-dimensional semantic object classification.
[0087] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.
Claims
1. A deep metric learning method for multi-dimensional semantic objects, characterized by : Includes at least two learning stages; In the first learning phase: two sets of latent feature vectors are calculated simultaneously, one set is used to learn a unique similarity measure within each dimension, and the other set is used to learn a common similarity measure across dimensions; In the second learning stage: the attention module is used to combine the two sets of latent feature vectors learned in the first learning stage to further learn the similarity measurement unique to each dimension. The dimension correlation information is considered through the common measurement, and the latent feature representation exclusive to each dimension is output. Finally, the 5 nearest neighbor classification criterion is used to classify multi-dimensional semantic objects.
2. A deep metric learning method for multi-dimensional semantic objects according to claim 1, characterized in that: The steps include: S1: For each dimension of the label space, two sets of nonlinear mappings are calculated for all samples based on the multidimensional semantic object training set and two sets of multilayer perceptrons with a single hidden layer; S2: For each dimension of the label space, construct the inter-class and intra-class similarity sets corresponding to all samples based on the multi-dimensional semantic object training set and the first set of nonlinear mappings of all samples obtained in step S1; S3: For each power set space, construct the inter-class and intra-class similarity sets corresponding to all samples based on the dimensional semantic object training set and the second set of nonlinear mappings of all samples obtained in step S1; S4: For each dimension of the label space, based on the two sets of nonlinear mappings obtained in step S1, a dot product attention module and a third set of multilayer perceptrons with a single hidden layer are used to calculate the latent feature representation of each sample; S5: constructing inter-class and intra-class similarity sets corresponding to all samples in each dimension based on the latent feature representation of each sample obtained in step S4; S6: Based on the inter-class and intra-class similarity sets of all samples obtained in steps S2, S3, and S5, a multi-dimensional metric loss objective function is constructed and the overall network is trained; S7: Obtain latent feature representation for the multi-dimensional semantic test sample in each dimensional semantic space based on the network parameters obtained in step S6, and determine its semantic label based on the 5-nearest neighbor criterion to achieve classification.
3. A deep metric learning method for multi-dimensional semantic objects as claimed in claim 2, characterized in that: The specific method of calculating the two sets of nonlinear mappings of all samples in each dimension in step S1 is: each multidimensional semantic sample in the multidimensional semantic object training set is represented by a d-dimensional feature vector Represents, corresponding to the label vector Let {φ j |1≤j≤m} is the first group of multi-layer perceptrons, {ψ j |1≤j≤q} is the second set of multilayer perceptrons. For the j-th dimension label space (1≤j≤q), the two sets of nonlinear mapping results for all samples are {φ j (x i )|1≤i≤m} and {ψ j (x i )|1≤i≤m}.
4. A deep metric learning method for multi-dimensional semantic objects as claimed in claim 3, characterized in that: The specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in step S2 is as follows: For the j-th dimension label space (1≤j≤q), sample x i The first group of nonlinear mappings corresponding to (1≤i≤m) is φ(x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is: in, express and The Euclidean distance of .
5. A deep metric learning method for multi-dimensional semantic objects according to the system of claim 4, characterized in that: The specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in each power set space in step S3 is: is a complete set of dimension indexes. For any non-empty set Note The power set space formed by the Cartesian product of the label space corresponding to the dimension of the index is Yizhiquan multi-dimensional labeling space For the j-th dimension label space (1≤j≤q), the corresponding power set space is defined as Consider the sample x i (1≤i≤m), the corresponding positive sample set And the negative sample set The calculation formula is as follows: Sample x i The corresponding nonlinear mapping is ψ(x i ), which is in the power set space The intra-class similarity set on and the inter-class similarity set The calculation formula is as follows:
6. A deep metric learning method for multi-dimensional semantic objects according to claim 5, characterized in that: The specific implementation method of calculating the latent feature representation of each sample in step S4 is as follows: for the j-th dimension label space (1≤j≤q), consider the sample x i , let {δ j |1≤j≤q} is the third group of multi-layer perceptrons, then x i The latent feature representation g j (x i ) is calculated as: in, represents the dot product attention formula, h is the dimension of latent feature representation; then the latent feature representation set {g i (x i )|1≤i≤m}.
7. A deep metric learning method for multi-dimensional semantic objects according to claim 6, characterized in that: The specific construction method of constructing the inter-class and intra-class similarity sets corresponding to all samples in each dimension in step S5 is as follows: For the j-th dimension label space (1≤j≤q), the sample x i The hidden feature corresponding to (1≤i≤m) is represented as g i (x i ), whose intra-class similarity set and the inter-class similarity set The calculation formula is as follows:
8. A deep metric learning method for multi-dimensional semantic objects according to claim 7, characterized in that: The specific implementation method of constructing the multi-dimensional metric loss objective function and training the overall network in step S6 is as follows: Defining a metric function is the contrast loss function, for the intra-class similarity set and the inter-class similarity set The calculation formula is: Among them, α represents a fixed interval; For all samples, the first set of nonlinear mappings {φ j (x i )|1≤j≤q, 1≤i≤m}, the second group of nonlinear mappings {ψ j (x i )|1≤j≤q, 1≤i≤m} and the latent feature representation set {g j (x i )|1≤j≤q, 1≤i≤m}, and its corresponding metric loss calculation formula and They are: Total metric loss objective function for:
Citation Information
Cited By
Multi-dimensional semantic object classification method based on similarity information extraction
CN121350682A
