Ordered regression method and system for processing table data

Abstract

The invention belongs to the field of table data prediction, and provides an ordered regression method and system for processing table data, and the method comprises the steps: carrying out the coding of a query sample and a table data training set through a multi-mode encoder, and obtaining initial and final query sample features, training features and a label candidate set; calculating an inter-class distance accumulated value of each type of features in the training feature candidate set, and retrieving sample features similar to the final query sample features and corresponding tags from the training feature and tag candidate set; calculating the contribution degree weight and the similarity between the similar sample features and the final query sample features by using the similar sample labels and the inter-class distance accumulated value; and fusing similarity and contribution degree weights, integrating with initial query sample features, and inputting into a predictor to obtain an ordered label. The inter-class distance accumulated value can reflect the grade, the non-equidistance and the inclusion of the ordered knowledge, and the classification prediction is promoted, so that the ordered label of the query sample can be predicted more accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of table data prediction, and in particular, relates to an ordered regression method and system for processing table data. Background Art

[0002] Ordinal regression is a classic problem in machine learning, which is specifically used to predict data with ordered labels. It is widely used in real life, such as movie ratings, product ratings, and lesion severity assessment. A table is a structured data set consisting of rows and columns.

[0003] At present, the research on ordered regression mainly focuses on fields such as computer vision and natural language processing. The inventor has retrieved ordered regression methods for image data and three-dimensional point cloud data, but no ordered regression methods for tabular data. For example, Patent Document 1: CN117333485A discloses a WSI survival prediction method based on a weakly supervised deep ordinal regression network, which inputs the image data into the ordinal regression network, performs survival prediction, and obtains the prediction result. Patent Document 2: CN106250818A discloses a face age estimation method based on full order preserving projection, which performs full order projection on the feature data extracted from the face image to obtain low-dimensional features, which are input into the ordered regression classification model to estimate the age. Patent Document 3: CN105975916A discloses an age estimation method based on a multi-output convolutional neural network and ordered regression, which uses face images and a binary ordered regression method to estimate age. Patent document 4: CN111428619A discloses a 3D point cloud head posture estimation system and method based on ordered regression and soft labels, which processes the 3D point cloud data of the head generated by the depth image to estimate the head posture. The above patent documents 1 to 3 all use images as input for level estimation, and patent document 4 uses 3D point cloud data as input for head posture estimation.

[0004] Due to the unique discreteness and heterogeneous multimodality of tabular data, ordered regression methods outside the field of tabular data prediction cannot be used for tabular data. Therefore, an ordered regression method for processing tabular data is urgently needed. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art, and proposes an ordered regression method for processing tabular data. The present invention also discloses an ordered regression system for processing tabular data.

[0006] In one aspect, the present invention provides an ordinal regression method for processing tabular data, comprising:

[0007] Step 1: Encode the query sample twice using a multimodal encoder to obtain initial query sample features and final query sample features, and encode the tabular data training set twice using a multimodal encoder to obtain a training feature candidate set and a label candidate set, wherein each sample in the tabular data training set is tabular data including entity features and corresponding ordered labels, and the query sample is tabular data including entity features;

[0008] Step 2: Calculate the cumulative value of the inter-class distances of each type of feature obtained after classification by ordered labels in the training feature candidate set, and use the retriever to retrieve multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample features from the training feature candidate set and the label candidate set;

[0009] Step 3: Calculate the contribution weights of multiple similar sample features using the accumulated values ​​of inter-class distances of multiple similar sample labels and various features, and calculate the similarity between the retrieved multiple similar sample features and the final query sample features;

[0010] Step 4: Refer to the attention mechanism, fuse the similarity and contribution weights to obtain context features;

[0011] Step 5: Integrate the initial query sample features and context features to obtain the final feature representation of the query sample;

[0012] Step 6: Use the predictor to predict the final feature representation and obtain the ordered labels of the query samples.

[0013] In an improved ordered regression method for processing tabular data, the step 2 of calculating the cumulative value of the inter-class distances of various features obtained after classification according to the ordered labels in the training feature candidate set includes:

[0014] Step 2-1: Calculate the inter-class distance between each type of feature obtained after classification by ordered labels in the training feature candidate set and the benchmark category feature;

[0015] Step 2-2: For each type of feature, the inter-class distances of each type of feature between the feature of this type and the benchmark feature are accumulated to obtain the inter-class distance accumulation value of the feature of this type.

[0016] In an improved ordered regression method for processing tabular data, the benchmark category is the first category, and step 2-1 includes:

[0017] Calculate the mean features of each type of features obtained after classification by ordered labels in the training feature candidate set;

[0018] Calculate the distance from the mean feature of each type of feature to the first type of feature to obtain the inter-class distance of each type of feature.

[0019] In an improved ordered regression method for processing tabular data, the step 3 uses the cumulative values ​​of inter-class distances of multiple similar sample labels and various features to calculate the contribution weights of multiple similar sample features, including:

[0020] Using the label candidate set and the cumulative inter-class distance values ​​of various features, a label inter-class weight dictionary is constructed, where the label inter-class weight dictionary includes the label of each category and the corresponding inter-class cumulative distance value;

[0021] Using multiple similar sample labels, find the corresponding inter-class distance accumulation value from the label inter-class weight dictionary;

[0022] The accumulated values ​​of the inter-class distances found are encoded to obtain the contribution weights of multiple similar sample features.

[0023] In an improved ordinal regression method for processing tabular data, step 4 includes:

[0024] Multiply the similarity and contribution weights to obtain context features.

[0025] In an improved ordinal regression method for processing tabular data, step 5 includes:

[0026] The initial query sample features and context features are added together to obtain the final feature representation of the query sample.

[0027] In an improved ordered regression method for processing tabular data, during training, the query sample is a sample in a training set; during verification, the query sample is a sample in a verification set; during testing, the query sample is a sample in a test set; in actual use, the query sample is a piece of data for which an ordered label is to be determined.

[0028] In an improved ordinal regression method for processing tabular data, the data type of the tabular data includes any one, any two or all of the following three types:

[0029] Numeric, character, and binary.

[0030] In an improved ordinal regression method for processing tabular data, the ordinal regression method is applied in product rating, age prediction, soil utility rating and retrieval related rating scenarios.

[0031] In another aspect, the present invention provides an ordered regression system based on the ordered regression method for processing tabular data, comprising:

[0032] A query encoding module, used to encode the query sample twice using a multimodal encoder to obtain an initial query sample feature and a final query sample feature, wherein the query sample is tabular data including entity features;

[0033] A training set encoding module is used to encode the tabular data training set twice using a multimodal encoder to obtain a training feature candidate set and a label candidate set, wherein each sample in the tabular data training set is tabular data including entity features and corresponding ordered labels;

[0034] The ordered knowledge enhancement module is used to calculate the cumulative value of the inter-class distance of each type of feature obtained after classification according to the ordered labels in the training feature candidate set;

[0035] A retrieval module, used for retrieving multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample features from the training feature candidate set and the label candidate set using a retriever;

[0036] The contribution weight calculation module is used to calculate the contribution weights of multiple similar sample features by using multiple similar sample labels and the accumulated values ​​of inter-class distances of various features;

[0037] A similarity calculation module is used to calculate the similarity between the retrieved multiple similar sample features and the final query sample feature;

[0038] The fusion module is used to refer to the attention mechanism, fuse the similarity and contribution weights, and obtain the contextual features;

[0039] An integration module, used to integrate the initial query sample features and the context features to obtain a final feature representation of the query sample; and

[0040] The prediction module is used to use the predictor to predict the final feature representation and obtain the ordered labels of the query samples.

[0041] Compared with the prior art, the advantages of the present invention are:

[0042] The solution of the embodiment of the present invention combines ordered regression and deep learning methods, and uses inter-class distance to represent the knowledge interval embodied by ordered data of different categories, that is, the hierarchy and unequal distance between classes. It reflects the inclusiveness of ordered knowledge through accumulation, and uses it as the basis for calculating contribution weights. It mines the hierarchical relationship between data features and promotes better classification prediction of ordered table data. Therefore, the ordered labels of query samples can be predicted more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of an ordered regression method for processing tabular data in Embodiment 1 of the present invention;

[0044] Figure 2 It is a schematic diagram of the process of processing table data by the table deep learning model in Example 2 of the present invention;

[0045] Figure 3 Schematic diagram of an ordered regression system for processing tabular data in Example 3 of the present invention. DETAILED DESCRIPTION

[0046] The ordered regression method for processing tabular data of the present invention first encodes the multimodal content and query samples of the tabular data training set through a multimodal encoder. Then, based on the query sample features, multiple similar features such as top-k similar features and similar labels are retrieved from the training feature candidate set, and the inter-class cumulative distance of each class in the training feature candidate set is calculated. Then, the contribution weight V value of n similar features in the attention mechanism is calculated using the inter-class cumulative distance and similar labels, and the similarity K*Q value in the attention mechanism is calculated by the top-k similar features. Finally, the label knowledge and similar features are fused through K*Q, V, and the query features and similar features are integrated to obtain the final feature representation of the query features, which is input into the predictor to obtain the ordered regression prediction value.

[0047] The ordered regression method of the present invention takes tabular data as input, uses the retrieval-based tabular deep learning model TabR model as the backbone network, adds an ordered knowledge enhancement module to this backbone network, and forms a new tabular deep learning model to predict ordered labels. The TabR model is described in the paper Yury Gorishniy, Ivan Rubachev, Nikolay Kartashev, Daniil Shlenskii, Akim Kotelnikov, and Artem Babenko. Tabr: Unlocking the power of retrieval-augmented tabular deep learning. arXiv preprint arXiv: 2307.14338, 2023.

[0048] The method and system of the present invention are used to process ordered table data, and application scenarios include but are not limited to product rating, age prediction, soil utility rating and retrieval relevance rating.

[0049] Product ratings, such as wine quality rating, can be found in the open source dataset at https: / / openml.org / d / 287.

[0050] Age prediction, such as abalone age prediction Abalone, its open source dataset can be found at https: / / archive.ics.uci.edu / dataset / 1 / abalone.

[0051] Soil utility level assessment eucalyptus, the open source dataset is available at https: / / www.openml.org / d / 188.

[0052] Retrieve relevance prediction, the open source dataset is available at https: / / www.microsoft.com / en-us / research / project / mslr / .

[0053] The data sets generated by these scenarios are all presented in the form of EXCEL tables. Each cell contains a value or a string, and the ordered labels are hierarchical labels (such as 1, 2, 3, 4, 5, which have hierarchical meanings).

[0054] The prediction goal is to predict which level the entity represented by a row of data belongs to. The meaning of each column in different data sets is different, but their common feature is that they are presented in a table format, each cell format is a numerical value or a single string, and the ordered label is a numerical identifier, indicating the meaning of the level.

[0055] Taking the wine quality rating dataset as an example, each row represents a sample, and there are 12 columns in total. Columns 1 to 11 are entity features, and columns 12 are corresponding ordered labels. Their meanings are shown in Table 1 below.

[0056] Table 1

[0057]

[0058] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, it is an ordered regression method for processing tabular data according to Embodiment 1 of the present invention, comprising:

[0061] Step 101: Encode the query sample twice using a multimodal encoder to obtain initial query sample features and final query sample features, and encode the tabular data training set using the multimodal encoder to obtain a training feature candidate set and a label candidate set.

[0062] Among them, each sample in the tabular data training set is tabular data including entity features and corresponding ordered labels, the query sample is tabular data including entity features, the multimodal encoder encodes the query sample for the first time to obtain the initial query sample features, and the multimodal encoder encodes the initial query sample features for the second time to obtain the final query sample features.

[0063] The query sample is the original existence of a row of data in the table. The first encoding is to encode the features of different modalities (types) to present them as numerical expressions, and pay attention to the important feature values ​​(because the multimodal encoder is learnable). The second encoding is just one more normalization layer than the first one to scale the features to a similar range, reduce the deviation of retrieving similar features from feature values ​​alone, and improve the retrieval generalization performance. When finally merging with the context features, the features of the first encoding are selected because they contain more accurate semantic information. The more multimodal encoders they pass through, the greater the possibility of causing semantic deviation.

[0064] The table data may contain numbers, characters, or data in different forms such as a sentence or an image. These different forms of data are multimodal, so a multimodal encoder is used in the embodiment of the present invention to process the table data.

[0065] Here, for different query samples, different tabular data sets are required, and the training sets are used to construct corresponding training feature candidate sets.

[0066] For example: when the query sample has 11 columns of feature data reflecting wine quality, it is necessary to use the wine quality table data training set to construct a wine quality training feature candidate set. This training set contains many samples, each of which has 12 columns of data, of which the first 11 columns are feature data reflecting wine quality, and the 12th column is wine quality grade data.

[0067] When the query sample is soil utility feature data, it is necessary to use the soil utility table data training set to construct a soil utility training feature candidate set.

[0068] Here, the query sample is 1 sample in the training set during training, 1 sample in the validation set during verification, and 1 sample in the test set during testing. In actual application, it is query data.

[0069] The candidate feature set in step 1 contains features, the candidate label set contains ordered labels, and the features in the candidate feature set still have a corresponding relationship with the labels in the candidate label set.

[0070] The new tabular deep learning model in the embodiment of the present invention is trained using a training set, and continuously learns and optimizes parameters through loss function feedback until the function converges and the training is stopped. After the training, the encoder, retriever, predictor, etc. all contain the trained parameters. In actual use, the query data and the training set can be directly used for ordered label prediction.

[0071] Step 102: Calculate the cumulative value of the inter-class distances of each type of feature obtained after classification according to the ordered labels in the training feature candidate set, and use the retriever to retrieve multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample feature from the training feature candidate set and the label candidate set.

[0072] Since the categories are ordered and the distances between these ordered categories are not equal, and the inclusiveness between ordered knowledge is found, that is, high-level knowledge contains low-level knowledge, the accumulation of inter-class distances is used to represent it in step 102. This makes the regression method of the present invention not only consider the sequential relationship between different categories, but also enhances the ordered knowledge between classes, making the subsequent ordered label prediction more accurate.

[0073] The above-mentioned calculation of the cumulative value of the inter-class distances of various types of features obtained after classification according to the ordered labels in the training feature candidate set may include step 102-1 and step 102-2:

[0074] Step 102-1: Calculate the inter-class distance between each class of features obtained after classification according to the ordered labels in the training feature candidate set and the benchmark class features;

[0075] Specifically, assuming that the ordered label values ​​can be 1, 2, and 3, the entity features with the ordered label of 1 in the training feature candidate set can be classified into the first category, the entity features with the ordered label of 2 in the training feature candidate set can be classified into the second category, the entity features with the ordered label of 2 in the training feature candidate set can be classified into the second category, and the entity features with the ordered label of 2 in the training feature candidate set can be classified into the third category.

[0076] The inter-class distance here can be Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, cosine distance, Hamming distance, etc. As long as the distance between features can be calculated, it can be used. When applying it, it is necessary to try to find the best distance formula one by one according to the characteristics of different data.

[0077] The benchmark category here can be selected based on the total number of categories, such as selecting the first category or the second category.

[0078] Specifically, the benchmark category is the first category, and step 102-1 includes:

[0079] Calculate the mean features of each type of features obtained after classification according to the ordered labels in the training feature candidate set; calculate the distance from the mean features of each type of features to the first type of features to obtain the inter-class distance of each type of features.

[0080] Unequally spaced class mean features are used here to approximate the unequal spacing of ordered knowledge between classes.

[0081] Step 102 - 2 : For each type of feature, the inter-class distances of each type of feature between the feature of that type and the reference feature are accumulated to obtain the inter-class distance accumulation value of the feature of that type.

[0082] The accumulated distance accumulated value here can be combined with the ordered labels to form a dictionary for determining the contribution weight.

[0083] Step 103: Calculate the contribution weights of the multiple similar sample features using the multiple similar sample labels and the accumulated values ​​of the inter-class distances of the various features, and calculate the similarity between the retrieved multiple similar sample features and the query sample feature.

[0084] Specifically, this step 103 specifically includes the following steps:

[0085] Step 1: Use the label candidate set and the cumulative distance between each type of feature to build a label inter-class weight dictionary, where the label inter-class weight dictionary includes the label of each category and the corresponding cumulative distance between classes;

[0086] Step 2: Using multiple similar sample labels, find the corresponding inter-class distance accumulation value from the label inter-class weight dictionary;

[0087] Step 3: Encode the cumulative value of the inter-class distance found to obtain the contribution weights of multiple similar sample features.

[0088] For example: according to the retrieved n similar sample feature rows, there are corresponding n ordered labels representing categories; for each of the n ordered labels representing categories, the inter-class distance cumulative value corresponding to the label that is the same as the label is searched from the dictionary, and finally n inter-class distance cumulative values ​​are obtained, but these inter-class distance cumulative values ​​cannot be directly used as the contribution weights of the n features, or the effect of directly using the inter-class distance cumulative values ​​as the contribution value calculation is not good, so an encoder is used here (including nn.linear, nn.Embedding layers, etc.), so that the contribution weights of n similar features can be learned during model training.

[0089] Step 104: Referring to the attention mechanism, the similarity and contribution weights are integrated to obtain context features.

[0090] Specifically, the similarity and contribution weights may be multiplied to obtain context features.

[0091] Step 105: Integrate the initial query sample features and context features to obtain the final feature representation of the query sample.

[0092] Specifically, the initial query sample features and context features are added to obtain the final feature representation of the query sample.

[0093] In this step 105, the contribution weight obtained based on the cumulative value of the inter-class distance is integrated into the query sample feature through the following features, thereby enriching the feature expression.

[0094] Step 106: Use the predictor to predict the final feature representation to obtain the ordered labels of the query samples.

[0095] In the scheme of Example 1 of the present invention, ordered regression and deep learning are combined, and corresponding training set features are extracted for different data sets to construct a candidate set of training features; then the distance between each type of feature is calculated, and this distance represents the distance between the ordered levels, and then a dictionary is formed with the ordered labels; multiple samples similar to the query sample are retrieved by similarity, and then the level distance (weight) and similar features are fused with the attention mechanism, integrated into the query sample features, and the feature expression is enriched; finally, the predictor is input for prediction. Not only is an ordered regression method for tabular data given, but the ordered labels of the query samples can also be accurately predicted.

[0096] Example 2

[0097] This embodiment 2 describes the solution of embodiment 1 of the present invention in detail, including the following Step 1 to Step 12.

[0098] Step 1: First, refer to the TabR model data format and divide the original table data into numeric type Xnum, character type Xcat, and binary type Xbin according to the data type.

[0099] Assume the dataset eucalyptus (https: / / www.openml.org / d / 188), one row is one sample, the training set is 515 rows, the validation set is 147 rows, the test set is 74 rows, there are 19 columns in total, including 13 numeric columns, 5 character columns, and 1 binary column, and the number of categories reflected by the ordered labels is 5.

[0100] The table deep learning model of the present invention processes the table data. Figure 2 , which can be compared Figure 2 To understand the solution of Example 2 of the present invention.

[0101] Step 2: Use the multimodal encoder in the TabR model to encode the training set sample set X twice to obtain the sample training feature candidate set F.

[0102] X=cat(X num ,X cat ,X bin ), where cat is a concatenation operation.

[0103] F=W k W x (X), where Wx and Wk are multimodal encoders.

[0104] Continuing with the above example, assuming that the feature dimension of each sample after training feature encoding is 128, F is a tensor of (515, 128).

[0105] Step 3: Use the training feature candidate set F of the training set to calculate the mean feature Fi of each type of feature:

[0106] Fi=M(F), where i=1,…,c;c is the number of categories, and M is the mean algorithm.

[0107] Continuing with the above example, since the number of data set categories is 5, Fi is a tensor of (5, 128).

[0108] Step 4: Calculate the distance between each type of feature and the first type of feature to obtain the inter-class distance D i .

[0109] D i =F i -F1, where i=1,...,c; c is the number of categories.

[0110] Continuing with the above example, if the Euclidean distance is used, the final tensor obtained is Di (1, 5).

[0111] Step 5: Calculate the cumulative distance S between each class i , and construct the dictionary Dict.

[0112] The accumulated distance is used to indicate that a high-level feature value contains a low-level feature value.

[0113] Where i=1,...,c,j is the category ordinal number.

[0114] Dict={key=y i :value=S i}, where y i is the label corresponding to the i-th category.

[0115] Using the above example, S i Yes D i The sum of the previous terms, so S iIt is also a tensor of (1, 5); the key value of Dict is 5 and the value value is also 5.

[0116] Step 6: Use the multimodal encoder W in the TabR model x and the multimodal encoder W k , for query x q Encoding is performed twice.

[0117] f x =W x (x q ), f k =W k (f x )

[0118] Query sample feature f x That is, the initial query sample feature, the query sample feature f k This is the final query sample feature.

[0119] Using the above example, here x q During training, it is 1 sample in the training set, during validation, it is 1 sample in the validation set, and during testing, it is 1 sample in the test set. q is of shape (1, 19), f k and f x Both are (1, 128) tesnsor.

[0120] Step 7: Use the retriever R in the TabR model to find n features f that match the query sample k Similar feature set Fs.

[0121] Fs=R(f k →F)

[0122] Continuing with the above example, if n is 96, then Fs is the tesnor of (1, 96, 128).

[0123] Step 8: Calculate the similarity KQ between the retrieved features and the query features:

[0124] KQ=softmax(-||f k -Fs|| 2 ), where softmax is a normalization function.

[0125] Continuing with the above example, KQ is a tensor of (1, 96).

[0126] Step 9: According to the ordered labels Y of the retrieved similar features, calculate the contribution weights of n similar features:

[0127] Value = W y(Dict[y=Y α ]), where α=1,...,n,W y It is a combination of nn.linear and nn.Embedding layers in the neural network. Its purpose is to encode integer or floating point values ​​to obtain the overall contribution weight. The physical meaning is to obtain n different contribution weights of n similar features.

[0128] Using the above example, the value of the tensor is (1, 96, 128).

[0129] Step 10: Refer to the attention mechanism and fuse similar features to obtain the context feature context_x.

[0130] context_x=KQ*Value, where * represents matrix multiplication, which is the multiplication between the KQ matrix and the Value matrix.

[0131] Continuing with the above example, context_x is a tensor of (1, 128).

[0132] Step 11: Integrate query sample features f x And context_x get the final feature representation context_fx.

[0133] context_fx=f x +context_x

[0134] Using the above example, context_fx is a tensor of (1, 128)

[0135] Step 12: Input the final features into the predictor P for prediction.

[0136]

[0137] Using the above example, The maximum value is selected from the tenor prediction probability of (1, 5), which is an ordered label for the prediction.

[0138] The embodiment of the present invention uses inter-class distance to represent the ordered knowledge interval, and reflects the inclusiveness of ordered knowledge through accumulation. After experimental verification, it is found that the wine quality rating is improved by 1.13%, 1.39% on eucalyptus, and 0.15% on abalone.

[0139] Example 3

[0140] This embodiment 3 proposes an ordered regression system based on the ordered regression method for processing tabular data in embodiments 1 and 2, such as Figure 3As shown, including:

[0141] A query encoding module 31 is used to encode the query sample twice using a multimodal encoder to obtain an initial query sample feature and a final query sample feature, wherein the query sample is tabular data including entity features;

[0142] A training set encoding module 32 is used to encode the tabular data training set twice using a multimodal encoder to obtain a training feature candidate set and a label candidate set, wherein each sample in the tabular data training set is tabular data including an entity feature and a corresponding ordered label;

[0143] The ordered knowledge enhancement module 33 is used to calculate the cumulative value of the inter-class distance of each type of feature obtained after classification according to the ordered labels in the training feature candidate set;

[0144] A retrieval module 34, configured to retrieve multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample feature from the training feature candidate set and the label candidate set using a retriever;

[0145] The contribution weight calculation module 35 is used to calculate the contribution weights of multiple similar sample features by using multiple similar sample labels and the accumulated values ​​of inter-class distances of various features;

[0146] A similarity calculation module 36, used to calculate the similarity between the retrieved multiple similar sample features and the final query sample feature;

[0147] A fusion module 37 is used to refer to the attention mechanism, fuse the similarity and contribution weights, and obtain context features;

[0148] An integration module 38, used to integrate the initial query sample features and the context features to obtain a final feature representation of the query sample; and

[0149] The prediction module 39 is used to predict the final feature representation using the predictor to obtain the ordered labels of the query samples.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. An ordinal regression method for processing tabular data, comprising: Step 1: Encode the query sample twice using a multimodal encoder to obtain initial query sample features and final query sample features, and encode the tabular data training set twice using a multimodal encoder to obtain a training feature candidate set and a label candidate set, wherein each sample in the tabular data training set is tabular data including entity features and corresponding ordered labels, and the query sample is tabular data including entity features; Step 2: Calculate the cumulative value of the inter-class distances of each type of feature obtained after classification by ordered labels in the training feature candidate set, and use the retriever to retrieve multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample features from the training feature candidate set and the label candidate set; Step 3: Calculate the contribution weights of multiple similar sample features using the accumulated values ​​of inter-class distances of multiple similar sample labels and various features, and calculate the similarity between the retrieved multiple similar sample features and the final query sample features; Step 4: Refer to the attention mechanism, fuse the similarity and contribution weights to obtain context features; Step 5: Integrate the initial query sample features and context features to obtain the final feature representation of the query sample; Step 6: Use the predictor to predict the final feature representation and obtain the ordered labels of the query samples.

2. The ordinal regression method for processing tabular data according to claim 1, characterized in that: The cumulative value of the inter-class distances of various features obtained by classifying the training feature candidate set according to the ordered labels in step 2 includes: Step 2-1: Calculate the inter-class distance between each type of feature obtained after classification by ordered labels in the training feature candidate set and the benchmark category feature; Step 2-2: For each type of feature, the inter-class distances of each type of feature between the feature of this type and the benchmark feature are accumulated to obtain the inter-class distance accumulation value of the feature of this type.

3. The ordered regression method for processing tabular data according to claim 2, characterized in that: The benchmark category is the first category, and step 2-1 includes: Calculate the mean features of each type of features obtained after classification by ordered labels in the training feature candidate set; Calculate the distance from the mean feature of each type of feature to the first type of feature to obtain the inter-class distance of each type of feature.

4. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: In step 3, the contribution weights of multiple similar sample features are calculated by using the accumulated values ​​of inter-class distances of multiple similar sample labels and various features, including: Using the label candidate set and the cumulative inter-class distance values ​​of various features, a label inter-class weight dictionary is constructed, where the label inter-class weight dictionary includes the label of each category and the corresponding inter-class cumulative distance value; Using multiple similar sample labels, find the corresponding inter-class distance accumulation value from the label inter-class weight dictionary; The accumulated values ​​of the inter-class distances found are encoded to obtain the contribution weights of multiple similar sample features.

5. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: Step 4 includes: Multiply the similarity and contribution weights to obtain context features.

6. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: Step 5 includes: The initial query sample features and context features are added together to obtain the final feature representation of the query sample.

7. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: During training, the query sample is a sample in the training set; during verification, the query sample is a sample in the verification set; during testing, the query sample is a sample in the test set; in actual use, the query sample is a piece of data with an ordered label to be determined.

8. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: The data types of the table data include any one, any two or all of the following three types: Numeric, character, and binary.

9. The ordered regression method for processing tabular data according to any one of claims 1 to 3, characterized in that: The ordinal regression method is applied in product rating, age prediction, soil utility rating and retrieval related rating scenarios.

10. An ordered regression system based on the ordered regression method for processing tabular data according to claim 1, comprising: A query encoding module, used to encode the query sample twice using a multimodal encoder to obtain an initial query sample feature and a final query sample feature, wherein the query sample is tabular data including entity features; A training set encoding module is used to encode the tabular data training set twice using a multimodal encoder to obtain a training feature candidate set and a label candidate set, wherein each sample in the tabular data training set is tabular data including entity features and corresponding ordered labels; The ordered knowledge enhancement module is used to calculate the cumulative value of the inter-class distance of each type of feature obtained after classification according to the ordered labels in the training feature candidate set; A retrieval module, used for retrieving multiple similar sample features and corresponding multiple similar sample labels that are similar to the final query sample features from the training feature candidate set and the label candidate set using a retriever; The contribution weight calculation module is used to calculate the contribution weights of multiple similar sample features by using multiple similar sample labels and the accumulated values ​​of inter-class distances of various features; A similarity calculation module is used to calculate the similarity between the retrieved multiple similar sample features and the final query sample feature; The fusion module is used to refer to the attention mechanism, fuse the similarity and contribution weights, and obtain the contextual features; An integration module, used to integrate the initial query sample features and the context features to obtain a final feature representation of the query sample; and The prediction module is used to use the predictor to predict the final feature representation and obtain the ordered labels of the query samples.

Images