Method and computing device for characterizing continuity features in data table

Abstract

A method for characterizing a continuity feature in a data table and a computing device, the method comprising: acquiring a first feature value of a target feature, the target feature being a continuity feature, the feature value range of the target feature being pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to identifier embedded data; based on a first feature value interval to which the first feature value belongs, determining first interval embedded data corresponding to the first feature value; converting each element included in the identification embedded data based on the first feature value to obtain intermediate embedded data; and generating first representation data of the first feature value based on the first interval embedded data and the middle embedded data.

Description

Technical Field

[0001] The embodiments of the present specification belong to the field of computer technology, and more particularly, to a method for characterizing continuity features in a data table. Background Art

[0002] With the rapid development of deep learning technology, its application in tabular data processing is becoming more and more extensive. As one of the main forms of information recording, data tables are widely used in many fields such as medicine, finance, and commerce. However, since data tables contain both continuous and discrete features, there are many challenges in directly applying them to deep learning models, especially how to effectively perform feature vectorization. Traditional methods often process discrete features through simple One-Hot Encoding and other methods, while ignoring the continuity information of continuous features.

[0003] In a related technology, the continuous features in the data table are converted into discrete categories through discretization processing (binning) to make it easier to perform feature characterization. Specifically, the numerical space of the continuous feature can be divided into several intervals (bins) in the prediction model, and the embedding vector of each bin is used as the feature characterization of each data in the interval of the bin. For example, the continuous features such as the user's height and weight can be binned. However, in this scheme, since the smaller the interval, the more accurate the feature characterization of the specific data of the continuous feature, in order to improve the accuracy of the feature characterization, the numerical space of the feature needs to be divided into more bins, which requires a large amount of storage resources to store the embedding vectors of each bin, and more bins also lead to increased difficulty in training the prediction model. Summary of the invention

[0004] The object of the present invention is to provide a solution for characterizing the continuity features in a data table, so as to improve the accuracy of the characterization.

[0005] A first aspect of the present specification provides a method for characterizing a continuous feature in a data table, the method comprising:

[0006] Acquire a first feature value of a target feature, wherein the target feature is a continuous feature, wherein a feature value range of the target feature is pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to the identifier embedded data;

[0007] Determining first interval embedded data corresponding to the first eigenvalue based on a first eigenvalue interval to which the first eigenvalue belongs;

[0008] Convert each element included in the identification embedded data based on the first characteristic value to obtain intermediate embedded data;

[0009] First characterization data of the first eigenvalue is generated based on the first interval embedded data and the intermediate embedded data.

[0010] In one implementation, obtaining first characterization data of the first eigenvalue based on the first interval embedded data and the intermediate embedded data includes:

[0011] Combining the first interval embedded data and the intermediate embedded data to obtain first embedded data of the first eigenvalue;

[0012] Inputting the first feature value into a pre-trained feedforward neural network to obtain second embedded data;

[0013] The first embedded data and the second embedded data are combined to obtain the first representation data.

[0014] In one embodiment, combining the first embedded data and the second embedded data includes:

[0015] converting the second embedded data into third embedded data based on a hyperbolic tangent function;

[0016] The first embedded data and the third embedded data are combined.

[0017] In one embodiment, combining the first embedded data and the third embedded data comprises: multiplying each element of the first embedded data with an element at the same position in the third embedded data.

[0018] In one embodiment, combining the first interval embedded data and the intermediate embedded data comprises: adding each element of the first interval embedded data to an element at the same position in the intermediate embedded data.

[0019] In one embodiment, the method is performed by an input module in a prediction model, the data table includes multiple rows of data, each row of data includes feature values ​​of multiple features, the multiple features include the target feature, and the method further includes:

[0020] Inputting a first sample into the prediction model, the first sample including feature values ​​of at least some features in a row of data in the data table, the at least some features including the target feature;

[0021] Based on the output value of the prediction model and the first sample, the n interval embedded data and the identification embedded data are updated.

[0022] In one embodiment, the method further includes: updating parameters of the feedforward neural network based on the output of the prediction model and the first sample.

[0023] In one embodiment, the method further comprises:

[0024] Inputting a plurality of samples into the prediction model to obtain a plurality of model output values, the plurality of samples including the first sample, each sample including a feature value of at least part of the features in a row of data in the data table;

[0025] The updating of the n interval embedded data and the identifier embedded data based on the output value of the prediction model and the first sample comprises:

[0026] According to the multiple model output values ​​and the multiple samples, the n interval embedding data and the identification embedding data are updated based on a contrastive learning algorithm.

[0027] In one embodiment, updating the n interval embedded data and the identifier embedded data based on the output value of the prediction model and the first sample includes: updating the n interval embedded data and the identifier embedded data based on the output value of the prediction model, the first sample and the label value corresponding to the first sample.

[0028] In one embodiment, the prediction model further includes a backbone network and an output layer, and the method further includes: updating parameters of the backbone network and the output module based on the output value of the prediction model and the first sample.

[0029] A second aspect of the present specification provides a computing device, including:

[0030] an acquisition unit, configured to acquire a first feature value of a target feature, wherein the target feature is a continuous feature, wherein a feature value range of the target feature is pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to the identifier embedded data;

[0031] A determining unit, configured to determine first interval embedded data corresponding to the first eigenvalue based on a first eigenvalue interval to which the first eigenvalue belongs;

[0032] a calculation unit, configured to convert each element included in the identification embedded data based on the first characteristic value to obtain intermediate embedded data;

[0033] A generating unit is used to generate first characterization data of the first eigenvalue based on the first interval embedded data and the intermediate embedded data.

[0034] A third aspect of the present specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described in the first aspect.

[0035] A fourth aspect of the specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0036] The fifth aspect of this specification provides a computer program product, including a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0037] In the method for characterizing continuous features in a data table provided in an embodiment of the present specification, characterization data of the feature value is obtained by combining the binning characteristics and continuity information of the feature value, so that the final characterization data is more dynamic and contains more information, thereby having higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 is an architecture diagram of a prediction model for processing data in a data table in an embodiment of this specification;

[0040] Figure 2 A method flow for characterizing continuous features in a data table in an embodiment of this specification;

[0041] Figure 3 A schematic diagram of a process of characterizing a continuous feature in an embodiment of this specification;

[0042] Figure 4 It is a schematic diagram of a process of characterizing a continuous feature in another embodiment of the present specification;

[0043] Figure 5 This is a flow chart of a method for obtaining data representing feature values ​​based on interval embedded data and intermediate embedded data in an embodiment of this specification;

[0044] Figure 6 A schematic diagram of the effect achieved by the embodiments of this specification;

[0045] Figure 7This is a flow chart of a method for training a prediction model in an embodiment of this specification;

[0046] Figure 8 An architectural diagram of a computing device in an embodiment of this specification. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0048] Figure 1 Schematic diagram of the prediction model used to process data table data in the embodiment of this specification. Figure 1 As shown, the prediction model may include an input module 10, a backbone network 20 and an output module 30. The data table to be processed is as follows Figure 1 For example, the columns shown in the figure include merchant ID, feature f1 (shipping location), feature f2 (whether self-operated), feature f3 (number of years in business), etc. ( Figure 1 Only 4 columns are shown as an example). That is, each row in the data table corresponds to a plurality of merchants, and each row includes characteristic values ​​of a plurality of characteristics (e.g., f1, f2, and f3) of the merchant. The prediction model can output a row and a plurality of rows of data in the data table, and the prediction model can output a prediction result corresponding to each row of data in the data table, that is, a prediction result for each merchant. It can be understood that the data table is not limited to a data table including characteristic data of merchants, but can be a data table including characteristic data of other target objects, and the target object can be a user, a transaction, etc., without limitation.

[0049] Specifically, when inputting a prediction model, for example Figure 1 After a row of data is inputted, the input module 10 converts the feature values ​​of feature f1, feature f2 and feature f3 into embedded features f1_emb, f2_emb and f3_emb, respectively. Figure 1 Rectangles of different colors represent embedded features f1_emb, f2_emb, and f3_emb, respectively. It can be understood that the embedded feature can be a matrix or a vector, and the small squares in the rectangle can represent multiple elements included in the embedded feature.

[0050] Then, the input module 10 inputs the embedded features f1_emb, f2_emb and f3_emb to the backbone network 20. Among them, the backbone network can be understood as the core network part in the deep learning model, which is mainly used to extract higher-level features from the input data. The backbone network may include ResNet (Residual Network), VGG (Visual Geometry Group), MobileNet, etc. Among them, ResNet is a residual network with skip connection (Skip Connection), which solves the gradient disappearance problem in the deep network by introducing residual blocks. ResNet is one of the most commonly used backbone networks at present. VGG is a deep convolutional neural network, which mainly includes a series of stacked convolutional layers and pooling layers, and a fully connected layer. MobileNet is mainly used in scenarios with limited computing resources such as mobile devices. It greatly reduces the amount of parameters and calculations through depthwise separable convolution (Depthwise SeparableConvolution). In a specific embodiment, the backbone network 20 may include a multi-head attention layer in a transformer model (transformer) to extract more accurate high-level features based on embedded features.

[0051] The backbone network 20 outputs high-level features based on multiple embedded features ( Figure 1 The high-level features are shown in the example of a blue rectangle including multiple blocks in FIG. 1 ), and the high-level features are input to the output module 30. The output module 30 can output a discrete or continuous output as a prediction result. For example, the prediction result can be used to indicate the integrity score of the merchant 123, or can be used to indicate the integrity level of the merchant 123, wherein the integrity level can include low, medium and high.

[0052] The data table generally includes the following three types of feature values:

[0053] Categorical features: the feature name (i.e., column name in the data table) and feature value of this type of feature can contain multiple text tokens. The column name and feature value can be connected to obtain multiple tokens. The multiple tokens can be embedded to obtain the representation data of the categorical feature. For example, Figure 1 The feature f1 shown is a classification feature, and the characterization data of the feature f1 can be obtained in the above manner. Specifically, the column name "shipping location" can be connected with the feature value "Beijing" to obtain "shipping location Beijing", and the tokens "shipping location" and "Beijing" therein can be embedded to obtain two corresponding embedding vectors, which can be combined into an embedding matrix as the characterization data of the feature value "Beijing".

[0054] Binary features. The feature values of this type of feature are usually yes or no, which can be represented by 1 or 0. When the feature value is 1, the embedded data of part of the feature names of this type of feature can be used as the representation data of this feature. When the feature value is 0, it can be represented as 0. For example, Figure 1 The feature f2 shown is a binary feature. Since the feature value of f2 is yes (i.e., "1"), the embedded vector of "self-operated" in the column name can be used as the representation data of this feature value.

[0055] Numerical features. The feature values of this type of feature are usually continuous numerical values. For example, Figure 1 The feature f3 shown is a numerical feature. As described above, for the numerical features in the data table, in the prior art, the numerical range of the numerical features is usually divided into multiple intervals by the method of binning, and each interval is embedded to obtain the interval embedded data of each interval, and the interval embedded data of the interval where the specific feature value falls is used as the embedded data of this feature value. For example, as Figure 1 shown, assume that in the prediction model, the number of operating years is divided into multiple intervals such as bin1 = 1 year - 5 years, bin2 = 6 years - 10 years, bin3 = 11 years - 15 years, etc. Figure 1 The feature f3 = 7 in falls into the bin2 interval, so the embedded data corresponding to the interval bin2 can be used as the embedded data of the feature value "7" of the feature f3. However, as shown above, in this method, if you want to obtain the embedded data of each feature value more accurately, there are problems such as occupying a large amount of storage resources and high model training difficulty.

[0056] It can be understood that Figure 1 The data table shown is only exemplary. The data table is not limited to including the above three types of features, and may also include one or two types of the above three types of features (including numerical features, that is, continuous features). Or, the data table may include multiple numerical features.

[0057] In view of the problems existing in the above binning process, the embodiments of this specification propose a method for characterizing continuous features in a data table. By combining the binning features and continuous information of the feature values, the representation data of the feature values is obtained, so that the final representation data is more dynamic and contains more information. Therefore, compared with the binning process described above, better model prediction effects can be achieved with fewer bins.

[0058] Figure 21 is a flow chart of a method for characterizing continuous features in a data table in an embodiment of the present specification. The method can be executed by any computing device. In one embodiment, the method can be executed by the input module 10 in the prediction model. It can be understood that in the embodiment of the present specification, the method is not limited to being executed by the input module 10 in the prediction model. Other computing devices can determine the parameters (e.g., Figure 3 After f3_param shown in FIG, the parameters in the input module 10 are obtained, and execution is performed based on the obtained parameters. Figure 2 The following description is made by taking the input module 10 executing the method as an example.

[0059] like Figure 2 As shown, in step S201, the feature value of the target feature is obtained.

[0060] Wherein, the target feature is a continuous feature. Assume that the data input into the prediction model includes Figure 1 If a row of data is shown in the data table, the target feature may be feature f3 in the data table.

[0061] After receiving the input data table data, the input module 10 can determine the feature value of the continuous feature in the data table data according to the types of each pre-stored feature, thereby obtaining the feature value of the continuous feature (i.e., the target feature) from the data table data, and characterize the feature value using the parameters corresponding to the target feature.

[0062] In step S203, based on the interval to which the eigenvalue belongs, interval embedding data corresponding to the eigenvalue is determined.

[0063] Figure 3 FIG. 1 is a schematic diagram of the process of characterizing the continuous feature in the embodiment of this specification. Figure 3As shown in , assuming that the target feature is feature f3, the input module 10 can obtain the parameter f3_param corresponding to feature f3, and the parameter f3_param can be stored as part of the data of the input module 10. The parameter f3_param includes the embedded data (bin_emb1, bin_emb2, ...bin_embn) of n bins corresponding to feature f3, and the embedded data idx_emb of the feature identifier of feature f3. Among them, the above-mentioned embedded data can be a vector or a matrix according to different embedding methods, which is not limited. For example, in one embodiment, the embedded data of each bin can be an embedded vector, and the embedded vector of each bin can initially be a randomly determined embedded vector, and is updated in the model training of the prediction model. In another embodiment, the embedded data of each bin can be an embedded matrix, specifically, similar to the embedding of the classification feature described above, the feature name and the interval range corresponding to the bin can be connected to obtain multiple tokens, and based on the vectors corresponding to the multiple tokens, the multiple vectors are combined into a matrix to obtain the embedded matrix of the bin. The embedded matrix of the bin can also be updated in the model training. Similar to the embedding of classification features described above, one or more vectors may be obtained based on one or more tokens included in the feature name of feature f3, and embedded data idx_emb of the feature identifier of feature f3 may be obtained based on the one or more vectors.

[0064] After obtaining the feature value f3_v1 (eg, 7) of the feature f3, the input module 10 determines the bin corresponding to the feature value (eg, bin2=6 years-10 years), and thereby determines the interval embedding data bin_emb2 corresponding to the feature value based on the bin.

[0065] In step S205, each element included in the identification embedded data of the target feature is converted based on the feature value to obtain intermediate embedded data.

[0066] refer to Figure 3 , using the symbol Indicates that the identifier embedded data idx_emb is converted based on the eigenvalue f3_v1. The conversion, for example, includes multiplying the eigenvalue f3_v1 with each element in the identifier embedded data idx_emb to obtain the intermediate embedded data mid_emb. Wherein, when the identifier embedded data idx_emb is an embedded vector, the intermediate embedded data mid_emb is also in the form of a vector, or, when the identifier embedded data idx_emb is an embedded matrix, the intermediate embedded data mid_emb is also in the form of a matrix. It can be understood that the conversion is not limited to multiplying the eigenvalue f3_v1 with each element in the identifier embedded data idx_emb, but may include a variety of other operations, such as exponentiation, square root operation, log operation, and combinations of the above operations. Specifically, for the exponentiation operation, each element in the identifier embedded data idx_emb can be converted to the power of the element f3_v1. For example, for element a, it is converted to a f3 _ v1 Other operations can be performed similarly, that is, f3_v1 is used as a parameter in the operation to transform each element in the identifier embedded data idx_emb.

[0067] In step S207, characterization data of the feature value is obtained based on the interval embedded data and the intermediate embedded data.

[0068] refer to Figure 3 After determining the interval embedded data bin_emb2 and the intermediate embedded data mid_emb corresponding to the eigenvalue f3_v1, the eigenvalue is obtained by combining the interval embedded data bin_emb2 and the intermediate embedded data mid_emb to represent the data f3_emb. Figure 3 Chinese-Israeli Symbol Indicates the combination. Specifically, when the number of elements of the interval embedded data bin_emb2 and the intermediate embedded data mid_emb is the same, for example, the interval embedded data bin_emb2 and the intermediate embedded data mid_emb are vectors of the same length, the values ​​of the corresponding elements of the interval embedded data bin_emb2 and the intermediate embedded data mid_emb can be added to obtain the representation data f3_emb. When the number of elements of the interval embedded data bin_emb2 and the intermediate embedded data mid_emb is different, for example, the interval embedded data bin_emb2 is a d-dimensional vector and the intermediate embedded data mid_emb is an m*d-dimensional matrix, the interval embedded data bin_emb2 and the intermediate embedded data mid_emb can be combined into a (m+1)*d-dimensional matrix.

[0069] exist Figure 3In the process of characterizing the eigenvalues ​​shown, the characterization data of the eigenvalues ​​is obtained by combining the interval embedded data and the intermediate embedded data, that is, the characterization data includes discrete information and continuous information, thereby reducing information loss. The process also allows the characterization data of the current eigenvalue to be dynamically generated according to the different input eigenvalues, thereby improving the accuracy of the characterization data.

[0070] In this case, the number n of bins in the numerical space of the continuous feature can be set to a smaller number, while still achieving a better model prediction effect, thereby saving storage resources, reducing the difficulty of model training, and improving the prediction accuracy of the model.

[0071] The embodiments of this specification are not limited to Figure 3 In the manner shown, characterization data of feature values ​​are obtained based on interval embedded data and intermediate embedded data. Figure 4 A schematic diagram of a process for characterizing continuity features in another embodiment of the present specification is shown. Figure 4 The input module 10 in Figure 3 The difference is that the input module 10 further includes a feedforward neural network (FFN) and a unit for performing Tanh processing.

[0072] Among them, FNN is an artificial neural network, and its working mechanism can be simply understood as a one-way flow of information from the input layer to the output layer. FNN may include the following layers: input layer, which is the first layer of FNN, and the number of nodes in the input layer is the same as the dimension of the input data; hidden layer, which is located between the input layer and the output layer, and the hidden layer may have multiple layers, each layer is composed of several neurons, each neuron receives the output of the neurons in the previous layer, and after a weighted summation of a set of weights and bias values, it is processed using an activation function and output to the next layer; output layer, the output layer is the last layer of the network, which is used to output the prediction results. In the embodiment of the present specification, FFN may include only an input layer and one or more hidden layers to obtain embedded data emb2 corresponding to the eigenvalue f3_v1. The parameters included in the FFN can be trained when the prediction model is trained.

[0073] Figure 5 Flowchart of a method for obtaining characteristic value characterization data based on interval embedded data and intermediate embedded data in the embodiment of this specification. Figure 4 and Figure 5 Describe the process.

[0074] like Figure 5 As shown, in step S501, the interval embedding feature and the intermediate embedding feature are combined to obtain the embedded data emb1.

[0075] refer to Figure 5 After determining the interval embedding data bin_emb2 and the intermediate embedding data mid_emb corresponding to the characteristic value f3_v1, the input module 10 can obtain the embedded data emb1 by combining the interval embedding data bin_emb2 and the intermediate embedding data mid_emb, such as Figure 4 Symbols in As shown, the combination can be as described above for the symbol The description will not be repeated here.

[0076] In step S503, the eigenvalue f3_v1 is input into the feedforward neural network FFN to obtain the embedded data emb2.

[0077] It can be understood that when the input data of the prediction model only includes the eigenvalue of one continuous feature, the input layer of the FFN may include only one input node. When the input data of the prediction model only includes the eigenvalues ​​of multiple continuous features, the input layer of the FFN may include multiple input nodes for respectively receiving the eigenvalues ​​of each continuous feature. In this case, the FFN outputs embedded data corresponding to each continuous feature.

[0078] In step S505, the embedded data emb2 is transformed by a tanh function to obtain embedded data emb3.

[0079] Hyperbolic Tangent Function (Tanh) is one of the commonly used activation functions in neural networks. Its main function is to introduce nonlinearity in neurons so that neural networks can handle complex patterns and nonlinear relationships. The mathematical expression of the Tanh function is:

[0080]

[0081] The input of the Tanh function is any real number, and the output value is between -1 and 1.

[0082] In one embodiment, each element of the embedded data emb2 is converted using the tanh function, and the value of each element is converted into a value in the range of [-1, 1] to obtain the embedded data emb3. Thus, the embedded data emb3 can be regarded as weight data to make the effective data more prominent. In addition, the tanh function also maps larger values ​​to between [-1, 1], which has the effect of converging values.

[0083] In another embodiment, when the embedded data emb2 is an embedded vector, the embedded vector emb2 may be operated as a whole by a tanh function to obtain an embedded vector emb3.

[0084] In step S507, characterization data of feature values ​​are generated based on the embedded data emb1 and the embedded data emb3.

[0085] refer to Figure 4 By combining the embedded data emb1 and the embedded data emb3, the characteristic value representation data f3_emb can be obtained. The combination here is represented by the symbol Specifically, when the embedded data emb1 and the embedded data emb3 include the same number of elements, the elements of the embedded data emb1 can be in one-to-one correspondence with the elements of the embedded data emb3, so that the elements of the embedded data emb1 can be multiplied with the corresponding elements of the embedded data emb3 to obtain the characteristic value representation data f3_emb. It can be understood that the combination here is not limited to multiplying the elements of the embedded data emb1 and the embedded data emb3. For example, when the embedded data emb1 and the embedded data emb3 are vectors of the same dimension, the embedded vector emb1 and the embedded vector emb3 can be multiplied in the vector space, that is, the vector outer product is performed, so as to obtain the characteristic value representation vector f3_emb.

[0086] Understandably, Figure 4 and Figure 5 In the process shown, it is not limited to using the Tanh function to transform the embedded data. For example, the representation data of the eigenvalues ​​can also be directly obtained based on the embedded vector emb1 and the embedded vector emb2.

[0087] exist Figure 4 and Figure 5 In the process of characterizing the eigenvalues ​​shown in Figure 3 In the process shown, the intermediate embedded data mid_emb and the embedded data emb3 (or emb2) are obtained by using the continuity information twice, and the characterization data of the feature value is obtained based on the discrete information (bin_emb2), the intermediate embedded data mid_emb and the embedded data emb3 (or emb2), which further reduces the loss of information and improves the accuracy of the characterization data. In this case, the number n of bins in the numerical space of the continuous feature can also be set to a smaller number, and a better model prediction effect can also be achieved, thereby saving storage resources, reducing the difficulty of model training, and improving the prediction accuracy of the model.

[0088] Figure 6 Schematic diagram of the effect achieved by the embodiment of this specification. Figure 6As shown, it is assumed that the coordinate diagram marked with (a) is the correspondence between the eigenvalue of the continuous feature and its representation data under ideal conditions. It can be understood that the two-dimensional coordinates are schematically shown here. In practice, the representation data is usually vectors and matrices, and the coordinates can be drawn as multi-dimensional coordinates. Figure 6 (b) is the correspondence between the eigenvalue of the continuous feature in the binning algorithm and its representation data. Obviously, under this algorithm, the representation data of each eigenvalue in each bin is the same, and the error of this representation data is larger than that of the representation data in the ideal case (i.e. (a)). Figure 6 (c) is the correspondence between the eigenvalue of the continuous feature obtained by the method in the embodiment of this specification and its characterization data. In the embodiment of this specification, the characterization data of each eigenvalue is obtained by combining the embedding vector of the bin and the continuity information of the eigenvalue, so that the characterization data of different eigenvalues ​​in the same bin are different, and the error between the characterization data of the eigenvalue obtained by this method and the characterization data under the ideal situation (i.e. (a)) is very small, and it has high accuracy.

[0089] Figure 7 This is a flow chart of a method for training a prediction model in an embodiment of this specification, and the method can be executed by a training device for training a prediction model.

[0090] like Figure 7 As shown, in step S701, a training sample is input into a prediction model, wherein the training sample includes at least part of the feature values ​​in a row in a data table, including the feature value of the target feature.

[0091] In the embodiments of the present specification, the prediction model can be trained by supervised learning or self-supervised learning, wherein supervised learning requires labeling of label values ​​for each training sample, while self-supervised learning does not require sample labeling.

[0092] In the case of training a prediction model through supervised learning, one or more training samples may be obtained, each of which may include all or part of the feature values ​​in a row in a data table (these all or part of the feature values ​​include the feature value of the target feature), and the one or more training samples are input into the prediction model to obtain the model output. After the one or more training samples are input, the prediction model performs a model output for each training sample. Figure 2 The process shown outputs the prediction results.

[0093] In the self-supervised learning method, multiple training samples can be obtained, each training sample includes a portion of feature values ​​in a row in a data table, wherein at least some of the multiple training samples include feature values ​​of the target feature, and the multiple training samples are input into the prediction model to obtain the output of the model. Similarly, after the multiple training samples are input, the prediction model performs the following steps for each training sample: Figure 2 The process shown outputs the prediction results.

[0094] In step S703, based on the output of the prediction model and the training samples, the n interval embedding data and the identification embedding data of the target feature are updated.

[0095] In the supervised learning mode, the training device can determine the prediction loss based on the output of the prediction model, the training sample and the label value of the training sample, and update the n interval embedding data and the identification embedding data of the target feature based on the prediction loss, so that the output of the prediction model relative to the training sample is closer to the label value of the training sample. During the training process, the training device can also update the parameters of the above-mentioned FFN, the backbone network 20 and the output module 30 based on the prediction loss.

[0096] In the self-supervised learning method, two training samples including the same row of data in the data table are regarded as positive sample pairs, and two training sample pairs from different rows of data are regarded as negative sample pairs. The training device can determine the contrast loss based on the output of the prediction model and multiple training samples. The contrast loss is inversely proportional to the similarity of the prediction results output by the prediction model for the positive sample pair and is proportional to the similarity of the prediction results output by the prediction model for the negative sample pair. By updating the n interval embedding data and the identification embedding data of the target feature based on the contrast loss, the similarity of the prediction results output by the prediction model for the positive sample pair is increased, and the similarity of the prediction results output by the prediction model for the negative sample pair is reduced, thereby improving the accuracy of the prediction model.

[0097] Figure 8 is an architecture diagram of a computing device in an embodiment of this specification, the computing device is used to perform the following Figure 2 or Figure 5 The method shown comprises:

[0098] An acquisition unit 81 is used to acquire a first feature value of a target feature, wherein the target feature is a continuous feature, wherein the feature value range of the target feature is pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to the identifier embedded data;

[0099] A determining unit 82, configured to determine first interval embedded data corresponding to the first eigenvalue based on a first eigenvalue interval to which the first eigenvalue belongs;

[0100] A calculation unit 83, configured to convert each element included in the identification embedded data based on the first characteristic value to obtain intermediate embedded data;

[0101] The generating unit 84 is configured to generate first representation data of the first eigenvalue based on the first interval embedded data and the intermediate embedded data.

[0102] In one embodiment, the generating unit 84 is further used to: combine the first interval embedding data and the intermediate embedding data to obtain first embedding data of the first eigenvalue; input the first eigenvalue into a pre-trained feedforward neural network to obtain second embedding data; and combine the first embedding data and the second embedding data to obtain the first representation data.

[0103] In one embodiment, combining the first embedded data and the second embedded data includes: converting the second embedded data into third embedded data based on a hyperbolic tangent function; and combining the first embedded data and the third embedded data.

[0104] In one embodiment, combining the first embedded data and the third embedded data comprises: multiplying each element of the first embedded data with an element at the same position in the third embedded data.

[0105] In one embodiment, combining the first interval embedded data and the intermediate embedded data comprises: adding each element of the first interval embedded data to an element at the same position in the intermediate embedded data.

[0106] The embodiment of the present specification also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the following Figure 2 or Figure 5 The method shown.

[0107] The embodiment of the present specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the following is implemented: Figure 2 or Figure 5 The method shown.

[0108] The embodiments of the present specification also provide a computer program product, including a computer program / instruction, which is executed by a processor to implement the following Figure 2 or Figure 5 The steps of the method shown.

[0109] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0110] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0111] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0112] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0113] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0118] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0119] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0120] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0122] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0123] The above description is only an example of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims.

Claims

1. A method for characterizing a continuous feature in a data table, the method comprising: Obtaining a first feature value of a target feature, wherein the target feature is a continuous feature, the feature value range of the target feature is pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to the identifier embedded data; Determining first interval embedded data corresponding to the first eigenvalue based on a first eigenvalue interval to which the first eigenvalue belongs; Convert each element included in the identification embedded data based on the first characteristic value to obtain intermediate embedded data; First characterization data of the first eigenvalue is generated based on the first interval embedded data and the intermediate embedded data.

2. The method according to claim 1, wherein generating first characterization data of the first eigenvalue based on the first interval embedded data and the intermediate embedded data comprises: Combining the first interval embedded data and the intermediate embedded data to obtain first embedded data of the first eigenvalue; Inputting the first feature value into a pre-trained feedforward neural network to obtain second embedded data; The first embedded data and the second embedded data are combined to obtain the first representation data.

3. The method according to claim 2, wherein combining the first embedded data and the second embedded data comprises: converting the second embedded data into third embedded data based on a hyperbolic tangent function; The first embedded data and the third embedded data are combined.

4. The method according to claim 3, wherein combining the first embedded data and the third embedded data comprises: Each element of the first embedded data is multiplied by an element at the same position in the third embedded data.

5. The method according to claim 2, wherein combining the first interval embedded data and the intermediate embedded data comprises: Each element of the first interval embedded data is added to an element at the same position in the intermediate embedded data.

6. The method according to claim 2, wherein the method is executed by an input module in a prediction model, the data table comprises a plurality of rows of data, each row of data comprises feature values ​​of a plurality of features, the plurality of features comprises the target feature, and the method further comprises: Inputting a first sample into the prediction model, the first sample including feature values ​​of at least some features in a row of data in the data table, the at least some features including the target feature; Based on the output value of the prediction model and the first sample, the n interval embedded data and the identification embedded data are updated.

7. The method according to claim 6, further comprising: Based on the output of the prediction model and the first sample, the parameters of the feedforward neural network are updated.

8. The method according to claim 6 or 7, further comprising: Inputting a plurality of samples into the prediction model to obtain a plurality of model output values, the plurality of samples including the first sample, each sample including a feature value of at least part of the features in a row of data in the data table; The updating of the n interval embedded data and the identifier embedded data based on the output value of the prediction model and the first sample comprises: According to the multiple model output values ​​and the multiple samples, the n interval embedding data and the identification embedding data are updated based on a contrastive learning algorithm.

9. The method according to claim 6, wherein updating the n interval embedded data and the identification embedded data based on the output value of the prediction model and the first sample comprises: The n interval embedded data and the identifier embedded data are updated based on the output value of the prediction model, the first sample, and the label value corresponding to the first sample.

10. The method according to claim 6, wherein the prediction model further comprises a backbone network and an output layer, and the method further comprises: Based on the output value of the prediction model and the first sample, the parameters of the backbone network and the output module are updated.

11. A computing device comprising: an acquisition unit, configured to acquire a first feature value of a target feature, wherein the target feature is a continuous feature, wherein a feature value range of the target feature is pre-divided into n feature value intervals, the n feature value intervals correspond to n interval embedded data respectively, and the identifier of the target feature corresponds to the identifier embedded data; A determining unit, configured to determine first interval embedded data corresponding to the first eigenvalue based on a first eigenvalue interval to which the first eigenvalue belongs; a calculation unit, configured to convert each element included in the identification embedded data based on the first characteristic value to obtain intermediate embedded data; A generating unit is used to generate first characterization data of the first eigenvalue based on the first interval embedded data and the intermediate embedded data.

12. A computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 10 is implemented.

Images