Training method, device, readable storage medium and computing device for representation model

Abstract

The embodiments of this specification provide a training method, apparatus, readable storage medium and computing device for a representation model. The method includes: obtaining multiple business samples, any business sample includes event features of a business event, and has label values ​​corresponding to several business classification tasks; using the representation model to obtain multiple representation vectors corresponding to the multiple business samples; for a target business sample among the multiple business samples, determining several first samples with the same label value as the sample and several second samples with different label values; based on the multiple representation vectors, calculating the similarity between each first sample and each second sample and the sample, respectively, to obtain a comparative learning result; based on the comparative learning result of the target business sample, training the representation model so that the representation model learns the relationship between similar business events and the difference between different business events, and improves the recall ability of the representation vector.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of machine learning, and in particular, to training methods, apparatuses, readable storage media, and computing devices for characterizing models. Background Art

[0002] Business events refer to business-related events, such as transaction events, buying and selling events, fund events, and payment events. In order to facilitate the analysis of business events, a representation model is usually required to represent the business events, and the obtained representation vector can be used to implement subsequent business. These subsequent businesses may be the retrieval of similar business events or the analysis of similar business events. Therefore, the representation vector needs to have recall capabilities. At present, there is an urgent need for a training scheme for the representation model to improve the recall capability of the representation vector.

[0003] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0004] One or more embodiments of the present specification describe a method, apparatus, computer-readable storage medium, and computing device for training a representation model, wherein the representation model is trained by supervised comparative learning so that the representation model learns the relationship between similar business events and the difference between different business events; subsequently, when the business event is represented by the representation model, the obtained representation vector has a good recall capability.

[0005] According to a first aspect, a method for training a representation model is provided, comprising:

[0006] Acquire multiple business samples, where any business sample includes event features of a business event and has label values ​​corresponding to a plurality of business classification tasks;

[0007] Using the representation model, respectively performing event feature encoding processing on the multiple business samples to obtain multiple representation vectors corresponding to the multiple business samples one by one;

[0008] For a target business sample among the multiple business samples, determine from the multiple business samples a plurality of first samples having the same label value as the target business sample under a target task among the multiple business classification tasks, and a plurality of second samples having different label values; based on the multiple representation vectors, calculate the similarity between each first sample and each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample;

[0009] The representation model is trained based at least on the comparative learning results of the target business samples.

[0010] According to a feasible implementation method, the method also includes: for any task among several business classification tasks, the multiple characterization vectors are respectively input into the task model corresponding to the task to perform task prediction, and a prediction result is obtained, which includes the prediction value of each of the multiple business samples for the task; then the characterization model is trained based at least on the comparative learning result of the target business sample, including: training the characterization model based at least on the prediction results corresponding to each of the several business classification tasks, the label values ​​of each of the multiple business samples, and the comparative learning result of the target business sample.

[0011] In one example, the method further includes: for any task among several business classification tasks, training a task model corresponding to the task based on the prediction values ​​of each of the multiple business samples in the corresponding prediction results and the label values ​​of the task each of the multiple business samples has.

[0012] In one example, the task model includes a coding layer and a prediction layer, and the task prediction includes: for any vector among the multiple representation vectors, processing the representation vector through the coding layer to achieve information fusion, and outputting a fusion vector; and performing a prediction for the task based on the fusion vector through the prediction layer to obtain a predicted value of a business sample corresponding to any vector.

[0013] By way of example, the several business classification tasks correspond to multiple task models, and the multiple task models share the coding layer. The method also includes: training the coding layer based on at least the prediction results corresponding to the several business classification tasks, the label values ​​of the multiple business samples, and the comparative learning results of the target business samples; training the prediction layer based on the prediction results corresponding to the several business classification tasks and the label values ​​of the multiple business samples.

[0014] According to a feasible implementation method, the representation model includes an embedding layer, a feature interaction layer, a linear layer and a representation layer; the event feature encoding processing includes: using the embedding layer to vectorize the feature values ​​of each of multiple event features in any business sample, and obtain multiple feature value vectors that correspond one to one with the multiple event features; using the feature interaction layer to perform feature interaction processing on the multiple event features based on the multiple feature value vectors, and determine the interaction processing results; using the linear layer to perform linear processing on the multiple feature value vectors, and determine the linear processing results; using the representation layer to determine the representation vector corresponding to the business sample based on the interaction processing results and the linear processing results.

[0015] In one example, the multiple event features include discrete features and continuous features, and the interaction processing results include continuous feature interaction vectors and discrete feature interaction vectors; the representation vector is a vector obtained by concatenating the continuous feature interaction vector, the discrete feature interaction vector and the linear processing result; the feature interaction layer includes a first interaction layer and a second interaction layer; the feature interaction processing includes: performing feature interaction based on the eigenvalue vector corresponding to the continuous feature through the first interaction layer to determine the continuous feature interaction vector; performing feature multi-order interaction based on the eigenvalue vector corresponding to the discrete feature through the second interaction layer to determine the discrete feature interaction vector.

[0016] According to a feasible implementation manner, the method further includes: acquiring structured business data of the business event; and determining the multiple business samples by taking a single record in the structured business data as a single business sample.

[0017] In one example, the structured business data is claims data.

[0018] For example, the multiple business classification labels include a first label and a second label, the first label is used to indicate whether a claim is made, and the second label is used to indicate the type of claim case.

[0019] According to a second aspect, a training device for a characterization model is provided, comprising:

[0020] A sample acquisition module is configured to acquire a plurality of business samples, wherein any business sample includes event features of a business event and has label values ​​corresponding to a plurality of business classification tasks;

[0021] A characterization module is configured to use the characterization model to perform event feature encoding processing on the multiple business samples respectively, so as to obtain multiple characterization vectors corresponding to the multiple business samples respectively;

[0022] The comparative learning module is configured to determine, for a target business sample among the multiple business samples, a plurality of first samples having the same label value as the target business sample under a target task among the multiple business classification tasks, and a plurality of second samples having different label values; based on the multiple representation vectors, calculate the similarity between each first sample and each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample;

[0023] The training module is configured to train the representation model based at least on the comparative learning results of the target business samples.

[0024] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect.

[0025] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0026] In the embodiments of the present specification, supervised contrastive learning is used to enable the representation model to learn the relationship between similar business events and the difference between different business events; subsequently, when the business events are represented by the representation model, the obtained representation vector has a good recall capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0028] Figure 1 A schematic diagram showing structured data in one embodiment;

[0029] Figure 2 A schematic diagram showing the structure of a target model in one embodiment is shown;

[0030] Figure 3 A schematic diagram showing a comparative learning result of determining a business sample in one embodiment;

[0031] Figure 4 A schematic diagram showing a flow chart of a method for training a representation model according to an embodiment;

[0032] Figure 5 A schematic diagram of the structure of a training device for a characterization model according to an embodiment is shown. DETAILED DESCRIPTION

[0033] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0034] As the business continues to develop, new demands are constantly being made on the business, such as the retrieval of similar business events, the characterization of similar business event groups, etc. In this context, the retrieval and recall technology in the recommendation field can meet the above business demands due to its mechanism of recalling similar historical samples. Therefore, combining specific business scenarios and demands, building a representation model corresponding to business events, and better applying retrieval and recall technology to the representation model so that the representation vector output by the representation model has retrieval and recall capabilities, that is, perceiving similar business events based on the representation vector, has become an urgent problem to be solved.

[0035] Based on the above problems, the embodiments of this specification provide a solution for training a representation model, which trains the representation model by combining supervised multi-task learning and contrastive learning, so that the representation model learns the relationship between similar business events, the differences between different business events, and multi-task information, so as to improve the retrieval recall capability of the representation vector output by the representation model under multi-tasks.

[0036] The following describes the business sample set for training the representation model, the structure of the representation model, the scheme for training the representation model, and the application scenario of the representation model.

[0037] Business sample set for training representation model

[0038] The business sample set is a set formed by multiple business samples. Specifically, based on the structured data of the business event, multiple business samples for training the representation model are determined. Among them, structured data also refers to flat data, which consists of rows and columns, and each row has the same columns as other rows and in the same order. A given column has the same field name, but different columns can have different field names. Figure 1 A schematic diagram of structured data in one embodiment is shown. Figure 1 As shown, the structured data is formed by m+1 rows and N columns, including field 1, field 2, ..., field N. Field 1 represents the data in the first column, including field name 1 and m field values ​​corresponding to field name 1. The m field values ​​are represented as field value 11, field value 12, ..., field value 1m. Field 2 to field N are similar and will not be repeated. Correspondingly, a record in the structured data, that is, a row of data, is regarded as a business sample; Figure 1As shown, the 2nd to m+1th rows correspond to m business samples, which are represented as business sample 1, business sample 2, ..., business sample m. Business sample 1 includes field name 1 to field name N in the 2nd row with field value 11, field value 12, ..., field value N1 as feature values. Business samples 2 to business samples m are similar and will not be repeated. Among them, business events represent events related to business, such as transaction events, buying and selling events, fund events, and payment events. The above "events" include but are not limited to: operations on certain devices or applications, or actions performed by certain devices or applications, such as data interaction between POS machines and bank cards, payment operations performed through devices, etc.

[0039] It should be noted that the field value is the characteristic value of the event feature represented by the field name. Multiple field names in structured data represent different event features. For example, Figure 1 The field names 1 to N shown represent different event features; wherein the event features can be discrete features or continuous features; then the multiple event features in the structured data can include continuous features, discrete features, or continuous features and discrete features. For example, the structured data is the claim data corresponding to the claim event, the continuous features can be the insurance amount, the number of claims, the medical expenses, etc., and the discrete features can be the type of event to be claimed, the claim rules, the insurance liability, the claim start date, the claim end date, the claim application date, the claim hospital, etc. For example, the structured data is the transaction data corresponding to the transaction event, the continuous features can include: the amount involved, the number of active party operations, the number of passive party operations; the discrete features can include: the payment method, the transaction subject, the payment card number. For example, the structured data is the loan data corresponding to the loan event, the continuous features can include: the loan amount, the number of overdue times, the monthly income of the lender, the number of installments of the lender, the credit score of the lender; the discrete features can include: the occupation of the lender, the loan platform.

[0040] It is worth noting that structured data is obtained by cleaning the original data. Data cleaning can be to remove invalid data such as missing values ​​and outliers. The specific method of data cleaning can be determined in combination with the actual business scenario, and the embodiments of this specification do not make specific limitations on this.

[0041] Characterizing the structure of the model

[0042] Figure 2 FIG. 1 is a schematic diagram showing the structure of a target model in one embodiment. The target model can be understood as a related model for implementing the training of the representation model. Figure 2As shown, the target model includes a representation model and multiple task models, and the multiple task models are represented as task model 1, task model 2, ..., wherein the representation model includes an embedding layer, a linear layer, a feature interaction layer and a representation layer.

[0043] The embedding layer is used to preprocess and vectorize the feature values ​​of event features in the input business samples.

[0044] Here, preprocessing includes normalization, binning (an operation that converts continuous feature data into discrete data, generally for the purpose of improving generalization ability), etc. The embodiments of this specification do not make specific limitations on this. Different preprocessing methods can be selected in combination with the characteristics of the feature values ​​of the event features. It is worth noting that the embedding layer will set the event features that need to be preprocessed in advance, as well as the relevant processing parameters of the event features. These relevant processing parameters usually comprehensively consider the feature values ​​of the event features in all business samples, that is, the situation of all field values ​​in the field where the event features are located.

[0045] For example, Figure 1 As shown, assuming that field 1 needs to be normalized, the relevant processing parameters can be the maximum and minimum values ​​of field values ​​11 to 1m; subsequently, for any service sample among the m service samples, for the convenience of description, it is called the i-th service sample, after the i-th service sample is input into the embedding layer, the embedding layer substitutes the size, maximum and minimum values ​​of the field value 1i into the normalization function calculation, and the field value after the normalization of the field value 1i can be determined. Optionally, the normalization function can refer to the following formula (1).

[0046] X inorm =(X i -X min ) / (X max -X min ) (1)

[0047] Among them, X inorm represents the normalized field value of the field value 1i of the i-th service sample among the m service samples; X i Indicates the field value 1i of the i-th service sample; X max Indicates the maximum value between field value 11 and field value 1m; X min Indicates the minimum value among field values ​​11 to 1m.

[0048] For example, Figure 1As shown in the figure, assuming that field 2 needs to be binned, multiple intervals can be obtained after data division based on field value 21 to field value 2m; correspondingly, the relevant processing parameters for binning are the data ranges of the multiple intervals. Subsequently, after the i-th business sample is input into the embedding layer, the embedding layer can determine the interval to which field value 2i belongs based on the size of field value 2i and the data ranges of the multiple intervals, and the interval is used as the feature value after preprocessing of field value 2i.

[0049] Here, the vectorized representation can be understood as an embedding mapping.

[0050] For example, if the feature value of the event feature is text, it is necessary to perform embedding mapping on each word in the text and then concatenate them to obtain the feature value vector of the feature; for example, for any word in the text, look up the word vector table (including the encoding vectors of a large number of words), and the encoding vector of the matching word is the token_embedding (word embedding) of the word; based on the sequential number of the word (obtained by sorting the words in the text), look up the sorted bit vector table (including the encoding vectors of multiple sequential numbers), and the encoding vector of the matching sequential number is the position embedding (position embedding) of the word; and merge the token_embedding and position embedding to complete the embedding mapping of the word.

[0051] For example, if the feature value of an event feature is a numerical value, it is necessary to perform embedding mapping on the numerical value to obtain the feature value vector of the event feature. For example, based on the size of the numerical value, a numerical vector table (including encoding vectors of multiple numerical values) is searched, and the encoding vector of the matching numerical value is the encoding vector of the numerical value.

[0052] For example, if the feature value of the event feature is an interval, it is necessary to perform embedding mapping on the interval to obtain the feature value vector of the feature. For example, the interval vector table (including the encoding vectors of multiple intervals) is searched, and the encoding vector of the matching interval is the encoding vector of the interval.

[0053] It is worth noting that the embedding layer outputs the eigenvalue vector of each business feature in the business sample.

[0054] Among them, the linear layer is used to linearly map each eigenvalue vector corresponding to the business sample output by the embedding layer to obtain a linearized vector; in an example, the linear layer can be a single-layer fully connected layer or an embedding aggregation, that is, integrating the event features in the business sample.

[0055] Among them, the feature interaction layer is used to interact with each eigenvalue vector corresponding to the business samples output by the embedding layer.

[0056] In one example, the feature interaction layer includes a discrete feature interaction layer and a continuous feature interaction layer; wherein the discrete feature interaction layer is used to perform multi-order feature interaction based on the eigenvalue vectors of each input discrete feature to obtain a discrete feature interaction vector; the continuous feature interaction layer is used to perform feature interaction based on the eigenvalue vectors of each input continuous feature to obtain a continuous feature interaction vector. By way of example, the discrete feature interaction layer can be a CIN (Compressed Interaction Network) network, which is a network structure proposed in the XDeepFM paper, used to realize multi-order cross-expression of features, and is an extension of the FM idea in high-order cross-expression. By way of example, the continuous feature interaction layer can be a multi-layer perceptron. It should be noted that business applications have certain restrictions on the real-time performance of the representation model calculation, and it is not appropriate to use model algorithms with complex parameters. Therefore, the feature interaction layer selects a network with smaller model parameters to meet the needs of high timeliness.

[0057] The representation layer is used to combine, for example, concatenate the outputs of the linear layer and the feature interaction layer to obtain a representation vector corresponding to the business sample. In one example, the representation layer is used to concatenate the linearized vector output by the linear layer, the continuous feature interaction vector output by the feature interaction module, and the discrete feature interaction vector to obtain a representation vector corresponding to the business sample.

[0058] It should be understood that in the embodiments of the present specification, considering that subsequent business may need to utilize the linear relationship between the label and event features of the business sample, and the subsequent business requires the representation vector output by the representation layer, the linear relationship can be directly reflected in the representation vector, so as to facilitate the direct use of the linear relationship to implement subsequent business.

[0059] Scheme for training representation models

[0060] The embodiments of this specification adopt a combination of supervised multi-task learning and supervised contrastive learning when training the characterization model. In addition, in order to reduce the number of iterations and increase the data processing speed, the embodiments of this specification adopt batch training. Among them, batch training can be understood as dividing the business sample set into several batches for training, and each batch selects n_num (the number of samples in the business sample set) / n_batch (the number of batches) business samples from the business sample set until all business samples in the business sample set are trained. For the convenience of description, the set formed by all business samples selected from a batch is called a batch business sample set. The following description takes the training of the characterization model using a batch business sample set as an example.

[0061] For supervised multi-task learning. First, the labels of business samples are explained. In view of the fact that the embodiments of this specification involve multiple business classification tasks, for any business sample, it is necessary to mark the label value of the business sample under each business classification task. For example, in the example where the business event is a claim event, multiple business classification tasks may include whether to claim and the type of claim case; in this example, for the business classification task of whether to claim, the label value can be 0 (indicating that no claim is required) and 1 (claim is required); for the business classification task of the claim case type, assuming that there are multiple types, for example, type 1, type 2, type 3, type 4,..., then the label value of the business sample can be a vector encoded using one-hot encoding, in which the value of the encoding position corresponding to the type of the business sample is 1, and the values ​​of other encoding positions are 0. In addition, for any business classification task, a corresponding task model will be constructed. Figure 1 Task model 1, task model 2, ... are shown. In one example, the task model may include a coding layer and a prediction layer, where the coding layer is used to further encode the representation vector and fuse the information represented by the representation vector to obtain a fused vector; by way of example, the coding layer may be a fully connected layer; by way of example, if the representation vector obtained includes a concatenation of three types of vectors, namely, a linearized vector, a continuous feature interaction vector, and a discrete feature interaction vector, the coding layer may further fuse the three types of vectors to obtain deeper semantics; the prediction layer performs prediction for the business classification task based on the fused vector to obtain a predicted value of the business sample represented by the representation vector.

[0062] Secondly, for each of the multiple business classification tasks, the representation vector corresponding to the business sample is input into the task model corresponding to the task, the predicted value of the business sample for the task is obtained, and it is merged into the prediction result corresponding to the task. After processing each business sample in the batch business sample set in the above manner, the prediction results of each of the multiple business classification tasks can be obtained. Among them, for any business classification task, the prediction result of the task includes the prediction value of each business sample in the batch business sample set for the task.

[0063] Afterwards, for any of the multiple business classification tasks, based on the prediction result of the task and the label value of each business sample in the batch business sample set for the task, determine the sample prediction loss of each business sample in the task; determine the task prediction loss based on the sample prediction loss of each business sample in the batch business sample set for the task, for example, the task prediction loss is the sum, mean, etc. of the sample prediction losses of each business sample in the batch business sample set for the task, which can be determined in combination with the actual situation, preferably sum; further, train the task model and the characterization model based on the task prediction loss. In an example, there can be N categories under a certain business classification task, and the prediction value of the business sample for the task includes the probability values ​​of each of the N categories.

[0064] For example, when N=2, the sample prediction loss L of the i-th service sample can be calculated by the following formula (2): i :

[0065] L i =y i ·log(P i )+(1-y i )·log(1-P i ) (2)

[0066] Among them, y i represents the label value of the i-th business sample for the task, indicating its true category under the task. When the true category is a positive example, the label value is 1, and when it is a negative example, the label value is 0; P i It represents the probability that the i-th business sample predicted by the task model is a positive example.

[0067] For example, when N is greater than 2, the sample prediction loss L of the service sample can be calculated by the following formula (3): i :

[0068]

[0069] Among them, y ij P represents the label value of the jth category of the i-th business sample under this task, which is expressed as 0 or 1, 0 means it does not belong to the j-th category, and 1 means it belongs to the j-th category; ij It represents the probability that the i-th business sample predicted by the task model belongs to the j-th category.

[0070] It should be noted that the above loss function is only an example and does not constitute a specific limitation. The loss function can be reasonably designed in combination with the specific situation of the representation model.

[0071] It is worth noting that if the task models corresponding to multiple business classification tasks share a coding layer, the shared coding layer can be trained based on the task prediction losses corresponding to the multiple business classification tasks, such as the sum or mean of these losses; in addition, for any business classification task of multiple business classification tasks, the prediction layer in the task model corresponding to the task is trained based on the task prediction loss corresponding to the task.

[0072] For supervised contrastive learning, we first explain the positive sample pairs and negative sample pairs in the contrastive learning process. Figure 3 A schematic diagram of determining a comparative learning result of a business sample in one embodiment is shown. Figure 3 The content shown here describes the positive sample pairs and negative sample pairs. Please refer to Figure 3, for any business sample in the batch business sample set, business samples are selected from the batch sample sample set, and each business sample with the same label as the business sample under the target task is respectively used as a positive sample, and the combination of the business sample and a positive sample is defined as a positive sample pair, and each positive sample is combined with the business sample to obtain several positive sample pairs; each business sample with a different label from the business sample under the target task is respectively used as a negative sample, and the combination of the business sample and a negative sample is defined as a negative sample pair, and each negative sample is combined with the business sample to obtain several negative sample pairs. In practical applications, the target task can be designed in combination with actual needs. The embodiments of this specification do not intend to limit the number of target tasks. For example, the performance of different business classification tasks can be verified by experiments, and one or more tasks with poor performance can be used as target tasks. It is also possible to use each business classification task as a target task, and various possible combinations of these business classification tasks can also be used as target tasks. Of course, for the target task, the processing process of each business sample is similar, and the only difference is the difference between the positive sample set and the negative sample set.

[0073] Next, the process of contrastive learning is described. Figure 3 The following content explains the contrastive learning process in detail. Figure 3 For any business sample in the batch business sample set, based on the respective characterization vectors of multiple business samples, calculate the similarity between the two business samples in the positive sample pair (for the convenience of description and distinction, referred to as the first similarity), and the similarity between the two business samples in the negative sample pair (for the convenience of description and distinction, referred to as the second similarity), and use the first similarity corresponding to each positive sample pair and the second similarity corresponding to each negative sample pair as the comparative learning results; based on the comparative learning results of the business sample, determine the sample comparative learning loss of the business sample. Here, the similarity between business samples is illustrated by the similarity between the characterization vectors, such as the cosine similarity. The above cosine similarity is only used as an example and does not constitute a specific limitation. In an example, the sample comparative learning loss L of the i-th business sample can be calculated by the following loss function (4): icon :

[0074]

[0075] Where h0(·) represents the defined distance function, which indicates the similarity between the representation vectors corresponding to two business samples; x in represents the nth positive sample pair of the ith business sample; x in represents the nth positive sample pair of the ith business sample; h0(x in) represents the first similarity between the representation vectors corresponding to the two business samples in the nth positive sample pair of the i-th business sample; y im represents the mth negative sample pair of the ith business sample; h0(y im ) represents the second similarity between the representation vectors corresponding to the two business samples in the mth negative sample pair of the ith business sample; p represents the number of positive sample pairs; q represents the number of negative sample pairs. It should be noted that the above loss function is only used as an example and does not constitute a specific limitation. The loss function can be reasonably designed in combination with the specific situation of the training representation model.

[0076] Finally, the representation model is trained based on the prediction loss of each business classification task and the contrastive learning loss of each business sample. Furthermore, if the task models of multiple business classification categories share a common coding layer, the coding layer of the task model can also be trained based on this, so that the coding layer can learn the relationship between similar business events, the difference between different business events, and the information of multiple tasks, so as to facilitate the classification of the prediction layer of the task model.

[0077] Application scenarios of characterization models

[0078] After training the representation model and task model in the above manner, in actual applications, the business events are input into the representation model and then into the task models of different business classification tasks, so that the prediction values ​​of the business samples in different business classification tasks can be determined. Furthermore, for scenarios that need to be retrieved and recalled, a business event set formed by multiple business events is determined, and each business event in the set is input into the representation model. The representation vectors of each business event obtained form a vector set. After that, the collected business events are input into the representation model to obtain the representation vector of the business event, and the similarity between the representation vector and each representation vector in the vector set is calculated, and the business events of each of the multiple representation vectors with the highest similarity are respectively regarded as similar business events. Subsequently, business personnel can analyze the business events and their corresponding similar business events.

[0079] In summary, in the embodiments of the present specification, the representation model is trained by combining supervised multi-task learning and contrastive learning, so that the representation model learns the relationship between similar business events, the differences between different business events, the information of several tasks, and the information of multiple tasks, so as to improve the retrieval recall capability of the representation vector output by the representation model under multiple tasks.

[0080] Figure 4 The flowchart of the training method of the representation model according to one embodiment is shown, and the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 4As shown, the method includes the following steps: step 41, obtaining multiple business samples, any business sample includes event features of business events, and has label values ​​corresponding to several business classification tasks; step 42, using the representation model, respectively performing event feature encoding processing on multiple business samples to obtain multiple representation vectors corresponding to the multiple business samples one by one; step 43, for a target business sample among the multiple business samples, from the multiple business samples, determine a number of first samples having the same label value as the target business sample under target tasks in several business classification tasks, and a number of second samples having different label values; based on the multiple representation vectors, calculate the similarity of each first sample, each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample; step 44, at least based on the comparative learning result of the target business sample, train the representation model.

[0081] First, in step 41, the above-mentioned multiple business samples can form the above-mentioned batch business sample set or business sample set. In practical applications, when forming the above-mentioned business samples, structured data can be obtained, and a single record in the structured data can be used as a single business sample to obtain a business sample set; when forming the above-mentioned batch business samples, it is further necessary to divide the business sample set according to the number of batches to obtain a batch business sample set. Among them, for any business sample, the sample includes multiple event features of the business event and the characteristic values ​​of each of the multiple event features. Among them, the event feature can be understood as the characteristics of the business event. For details, see above and will not be repeated here. Furthermore, the business sample also has a label value corresponding to each of several business classification tasks. Here, the label value can be manually marked, reflecting the actual classification of the business sample under the business classification task, and can be a vector encoded in a unique hot encoding manner. The embodiments of this specification are not intended to limit the number of business classification tasks, which can be determined in combination with actual needs.

[0082] Next, in step 42, for a single service sample, the service sample is input into the characterization model, and the characterization model performs event feature encoding processing on the service sample to obtain a characterization vector corresponding to the service sample. In this way, after processing multiple service samples, the characterization vectors corresponding to the multiple service samples can be obtained.

[0083] According to a feasible implementation method, the representation model includes an embedding layer, a feature interaction layer, a linear layer and a representation layer. Specifically, the event feature encoding processing of a single business sample is implemented in the following manner: the embedding layer is used to vectorize the feature values ​​of each of the multiple event features in the business sample to obtain multiple feature value vectors corresponding to the multiple event features; then, the feature interaction layer is used to perform feature interaction processing on the multiple event features based on the multiple feature value vectors to determine the interaction processing results; then, the linear layer is used to perform linear processing on the multiple feature value vectors to determine the linear processing results; thereafter, the representation layer is used to determine the representation vector corresponding to the business sample based on the interaction processing results and the linear processing results, such as splicing.

[0084] In one example, multiple event features include discrete features and continuous features. Based on this, the feature interaction layer includes a first interaction layer (corresponding to the above discrete feature interaction layer) and a second interaction layer (corresponding to the above continuous feature interaction layer). Correspondingly, the interaction processing result includes a continuous feature interaction vector output by the first interaction layer based on the eigenvalue vectors corresponding to each continuous feature, and a discrete feature interaction vector output by the second interaction layer based on the eigenvalue vectors corresponding to each discrete feature for multi-order feature interaction. In this case, the representation vector is obtained by concatenating the linear processing result, the discrete feature interaction vector, and the continuous feature interaction vector.

[0085] Please refer to the above for details, which will not be repeated here.

[0086] Then, in step 43, for a target business sample among multiple business samples (any business sample among multiple business samples), multiple business samples with the same label as the target business sample under the target task (referred to as first samples for the convenience of description and distinction) and multiple business samples with different labels from the target business sample under the target task (referred to as second samples for the convenience of description and distinction) are determined; further, based on the representation vectors of the first sample and the target business sample respectively, the similarity between the first sample and the target business sample is calculated (referred to as the first similarity for the convenience of description and distinction), based on which, the first similarity between each first sample and the target business sample is obtained, and these similarities are included in the comparative learning results; based on a similar method, the similarity between the second sample and the target business sample is calculated (referred to as the first similarity for the convenience of description and distinction), and the second similarity between each second sample and the target business sample is included in the comparative learning results of the target business sample; in step 44, based on the first similarity and the second similarity in the comparative learning results, the prediction loss of the target business sample is determined (referred to as the first prediction loss for the convenience of description and distinction, corresponding to the above-mentioned sample comparative learning loss). Here, when there is only one business classification task, the target task is the business classification task; when there are multiple business classification tasks, the target task is formed by one or more of these business classification tasks. For details, please refer to the above and will not be repeated here; and in actual applications, there can be multiple target tasks, and the embodiments of this specification do not make specific limitations, and the specific details can be determined based on actual conditions.

[0087] The above describes in detail the process of determining the first predicted loss of the target business sample under the target task. For each of the multiple business samples, it can be used as a target business sample, and the above steps 43 and 44 are repeated to determine the first predicted loss of each business sample under the target task. In addition, if there are multiple target tasks, the above steps 43 and 44 are repeated for other target tasks to determine the first predicted loss of each business sample under other target tasks; after processing each target task, the first predicted loss of each business sample under each target task can be obtained.

[0088] In addition, based on the first prediction loss of each business sample, the representation model can also be trained in combination with the prediction loss corresponding to each business classification task (for the sake of ease of description and distinction, it is called the second prediction loss, corresponding to the above-mentioned task prediction loss), so that the representation model learns the relationship between similar business events, the difference between different business events, and the information of several tasks.

[0089] Then, in step 44, for any business classification task, the characterization vector corresponding to the business sample is input into the task model corresponding to the task for task prediction, and the predicted value of the business sample for the task is obtained, and it is merged into the prediction result corresponding to the task. After processing multiple characterization vectors, the prediction result of the task can be obtained, which includes the predicted values ​​of multiple business samples for the task. Then in step 44, based on the label values ​​of the multiple business samples for the task, and the predicted values ​​of the multiple business samples in the prediction result, the second prediction loss of the task is determined.

[0090] In addition, the task model may include a coding layer and a prediction layer. Please refer to the above for details and will not be repeated here.

[0091] The above describes in detail how to determine the second predicted loss of a single service classification task. When there are multiple service classification tasks, each service classification task can be processed in the above manner to determine the second predicted loss corresponding to each service classification task.

[0092] Furthermore, considering that the input of the task model is a representation vector, it cannot be trained alone, so the task model needs to be trained.

[0093] In an example, when there is only one business classification task, the task model corresponding to the task can be trained based on the second prediction loss of the task.

[0094] In one example, when there are multiple business classification tasks and the task models are independent of each other, for any business classification task, the task model corresponding to the task can be trained based on the second prediction loss of the task; if multiple task models share a coding layer, the shared coding layer can be trained based on the second prediction losses corresponding to the multiple business classification tasks; and for any business classification task, the prediction layer in the task model corresponding to the task can be trained based on the second prediction loss of the task.

[0095] In addition, when there are multiple business classification tasks, in some possible implementations, the representation model can be trained in the following manner.

[0096] Determine the weight value corresponding to each business classification task; then, for any business classification task, adjust the second predicted loss corresponding to the task based on the weight value corresponding to the task, and determine the adjusted loss; then, based on the adjusted loss corresponding to each business classification task and the first predicted loss corresponding to each business sample, for example, the sum or mean of these losses, obtain the final loss, and train the representation model based on the loss.

[0097] It should be noted that since the data distribution and importance of different business classification tasks are usually different, the prediction losses of different business classification tasks contribute differently to the representation model. The embodiments of this specification adjust the prediction losses by weight values ​​to balance different business classification tasks so that the adjusted losses of different business classification tasks contribute similarly or identically to the representation model, thereby ensuring the retrieval recall capability of the representation vector output by the representation model under multiple tasks.

[0098] Reviewing the above process, in the embodiments of this specification, the representation model is trained by combining supervised contrastive learning and tasks, so that the representation model learns the relationship between similar business events, the differences between different business events, and the information of several tasks, so as to improve the recall ability of the representation vector output by the representation model.

[0099] According to an embodiment of another aspect, a training device for a characterization model is also provided. Figure 5 A schematic diagram of a training device for characterizing a model according to an embodiment is shown, and the device can be deployed in any device, platform or device cluster with data storage, computing and processing capabilities. Figure 5 As shown, the device 500 includes:

[0100] The sample acquisition module 51 is configured to acquire a plurality of business samples, wherein any business sample includes event features of a business event and has label values ​​corresponding to a plurality of business classification tasks;

[0101] The characterization module 52 is configured to use the characterization model to perform event feature encoding processing on the multiple business samples respectively, so as to obtain multiple characterization vectors corresponding to the multiple business samples respectively;

[0102] The comparative learning module 53 is configured to determine, for a target business sample among the multiple business samples, a plurality of first samples having the same label value as the target business sample under the target task among the multiple business classification tasks, and a plurality of second samples having different label values; based on the multiple representation vectors, calculate the similarity between each first sample and each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample;

[0103] The task prediction module 54 is configured to train the representation model based at least on the comparative learning result of the target business sample.

[0104] In various embodiments, each of the above units is specifically configured to perform the above combination Figure 4 The various steps in the described method will not be repeated here.

[0105] Through the above device, the representation model is trained by combining supervised contrastive learning and tasks, so that the representation model learns the relationship between similar business events, the difference between different business events, and the information of several tasks, so as to improve the recall ability of the representation vector output by the representation model.

[0106] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 4 The method described.

[0107] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 4 The method described.

[0108] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0109] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A training method for a representation model, comprising: Acquire multiple business samples, where any business sample includes event features of a business event and has label values ​​corresponding to a plurality of business classification tasks, wherein the business event includes a transaction event and / or a payment event, the event features are fields describing the business event, and the label values ​​are used to describe the classification categories in the business classification tasks; Using the representation model, respectively performing event feature encoding processing on the multiple business samples to obtain multiple representation vectors corresponding to the multiple business samples one by one; For a target business sample among the multiple business samples, determine from the multiple business samples a plurality of first samples having the same label value as the target business sample under a target task among the multiple business classification tasks, and a plurality of second samples having different label values; based on the multiple representation vectors, calculate the similarity between each first sample and each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample; Training the representation model based at least on the comparative learning result of the target business sample; Wherein, the representation model includes an embedding layer, a feature interaction layer, a linear layer and a representation layer; The event feature coding process includes: Using the embedding layer to vectorize the feature values ​​of each of the multiple event features in any business sample, and obtain multiple feature value vectors corresponding to the multiple event features one by one; Using the feature interaction layer to perform feature interaction processing on the multiple event features based on the multiple feature value vectors, and determine an interaction processing result; Using the linear layer to perform linear processing on the multiple eigenvalue vectors to determine a linear processing result; The characterization layer is used to determine a characterization vector corresponding to the business sample based on the interactive processing result and the linear processing result.

2. The method according to claim 1, wherein: The method further comprises: For any task among the plurality of business classification tasks, the plurality of representation vectors are respectively input into a task model corresponding to the task to perform task prediction, and a prediction result is obtained, which includes the prediction value of each of the plurality of business samples for the task; The step of training the representation model based at least on the comparative learning result of the target business sample comprises: The representation model is trained based on at least the prediction results corresponding to the respective business classification tasks, the respective label values ​​of the plurality of business samples, and the comparative learning results of the target business samples.

3. The method according to claim 2, wherein: The method further comprises: For any task among several business classification tasks, a task model corresponding to the task is trained based on the respective prediction values ​​of the multiple business samples in the corresponding prediction results and the label values ​​of the task respectively possessed by the multiple business samples.

4. The method according to claim 2, wherein: The task model includes a coding layer and a prediction layer, and the task prediction includes: For any vector among the multiple representation vectors, the representation vector is processed by the coding layer to achieve information fusion, and a fusion vector is output; and prediction for the task is performed based on the fusion vector by the prediction layer to obtain a predicted value of a business sample corresponding to any vector.

5. The method according to claim 4, wherein: The plurality of service classification tasks correspond to a plurality of task models, the plurality of task models share the coding layer, and the method further comprises: Training the encoding layer based on at least the prediction results corresponding to the respective business classification tasks and the respective label values ​​of the plurality of business samples; The prediction layer is trained based on the prediction results corresponding to the respective business classification tasks and the respective label values ​​of the plurality of business samples.

6. The method according to claim 1, wherein: The multiple event features include discrete features and continuous features, and the interaction processing result includes a continuous feature interaction vector and a discrete feature interaction vector; The representation vector is a vector obtained by concatenating the continuous feature interaction vector, the discrete feature interaction vector and the linear processing result; The feature interaction layer includes a first interaction layer and a second interaction layer; the feature interaction processing includes: Performing feature interaction based on the eigenvalue vector corresponding to the continuous feature through the first interaction layer to determine a continuous feature interaction vector; The second interaction layer performs multi-order feature interaction based on the eigenvalue vector corresponding to the discrete feature to determine a discrete feature interaction vector.

7. The method according to claim 1, wherein: The method further comprises: Acquire structured business data of the business event; A single record in the structured business data is used as a single business sample to determine the multiple business samples.

8. The method according to claim 7, wherein: The structured business data is claims data.

9. The method according to claim 8, wherein: The multiple business classification labels include a first label and a second label, the first label is used to indicate whether a claim is made, and the second label is used to indicate the type of claim case.

10. A training device for a representation model, comprising: A sample acquisition module is configured to acquire a plurality of business samples, wherein any business sample includes an event feature of a business event and has a label value corresponding to each of a plurality of business classification tasks, wherein the business event includes a transaction event and / or a payment event, the event feature is a field describing the business event, and the label value is used to describe a classification category in the business classification task; A characterization module is configured to use the characterization model to perform event feature encoding processing on the multiple business samples respectively, so as to obtain multiple characterization vectors corresponding to the multiple business samples respectively; The comparative learning module is configured to determine, for a target business sample among the multiple business samples, a plurality of first samples having the same label value as the target business sample under a target task among the multiple business classification tasks, and a plurality of second samples having different label values; based on the multiple representation vectors, calculate the similarity between each first sample and each second sample and the target business sample, respectively, to obtain a comparative learning result of the target business sample; A training module, configured to train the representation model based at least on the comparative learning result of the target business sample; Wherein, the representation model includes an embedding layer, a feature interaction layer, a linear layer and a representation layer; The event feature coding process includes: Using the embedding layer to vectorize the feature values ​​of each of the multiple event features in any business sample, and obtain multiple feature value vectors corresponding to the multiple event features one by one; Using the feature interaction layer to perform feature interaction processing on the multiple event features based on the multiple feature value vectors, and determine an interaction processing result; Using the linear layer to perform linear processing on the multiple eigenvalue vectors to determine a linear processing result; The characterization layer is used to determine a characterization vector corresponding to the business sample based on the interactive processing result and the linear processing result.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.

12. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 9 is implemented.

Images