Data generation method and apparatus, and electronic device

Abstract

The present disclosure provides a data generation method and device and electronic equipment, relates to the technical field of artificial intelligence, specifically relates to the technical field of data mining and deep learning, and can be applied to financial risk control, intelligent retail and the like. The specific implementation scheme is as follows: source domain data and target domain data are acquired, the source domain data includes first table data and first identification data under a target scene, and the target domain data includes second table data under the target scene; the first table data and the second table data are subjected to feature processing to obtain first compressed features and second compressed features; based on the first compressed features and the second compressed features, first attribution objects and second attribution objects of the second table data are clustered to obtain M clustering groups; based on the first identification data, a first proportion rate of the number of the first attribution objects under a first classification in each clustering group relative to the total number of the first attribution objects is determined; and based on the first proportion rate, second identification data is generated.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of data mining and deep learning, and specifically to a data generation method and device and electronic equipment. BACKGROUND

[0002] In many scenarios such as financial risk control scenarios, it is often necessary to judge a user according to user information and financial activity related information of the user to identify the classification of the sample and provide it for use by downstream tasks.

[0003] Currently, for users without an identifier, a hierarchical labeling method is usually used to identify the classification of the user, such as hierarchical classification and identification according to the financial activities of the user. SUMMARY

[0004] The present disclosure provides a data generation method, device and electronic equipment.

[0005] According to a first aspect of the present disclosure, a data generation method is provided, comprising:

[0006] obtaining source domain data and target domain data, the source domain data comprising first table data and first identification data under a target scenario, the target domain data comprising second table data under the target scenario, the first identification data being used to identify the classification of a first attribution object of the first table data under the target scenario;

[0007] performing feature processing on the first table data and the second table data to obtain first compressed features of the first table data and second compressed features of the second table data;

[0008] based on the first compressed features and the second compressed features, clustering the first attribution object and a second attribution object of the second table data to obtain M clustering groups, the first attribution object and the second attribution object included in different clustering groups being different, M being an integer greater than 1;

[0009] based on the first identification data, determining a first proportion rate of the number of the first attribution object under a first classification in each clustering group relative to the total number of the first attribution object;

[0010] based on the first proportion rate, generating second identification data, the second identification data being used to identify the classification of the second attribution object under the target scenario, a second proportion rate of the number of the second attribution object under the first classification in the clustering group relative to the total number of the second attribution object having a difference value from the first proportion rate less than a preset threshold.

[0011] According to a second aspect of the present disclosure, a data generation device is provided, comprising:

[0012] The acquisition module is configured to acquire source domain data and target domain data, the source domain data comprising first table data and first identification data under a target scene, and the target domain data comprising second table data under the target scene, the first identification data being used to identify a classification of a first attribution object of the first table data under the target scene;

[0013] The feature processing module is configured to perform feature processing on the first table data and the second table data to obtain first compressed features of the first table data and second compressed features of the second table data.

[0014] The clustering module is configured to perform clustering on the first attribution object and a second attribution object of the second table data based on the first compressed features and the second compressed features to obtain M clustering groups, the first attribution object and the second attribution object included in different clustering groups being different, and M being an integer greater than 1.

[0015] The determination module is configured to determine, based on the first identification data, a first proportion of a number of the first attribution object under a first classification in each of the clustering groups relative to a total number of the first attribution object.

[0016] The generation module is configured to generate second identification data based on the first proportion, the second identification data being used to identify a classification of the second attribution object under the target scene, a second proportion of a number of the second attribution object under the first classification in the clustering group relative to a total number of the second attribution object having a difference value from the first proportion less than a preset threshold.

[0017] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory connected with the at least one processor in communication; wherein

[0020] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any of the methods in the first aspect.

[0021] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to enable a computer to perform any of the methods in the first aspect.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements any of the methods in the first aspect.

[0023] The technical solution according to the present disclosure solves the problem of low accuracy of identification data generation for objects without classification identification in the related art, and improves the accuracy of identification data generation for objects without classification identification.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0026] Figure 1 is a flowchart of a data generation method according to a first embodiment of the present disclosure;

[0027] Figure 2 is a structural schematic diagram of a data generation device according to a second embodiment of the present disclosure;

[0028] Figure 3 is a schematic block diagram of an example electronic device to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0030] First Embodiment

[0031] As shown in Figure 1 , the present disclosure provides a data generation method, comprising the following steps:

[0032] Step S101: acquiring source domain data and target domain data, the source domain data comprising first table data and first identification data under a target scene, the target domain data comprising second table data under the target scene, the first identification data being used to identify the classification of a first belonging object of the first table data under the target scene.

[0033] The data generation method in the embodiment relates to the field of artificial intelligence, in particular to the fields of data mining and deep learning, and can be widely applied to financial risk control, intelligent retail and the like. The data generation method of the embodiment can be executed by the data generation apparatus of the embodiment. The data generation apparatus of the embodiment can be configured in any electronic device to execute the data generation method of the embodiment. The electronic device can be a server or a terminal device, which is not limited here.

[0034] The target scenario can be a financial risk control scenario, or other scenarios such as insurance, intelligent retail, and the like. In the financial risk control scenario, the attribution object can be a user, and in the intelligent retail scenario, the attribution object can be an article. In the following embodiments, the target scenario will be described in the financial risk control scenario and the intelligent retail scenario.

[0035] The source domain data can refer to tabular data of attribution objects with classification identifiers, wherein the classification identifier is used to indicate the classification of the attribution object of the tabular data. For example, in the financial risk control scenario, the classification identifier of a user indicates that the user has overdue risk, and for example, in the intelligent retail scenario, the classification identifier of an article indicates that the article has a risk of being unsold.

[0036] For example, in the financial risk control scenario, there are some users whose evaluation period has reached, and these users can be labeled based on their related information such as financial activity information. These users can be referred to as first attribution objects, and the related data of the first attribution objects in the financial risk control scenario (including the identity information, financial activity information, Internet search information, and the like of the first attribution objects) are combined into first tabular data.

[0037] For example, user A registered an account on a financial platform in January, and performed a loan and credit card application on the financial platform. User A performed some financial activities such as repayment and credit card consumption for the loan and credit card. The financial platform can count the loan record, repayment record, and credit card consumption record of user A, and in the case that the evaluation period has reached, such as 6 months, the financial platform can at least label user A based on the financial activity information of user A from January to July to obtain the classification identifier of user A. Correspondingly, the first tabular data of user A in the financial risk control scenario and the classification identifier thereof can be summarized into the source domain data in the financial risk control scenario.

[0038] For example, in the intelligent retail scenario, an item A is sold on an intelligent retail platform, which can count the replenishment and sales of the item A in each retail subzone every month. When the evaluation period (e.g., 6 months) arrives, the intelligent retail platform can at least according to the replenishment and sales of the item A in each retail subzone in 6 months, label the item A to obtain the classification identifier of the item A, to identify whether the item A has a risk of being unsalable, so that corresponding strategies can be taken according to the classification identifier. Correspondingly, the first table data (which can include replenishment and sales data) of the item A in the intelligent retail scenario and the classification identifier thereof are included in the source domain data in the intelligent retail scenario.

[0039] The target domain data can refer to the table data of the belonging object without the classification identifier. In the financial risk control scenario, there are some users, and because the evaluation data available to the financial platform is too little or the evaluation period is not long enough, the users cannot be labeled. The second table data of the users can be included in the target domain data in the financial risk control scenario. In the intelligent retail platform, there are also some items, and because the selling time is not long, the manual or system cannot determine whether the items have a risk of being unsalable. The second table data of the items can be included in the target domain data in the intelligent retail scenario.

[0040] The purpose of the embodiment is to use the source domain data and the target domain data to perform label inference on the second belonging object of the second table data in the target domain data, to generate the classification identifier of the second belonging object, and to improve the accuracy of the identification data generation of the object without the classification identifier.

[0041] The first table data and the second table data are respectively the data of the first belonging object and the second belonging object in the target scenario, which are table type data. For example, in the financial risk control scenario, the table data can include the number of loan applications of a user in 1 year, the number of credit card applications in 1 year, the usage rate of the credit card, the approval rate, etc. In the intelligent retail scenario, the table data can include the replenishment number of an item in 1 year, the sales quantity, etc.

[0042] The first identification data can include the classification identifier of each first belonging object in the target scenario. For example, in the financial risk control scenario, the first identification data can include the classification identifier indicating that a user has a risk of being overdue and the classification identifier indicating that a user does not have a risk of being overdue. In the intelligent retail scenario, the first identification data can include the classification identifier indicating that an item has a risk of being unsalable and the classification identifier indicating that an item does not have a risk of being unsalable.

[0043] The source domain data and the target domain data can be pre-stored or received from other electronic devices, and the acquisition manner is not limited here.

[0044] Step S102: performing feature processing on the first table data and the second table data to obtain first compressed features of the first table data and second compressed features of the second table data.

[0045] In this step, the first table data and the second table data can be processed by a deep learning model to obtain the first compressed features of the first table data and the second compressed features of the second table data.

[0046] The deep learning model can be a deep-learning neural network (DNN) model, a dimension transformation layer + residual network (ResNet) model, or a DeepFFM model.

[0047] In an optional implementation, the first table data and the second table data can be input into the DNN model for feature processing to obtain the first compressed features and the second compressed features.

[0048] In another optional implementation, the first table data and the second table data can be converted into image data by a dimension transformation layer, and then the image data can be input into the ResNet model for feature processing to obtain the first compressed features and the second compressed features.

[0049] In yet another optional implementation, the first table data and the second table data can be discretized respectively, and then the discretized features can be embedded into the DeepFFM model for feature coding to obtain the first compressed features and the second compressed features.

[0050] It should be noted that the deep learning model can be trained based on the source domain data, and when the training is completed, the deep learning model can output the first compressed features, and the second table data can be processed by the deep learning model to obtain the second compressed features. The deep learning model can also be trained based on the source domain data and the target domain data, and when the training is completed, the deep learning model can output the first compressed features and the second compressed features.

[0051] Step S103: clustering the first attribution objects and the second attribution objects based on the first compressed features and the second compressed features to obtain M clustering groups, and the first attribution objects and the second attribution objects included in different clustering groups are different.

[0052] Wherein, M is an integer greater than 1.

[0053] In this step, the first and second belonging objects can be clustered using a Gaussian Mixture Model (GMM) based on the first and second compression features.

[0054] Specifically, the number of clusters can be determined using the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) encapsulated within the GMM model component. After determining the number of clusters M, based on the first and second compression features, the GMM model is used to divide the samples (including the first and second belonging objects) into M groups according to the data distribution, resulting in M ​​cluster groups. Each cluster group can include both the first and second belonging objects simultaneously, and the first and second belonging objects included in different cluster groups are different.

[0055] Step S104: Based on the first identification data, determine the first percentage ratio of the number of the first belonging objects under the first category in each cluster group relative to the total number of the first belonging objects.

[0056] In this step, for each cluster group, the first belonging object in the cluster group can be obtained, and the identification data of the first belonging object in the cluster group can be obtained from the first identification data. The identification data of the first belonging object in the cluster group may include a classification identifier that indicates that the first belonging object is classified as a first category and a classification identifier that indicates that the first belonging object is classified as a second category.

[0057] The classification identifier under the first category can be the classification identifier for positive samples. For example, in a financial risk control scenario, the first category could be a user with overdue payment risk, while in a smart retail scenario, the first category could be an item with unsold inventory risk. Correspondingly, the first proportion of the number of the first attributed objects under the first category relative to the total number of the first attributed objects can be the proportion of positive samples.

[0058] The labels under the first category can also be classification identifiers for negative samples. For example, in a financial risk control scenario, the first category could be that the user has no risk of default; in a smart retail scenario, the first category could be that the item has no risk of being unsold. Correspondingly, the first proportion of the number of objects belonging to the first category relative to the total number of objects belonging to the first category can be the proportion of negative samples.

[0059] For example, in the financial risk control scenario, the first classification can represent that the user has overdue risk, the first attribution object under the first classification can be referred to as a positive sample, the number of first attribution objects that are positive samples in the clustering group and the total number of first attribution objects can be counted, and the number of first attribution objects that are positive samples divided by the total number of first attribution objects can obtain the first proportion of source first attribution objects in the clustering group.

[0060] Step S105: based on the first proportion, generating second identification data, the second identification data is used to identify the classification of the second attribution object in the target scenario, and the second proportion of the number of second attribution objects under the first classification in the clustering group with respect to the total number of second attribution objects is less than a preset threshold.

[0061] In this step, for each clustering group, for the second attribution object, a random generation function can be used to randomly generate labels of 0 and 1, so that the difference between the second proportion of the second attribution object under the first classification in the clustering group and the first proportion is less than a preset threshold. The preset threshold can be set according to the actual situation, and is usually set to be relatively small, so that the second proportion in the clustering group is close to the first proportion. Wherein, 0 can represent the classification identification of negative sample, and 1 can represent the classification identification of positive sample.

[0062] In this embodiment, by using the first compressed feature representing the first table data and the second compressed feature representing the second table data, the first attribution object and the second attribution object are clustered according to the data distribution of the attribution object, and for each clustering group, the proportion of the first attribution object that is a positive sample in the clustering group is calculated, and for the second attribution object in the clustering group, a classification identification of 0 and 1 can be randomly generated, so that the proportion of the second attribution object that is a positive sample is close to the proportion of the first attribution object that is a positive sample. In this way, the global label inference of the second attribution object can be performed based on the global distribution of the first attribution object, so that the proportions of positive samples of the two are close to each other, which is beneficial to the label inference of the singular point and improves the generalization of the label inference, thereby improving the accuracy of the identification data generation of the object without classification identification.

[0063] In actual application, the data generation method of classification identification in the financial risk control scenario can be packaged into an operator, and then the operators can be connected in series according to a fixed process to form an automatic iterative process. The end-to-end label inference task can be completed by configuring the source domain data, and the newly generated user classification identification can be provided to the downstream task for use.

[0064] Optionally, before the step S102, the method further comprises:

[0065] Classify the table data under the target scene based on the data feature type and the data index type of the target scene, to obtain N classification clusters, N being a positive integer;

[0066] Discretize the N data index intervals corresponding to the N classification clusters, to obtain N first embedding matrices corresponding to the N classification clusters;

[0067] The feature processing on the first table data and the second table data to obtain the first compressed feature of the first table data and the second compressed feature of the second table data comprises:

[0068] The feature processing on the first table data and the second table data based on the N first embedding matrices to obtain the first compressed feature and the second compressed feature.

[0069] In the embodiment, the data feature type can represent the feature type of the table data under the target scene, such as statistical data, non-statistical data, and proportional data.

[0070] The table data under the target scene can be classified based on the data feature type and the data index type, to obtain N classification clusters. For example, the table data under the target scene includes 10, namely X1, X2, X3, X4, X5, X6, X7, X8, X9, and X10. X1-X5 are all statistical data, and their indexes are between [0, 10]. X6-X10 are all proportional decimal data, and their indexes are between [0, 1]. Therefore, the table data under the target scene can be divided into two, to obtain two classification clusters.

[0071] The table data under the target scene can be continuous data. Therefore, for each classification cluster, a data index interval is needed, and the data index interval needs to be discretized. Specifically, the data index interval can be binned to obtain at least two data bins, and the at least two data bins are represented by vectors. Each vector represents a data bin, so that a first embedding matrix corresponding to the classification cluster can be obtained.

[0072] For example, the data index interval [0, 10] can be divided into three data bins, [0, 3], [3, 6], and [6, 10], and the data index interval [0, 1] can also be divided into three data bins, [0, 0.3], [0.3, 0.6], and [0.6, 1]. Then, two first embedding matrices can be used to represent the dimensions of the discretized [0, 10] and [0, 1].

[0073] The number of vectors included in the first embedding matrix is the number of data bins, for example, if the data bins are divided into three, then the number of vectors included in the first embedding matrix is three. The dimension of the vector can be set according to the actual situation, which is not limited here.

[0074] It should be noted that in the discretization process, the first embedding matrix can be initially set, and different vectors represent different data bins, which are in one-to-one correspondence. For example, the initial setting is {[0.121, 1.213, 4.222, 3.231]; [-0.232, 1.87, -2.2651, 4.123]; [0.421, -1.231, 0.4123, 2.123]}, from front to back, corresponding to data bins [0, 3], [3, 6], [6, 10], respectively, and the initial setting is {[0.321, 1.2783, 2.222, 1.231]; [-0.632, 2.87, -3.8251, 7.871]; [0.879, -6.658, 0.9817, 5.871]}, from front to back, corresponding to data bins [0, 0.3], [0.3, 0.6], [0.6, 1], respectively.

[0075] Correspondingly, the first table data and the second table data can be processed based on the N first embedding matrices. Specifically, the first table data and the second table data can be characterized based on the N first embedding matrices, and the features obtained by the feature characterization are embedded into the deep learning model for feature coding to obtain the first compressed feature and the second compressed feature.

[0076] For example, for the first table data, X1~X10 take values [2, 4, 7, 2, 4, 0.2, 0.4, 0.7, 0.2, 0.4], after the above two first embedding matrices are characterized, for example, 2 corresponds to data bin [0, 3], and the corresponding vector is [0.121, 1.213, 4.222, 3.231], so 10 feature vectors can be obtained, which are spliced and embedded into the deep learning model for feature coding to obtain the first compressed feature and the second compressed feature.

[0077] In this embodiment, the table data under the target scene is classified based on the data feature type and the data index type of the target scene, the N data index intervals corresponding to the N classification clusters obtained by classification are discretized to obtain N first embedding matrices, and accordingly, the first table data and the second table data can be processed based on the N first embedding matrices to obtain the first compressed feature and the second compressed feature. In this way, the table data with the same data bin value corresponds to the same embedding vector in the first embedding matrix of the same dimension, and when the deep learning model is updated by back propagation, the parameter is updated on the same embedding vector. On the one hand, the discretization of the dimension of the classified continuous data can greatly compress the number of embedding matrices required compared to related technologies, and on the other hand, the training complexity of the deep learning model can be reduced, and the risk of overfitting can be reduced.

[0078] Optionally, the feature processing of the first table data and the second table data based on the N first embedding matrices to obtain the first compressed feature and the second compressed feature comprises:

[0079] The first table data and the second table data are respectively processed based on the N first embedding matrices to obtain the first representation feature of the first table data and the second representation feature of the second table data.

[0080] The first representation feature is input into a first model for feature encoding to obtain a first target feature, and the second representation feature is input into the first model for feature encoding to obtain a second target feature.

[0081] The N first embedding matrices and the network parameters of the first model are updated based on the first target feature, the second target feature and the first identification data.

[0082] The first table data and the second table data are processed based on the updated N first embedding matrices and the network parameters of the first model to obtain the first compressed feature and the second compressed feature.

[0083] In this embodiment, the feature representation of the first table data and the second table data is similar. Taking the first table data as an example, the values of the data in the first table data can be obtained, the data bin where the value is located can be determined, and the embedding vector corresponding to the data bin in the N first embedding matrices is determined as the representation feature of the data in the first table data. The representation features of the data are spliced to obtain the first representation feature of the first table data.

[0084] The first representation feature can be input into the first model for feature encoding to obtain the first target feature, and the second representation feature can be input into the first model for feature encoding to obtain the second target feature. The first model can be a DeepFFM model, and the DeepFFM model can encode the first representation feature and the second representation feature in an existing or new way to obtain the first target feature and the second target feature.

[0085] Subsequently, the network loss value of the first model can be determined based on the first target feature, the second target feature and the first identification data, and the N first embedding matrices and the network parameters of the first model can be updated in a way of backward gradient descent propagation based on the network loss value.

[0086] Correspondingly, the first table data and the second table data can be processed for features based on the updated N first embedding matrices and the network parameters of the first model, and the N first embedding matrices and the network parameters of the first model can be updated iteratively until the updated N first embedding matrices and the network parameters of the first model can accurately extract features representing the first table data and the second table data. At this time, the first model training is completed, and the first compressed feature of the first table data and the second compressed feature of the second table data can be output.

[0087] In the embodiment, the N first embedding matrices are initially set to process the first table data and the second table data for discretization, and then the representation features after the discretization are embedded into the first model for training. During the training process, the N first embedding matrices are updated so that the updated N first embedding matrices can more accurately represent the features of the table data. In the case of completed training, the first model outputs the first compressed feature and the second compressed feature. In this way, by simultaneously updating the N first embedding matrices and the network parameters of the first model, the features of the table data can be more accurately represented, and the accuracy of feature extraction is improved.

[0088] Optionally, the updating of the N first embedding matrices and the network parameters of the first model based on the first target feature, the second target feature and the first identification data comprises:

[0089] performing classification prediction based on the first target feature to obtain classification data of the first attribution object;

[0090] determining a difference value between the classification data and the first identification data to obtain a first loss value of the first model;

[0091] determining a difference value between the first target feature and the second target feature to obtain a second loss value of the first model;

[0092] aggregate the first loss value and the second loss value to obtain a target loss value;

[0093] update the N first embedding matrices and network parameters of the first model based on the target loss value.

[0094] In this embodiment, the classification loss of the source domain data and the feature distance loss between the source domain data and the target domain data can be combined for training the first model.

[0095] Specifically, a fully connected layer can be used to perform classification prediction based on the first target feature to obtain classification data of the first attribution object, a logloss function can be used to determine the difference value between the classification data and the first identification data to obtain the first loss value of the first model, and the first loss value is the classification loss of the source domain data. A Maximum Mean Discrepancy (MMD) function can be used to determine the difference value between the first target feature and the second target feature to obtain the second loss value, and the second loss value is the feature distance loss between the source domain data and the target domain data.

[0096] The two loss values are aggregated to obtain a target loss value, and the N first embedding matrices and the network parameters of the first model are updated based on the target loss value using the backpropagation gradient propagation method. In this way, the target domain data can be fully utilized to train the first model, the training effect of the first model can be improved, and the generation accuracy of the classification identification of the second attribution object can be further improved.

[0097] Optionally, the inputting the first representation feature into the first model for feature encoding to obtain the first target feature comprises:

[0098] crossing the first representation feature to obtain a third target feature;

[0099] extracting the first representation feature to obtain a fourth target feature, the fourth target feature and the third target feature have the same feature length;

[0100] adding the third target feature and the fourth target feature to obtain the first target feature.

[0101] In this embodiment, in the feature encoding process, the first model can use a Factorization Machine (FM) cross layer to cross the first representation feature to obtain the third target feature, and use a double-layer DNN to perform deep operation to extract the first representation feature to obtain the fourth target feature.

[0102] Then, the third target feature and the fourth target feature are added to obtain the first target feature. In this way, the feature extraction of the first table data can be performed in combination with feature cross and deep operation, and the accuracy of the feature extraction can be improved.

[0103] Optionally, the step S102 specifically includes:

[0104] The first table data and the second table data are respectively subjected to feature dimension conversion to obtain a first image feature corresponding to the first table data and a second image feature corresponding to the second table data.

[0105] The first image feature and the second image feature are respectively subjected to a transformation operation to obtain a first image corresponding to the first image feature and a second image corresponding to the second image feature.

[0106] The first image and the second image are respectively input into a second model for feature processing to obtain the first compressed feature and the second compressed feature.

[0107] In this embodiment, the first table data and the second table data can be subjected to feature extraction by a dimension conversion layer + a residual network model (ResNet), and the second model can be a ResNet model.

[0108] Specifically, the dimension conversion layer can be a fully connected layer, which is used to convert the table data features into image features of a specified length to obtain the first image feature and the second image feature.

[0109] The image features after the fully connected layer are converted into images of a size of a*b*c through a reshape operation to obtain the first image and the second image. Wherein, a and b correspond to the pixels of the image, and c corresponds to the number of channels of the image. Then, the first image and the second image are subjected to feature processing by the Resnet model, and the network parameters of the Resnet model are updated based on the features after the feature processing and the first identification data to train the Resnet model. When the Resnet model is trained, the first compressed feature and the second compressed feature are output. In this way, the feature extraction of the table data can be realized.

[0110] Second embodiment

[0111] As shown in Figure 2 The present disclosure provides a data generation apparatus 200, which comprises:

[0112] The acquisition module 201 is configured to acquire source domain data and target domain data, the source domain data comprising first table data and first identification data in a target scene, the target domain data comprising second table data in the target scene, and the first identification data being used to identify a classification of a first attribution object of the first table data in the target scene.

[0113] The feature processing module 202 is configured to perform feature processing on the first table data and the second table data to obtain first compressed features of the first table data and second compressed features of the second table data.

[0114] The clustering module 203 is configured to perform clustering on the first attribution object and a second attribution object of the second table data based on the first compressed features and the second compressed features to obtain M clustering groups, the first attribution object and the second attribution object included in different clustering groups being different, and M being an integer greater than 1.

[0115] The determination module 204 is configured to determine, based on the first identification data, a first proportion rate of a number of the first attribution object in a first classification in each of the clustering groups relative to a total number of the first attribution object.

[0116] The generation module 205 is configured to generate second identification data based on the first proportion rate, the second identification data being used to identify a classification of the second attribution object in the target scene, a second proportion rate of a number of the second attribution object in the first classification in the clustering groups relative to a total number of the second attribution object having a difference value from the first proportion rate less than a preset threshold.

[0117] Optionally, the method further comprises:

[0118] The classification module is configured to classify table data in the target scene based on a data feature type and a data index type of the target scene to obtain N classification clusters, N being a positive integer.

[0119] The discretization processing module is configured to perform discretization processing on N data index intervals corresponding to the N classification clusters to obtain N first embedding matrices corresponding to the N classification clusters.

[0120] The feature processing module 202 is specifically configured to perform feature processing on the first table data and the second table data based on the N first embedding matrices to obtain the first compressed features and the second compressed features.

[0121] Optionally, the feature processing module 202 comprises:

[0122] The feature representation unit is configured to perform feature representation on the first table data and the second table data respectively based on the N first embedding matrices, to obtain first representation features of the first table data and second representation features of the second table data.

[0123] The feature encoding unit is configured to input the first representation features into a first model to perform feature encoding, to obtain first target features, and input the second representation features into the first model to perform feature encoding, to obtain second target features.

[0124] The updating unit is configured to update the N first embedding matrices and network parameters of the first model based on the first target features, the second target features, and the first identification data.

[0125] The feature processing unit is configured to perform feature processing on the first table data and the second table data based on the updated N first embedding matrices and network parameters of the first model, to obtain the first compressed features and the second compressed features.

[0126] Optionally, the updating unit is specifically configured to:

[0127] perform classification prediction based on the first target features, to obtain classification data of the first attribution object;

[0128] determine a difference value between the classification data and the first identification data, to obtain a first loss value of the first model;

[0129] determine a difference value between the first target features and the second target features, to obtain a second loss value of the first model;

[0130] aggregate the first loss value and the second loss value, to obtain a target loss value;

[0131] update the N first embedding matrices and network parameters of the first model based on the target loss value.

[0132] Optionally, the feature encoding unit is specifically configured to:

[0133] perform feature cross on the first representation features, to obtain third target features;

[0134] perform feature extraction on the first representation features, to obtain fourth target features, the fourth target features and the third target features having the same feature length;

[0135] perform addition processing on the third target features and the fourth target features, to obtain the first target features.

[0136] Optionally, the feature processing module 202 is specifically configured to:

[0137] respectively performing feature dimension conversion on the first table data and the second table data to obtain first image features corresponding to the first table data and second image features corresponding to the second table data;

[0138] respectively performing transformation operations on the first image features and the second image features to obtain a first image corresponding to the first image features and a second image corresponding to the second image features;

[0139] inputting the first image and the second image into a second model for feature processing to obtain the first compressed features and the second compressed features.

[0140] The data generation apparatus 200 provided by the present disclosure can implement each process of the data generation method embodiment and achieve the same beneficial effects. To avoid repetition, details are not described here.

[0141] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0142] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0143] Figure 3 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0144] As Figure 3As shown, the device 300 includes a computing unit 301 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0145] A plurality of components in the device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, and the like; an output unit 307, such as various types of displays, speakers, and the like; a storage unit 308, such as a magnetic disk, an optical disk, and the like; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 309 allows the device 300 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0146] The computing unit 301 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 301 performs various methods and processes described above, such as the data generation method. For example, in some embodiments, the data generation method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the data generation method described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the data generation method by any other appropriate means, such as by means of firmware.

[0147] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0148] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0149] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0151] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0152] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0153] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the spirit and scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technology disclosed in the present disclosure, which are not limited herein.

[0154] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and scope of the disclosure. Any modifications, equivalent substitutions, improvements, and the like, made within the spirit and principles of the disclosure, are intended to be included in the scope of the disclosure.

Claims

1. A data generation method, comprising: obtaining source domain data and target domain data, the source domain data comprising first table data and first identification data under a target scene, the target domain data comprising second table data under the target scene, the first identification data being used to identify a classification of a first attribution object of the first table data under the target scene; performing feature processing on the first table data and the second table data to obtain first compressed features of the first table data and second compressed features of the second table data; based on the first compressed features and the second compressed features, clustering the first attribution object and a second attribution object of the second table data to obtain M clustering groups, different clustering groups comprising different first attribution objects and second attribution objects, M being an integer greater than 1; based on the first identification data, determining a first proportion rate of a number of the first attribution objects under a first classification in each of the clustering groups relative to a total number of the first attribution objects; based on the first proportion rate, generating second identification data, the second identification data being used to identify a classification of the second attribution object under the target scene, a second proportion rate of a number of the second attribution objects under the first classification in the clustering groups relative to a total number of the second attribution objects having a difference value from the first proportion rate less than a preset threshold value; the performing feature processing on the first table data and the second table data to obtain the first compressed features of the first table data and the second compressed features of the second table data, comprising: respectively performing feature dimension conversion on the first table data and the second table data to obtain first image features corresponding to the first table data and second image features corresponding to the second table data; respectively performing transformation operations on the first image features and the second image features to obtain first images corresponding to the first image features and second images corresponding to the second image features; inputting the first images and the second images into a second model for feature processing to obtain the first compressed features and the second compressed features.

2. The method of claim 1, before the performing feature processing on the first table data and the second table data to obtain the first compressed features of the first table data and the second compressed features of the second table data, further comprising: based on a data feature type and a data index type of the target scene, classifying table data under the target scene to obtain N classification clusters, N being a positive integer; performing discretization processing on N data index intervals corresponding to the N classification clusters to obtain N first embedding matrices corresponding to the N classification clusters; the performing feature processing on the first table data and the second table data to obtain the first compressed features of the first table data and the second compressed features of the second table data, comprising: based on the N first embedding matrices, performing feature processing on the first table data and the second table data to obtain the first compressed features and the second compressed features.

3. The method of claim 2, wherein, The feature processing is performed on the first table data and the second table data based on the N first embedding matrices, to obtain the first compressed feature and the second compressed feature, including: The first table data and the second table data are respectively characterized based on the N first embedding matrices, to obtain first characterization features of the first table data and second characterization features of the second table data; The first characterization features are input into a first model for feature coding to obtain first target features, and the second characterization features are input into the first model for feature coding to obtain second target features; The network parameters of the N first embedding matrices and the first model are updated based on the first target features, the second target features, and the first identification data; The first table data and the second table data are processed based on the updated N first embedding matrices and the network parameters of the first model, to obtain the first compressed feature and the second compressed feature.

4. The method of claim 3, wherein, The network parameters of the N first embedding matrices and the first model are updated based on the first target features, the second target features, and the first identification data, including: Classification data of the first attribution object is obtained by performing classification prediction based on the first target features; A difference value between the classification data and the first identification data is determined to obtain a first loss value of the first model; A difference value between the first target features and the second target features is determined to obtain a second loss value of the first model; The first loss value and the second loss value are aggregated to obtain a target loss value; The network parameters of the N first embedding matrices and the first model are updated based on the target loss value.

5. The method of claim 3, wherein, The first characterization features are input into a first model for feature coding to obtain first target features, including: The first characterization features are cross-featured to obtain third target features; The first characterization features are extracted to obtain fourth target features, and the fourth target features and the third target features have the same feature length; The third target features and the fourth target features are added to obtain the first target features.

6. A data generation apparatus, comprising: An acquisition module configured to acquire source domain data and target domain data, the source domain data including first table data and first identification data in a target scene, and the target domain data including second table data in the target scene, the first identification data being used to identify a classification of a first attribution object of the first table data in the target scene; A feature processing module configured to perform feature processing on the first table data and the second table data to obtain a first compressed feature of the first table data and a second compressed feature of the second table data. The clustering module is configured to cluster the first attribution object and a second attribution object of the second table data based on the first compressed feature and the second compressed feature, to obtain M clustering groups, different clustering groups include different first attribution objects and second attribution objects, and M is an integer greater than 1. The determining module is configured to determine, based on the first identification data, a first proportion of the number of the first attribution objects in each clustering group under a first classification relative to the total number of the first attribution objects. The generating module is configured to generate second identification data based on the first proportion, the second identification data being used to identify the classification of the second attribution object under the target scene, and a second proportion of the number of the second attribution objects in the clustering group under the first classification relative to the total number of the second attribution objects having a difference value from the first proportion less than a preset threshold. The feature processing module is specifically configured to: perform feature dimension conversion on the first table data and the second table data respectively, to obtain first image features corresponding to the first table data and second image features corresponding to the second table data; perform transformation operations on the first image features and the second image features respectively, to obtain a first image corresponding to the first image features and a second image corresponding to the second image features; input the first image and the second image into a second model respectively for feature processing, to obtain the first compressed feature and the second compressed feature.

7. The apparatus of claim 6, further comprising: a classification module configured to classify table data under the target scene based on data feature types and data indicator types of the target scene, to obtain N classification clusters, N being a positive integer; a discretization processing module configured to perform discretization processing on N data indicator intervals corresponding to the N classification clusters, to obtain N first embedding matrices corresponding to the N classification clusters; the feature processing module is specifically configured to perform feature processing on the first table data and the second table data based on the N first embedding matrices, to obtain the first compressed feature and the second compressed feature.

8. The apparatus of claim 7, wherein, The feature processing module comprises: a feature representation unit configured to perform feature representation on the first table data and the second table data based on the N first embedding matrices, to obtain first representation features of the first table data and second representation features of the second table data; a feature encoding unit configured to input the first representation features into a first model for feature encoding, to obtain first target features, and input the second representation features into the first model for feature encoding, to obtain second target features; an updating unit configured to update the N first embedding matrices and network parameters of the first model based on the first target features, the second target features, and the first identification data. The feature processing unit is configured to perform feature processing on the first table data and the second table data based on the updated N first embedding matrices and network parameters of the first model, to obtain the first compressed feature and the second compressed feature.

9. The apparatus of claim 8, wherein, The updating unit is specifically configured to: perform classification prediction based on the first target feature, to obtain classification data of the first attribution object; determine a difference value between the classification data and the first identification data, to obtain a first loss value of the first model; determine a difference value between the first target feature and the second target feature, to obtain a second loss value of the first model; aggregate the first loss value and the second loss value, to obtain a target loss value; update the N first embedding matrices and the network parameters of the first model based on the target loss value.

10. The apparatus of claim 8, wherein, The feature encoding unit is specifically configured to: perform feature intersection on the first representation feature, to obtain a third target feature; perform feature extraction on the first representation feature, to obtain a fourth target feature, the fourth target feature and the third target feature having the same feature length; perform addition processing on the third target feature and the fourth target feature, to obtain a first target feature. 11.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method according to any one of claims 1-5. 13.A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.

Images