Table splicing method, device, storage medium and system

Abstract

A table splicing method, device, storage medium and system are disclosed. The method comprises: for any one of the secondary tables corresponding to the main table, based on the connection key between the main table and the secondary table, splicing the first label of each main table data object in the main table to the secondary table to obtain a secondary table containing a second label; using a preset label classification model to process the secondary table containing the second label to obtain the prediction result of each secondary table data object in the secondary table; aggregating the prediction result of each secondary table data object in the secondary table to obtain the secondary table statistical information corresponding to the connection key; and splicing the secondary table statistical information into the main table based on the connection key. The table splicing method can efficiently splice the information obtained by integrating the secondary table into the main table, thereby fully utilizing the information in the secondary table.

Description

Technical Field

[0001] This disclosure generally relates to the field of data processing, and more specifically, to a table splicing method, apparatus, storage medium, and system. Background Technology

[0002] In current big data applications, artificial intelligence models are often used to analyze the data. This analysis involves numerous tabular data operations, and table concatenation is an indispensable part of these operations. Specifically, table concatenation combines disparate tables, bringing together the business information contained in various tables into a single table, facilitating subsequent analysis and application of that information.

[0003] However, in related technologies, when the table relationship between the main table and the sub-table is one-to-many, the sub-table is often simply aggregated and then directly appended to the main table. This results in the inability to fully utilize the information in the sub-table during subsequent data analysis. Summary of the Invention

[0004] This disclosure provides a table splicing method, apparatus, storage medium, and system for at least solving some of the above-mentioned problems.

[0005] According to one aspect of this disclosure, a table concatenation method is provided, comprising: for any secondary table corresponding to a primary table, concatenating a first label of each primary table data object in the primary table to the secondary table based on the join key between the primary table and the secondary table, to obtain a secondary table containing a second label; processing the secondary table containing the second label using a preset label classification model to obtain a prediction result for each secondary table data object in the secondary table; aggregating the prediction results for each secondary table data object in the secondary table to obtain secondary table statistics corresponding to the join key; and concatenating the secondary table statistics to the primary table based on the join key.

[0006] Optionally, the first label of any main table data object indicates whether the main table data object belongs to the target category, and the prediction result of any sub-table data object indicates the probability that the sub-table data object belongs to the target category.

[0007] Optionally, the main table data object includes multiple data items, wherein the first label is one of the multiple data items.

[0008] Optionally, the statistical information in the sub-table includes at least one of the following: maximum value, minimum value, median, mean, variance, and count.

[0009] Optionally, the step of processing the sub-table containing the second label using a preset label classification model to obtain the prediction results of each sub-table data object in the sub-table includes: using the sub-table containing the second label as a first training sample set, wherein each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table; using the first training sample set to perform cross-training on the label classification model, so as to predict each first training sample in the first training sample set through the cross-training, and obtain the prediction result of each first training sample in the first training sample set, which is used as the prediction result of each sub-table data object in the corresponding sub-table.

[0010] Optionally, the sample features used for training in the first training sample do not include connection keys.

[0011] Optionally, the step of cross-training the label classification model using the first training sample set includes: dividing the first training sample set into a first number of training subsets, wherein first training samples with the same connection key are divided into the same training subset; and training the first model using a second number of training subsets from the first number of training subsets during each training session, wherein the second number is less than the first number, and wherein the second number of training subsets used in any one training session is not exactly the same as the second number of training subsets used in other training sessions.

[0012] Optionally, the step of predicting each first training sample in the first training sample set through cross-training to obtain the prediction result of each first training sample in the first training sample set includes: for any training session, using the label classification model obtained from that training session, predicting each first training sample in the remaining training subsets of the first number of training subsets that were not used during that training session, to obtain the prediction result of each first training sample in the remaining training subsets, wherein the remaining training subsets are training subsets other than the second number of training subsets.

[0013] Optionally, the main table is a second training sample set used to train the target business model, and the sub-table is an extended sample set. The main table data object is the second training sample in the second training sample set, and the sub-table data object is the extended sample in the extended sample set. The extended sample is used to expand the information of the second training sample with the same connection key.

[0014] Optionally, it further includes: training the target business model based on the concatenated main table to obtain a trained target business model, wherein the trained target business model is used to execute tasks associated with the main table.

[0015] Optionally, the main table is a user information table, the main table data object is the basic public information of the corresponding user, the connection key is the user identifier, and the sub-table includes at least one of the user's transaction information table, user's operation information table, user's consultation information table and user's social evaluation information table.

[0016] According to another aspect of this disclosure, a table splicing apparatus is provided, comprising: a tag transfer unit configured to, for any sub-table corresponding to a main table, splice a first tag of each main table data object in the main table to the sub-table based on the connection key between the main table and the sub-table, to obtain a sub-table containing a second tag; a sub-table prediction unit configured to process the sub-table containing the second tag using a preset tag classification model to obtain a prediction result for each sub-table data object in the sub-table; an information aggregation unit configured to aggregate the prediction results of each sub-table data object in the sub-table to obtain sub-table statistical information corresponding to the connection key; and an information splicing unit configured to splice the sub-table statistical information to the main table based on the connection key.

[0017] Optionally, the first label of any main table data object indicates whether the main table data object belongs to the target category, and the prediction result of any sub-table data object indicates the probability that the sub-table data object belongs to the target category.

[0018] Optionally, the main table data object includes multiple data items, wherein the first label is one of the multiple data items.

[0019] Optionally, the statistical information in the sub-table includes at least one of the following: maximum value, minimum value, median, mean, variance, and count.

[0020] Optionally, the sub-table prediction unit is configured to: use the sub-table containing the second label as a first training sample set, wherein each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table; use the first training sample set to perform cross-training on the label classification model, so as to predict each first training sample in the first training sample set through the cross-training, and obtain the prediction result of each first training sample in the first training sample set, so as to serve as the prediction result of each sub-table data object in the corresponding sub-table.

[0021] Optionally, the sample features used for training in the first training sample do not include connection keys.

[0022] Optionally, the sub-table prediction unit is further configured to: divide the first training sample set into a first number of training subsets, wherein first training samples with the same connection key are divided into the same training subset; and train the first model using a second number of training subsets from the first number of training subsets during each training session, wherein the second number is less than the first number, and wherein the second number of training subsets used in any one training session is not exactly the same as the second number of training subsets used in other training sessions.

[0023] Optionally, the sub-table prediction unit is further configured to: for any training session, use the label classification model obtained from that training session to predict each first training sample in the remaining training subsets of the first number of training subsets that were not used during that training session, and obtain the prediction result for each first training sample in the remaining training subsets, wherein the remaining training subsets are training subsets other than the second number of training subsets.

[0024] Optionally, the main table is a second training sample set used to train the target business model, and the sub-table is an extended sample set. The main table data object is the second training sample in the second training sample set, and the sub-table data object is the extended sample in the extended sample set. The extended sample is used to expand the information of the second training sample with the same connection key.

[0025] Optionally, it further includes: a model training unit, configured to train the target business model based on the concatenated main table to obtain a trained target business model, wherein the trained target business model is used to perform tasks associated with the main table.

[0026] Optionally, the main table is a user information table, the main table data object is the basic public information of the corresponding user, the connection key is the user identifier, and the sub-table includes at least one of the user's transaction information table, user's operation information table, user's consultation information table and user's social evaluation information table.

[0027] According to another aspect of this disclosure, a computer-readable storage medium for storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform the table concatenation method as described above.

[0028] According to another aspect of this disclosure, a system is provided that includes a storage device comprising at least one computing device and at least one storage instruction, wherein the instruction, when executed by the at least one computing device, causes the at least one computing device to perform the table concatenation method as described above.

[0029] The table splicing method, apparatus, storage medium, and system according to the exemplary embodiments of this disclosure can integrate information in the sub-table by passing the tags of the main table to the sub-table and processing the sub-table using a preset classification model, and then splicing the integrated information into the main table. This allows for efficient and full utilization of the information in the sub-table when performing data analysis using the spliced ​​main table in the future.

[0030] Further aspects and / or advantages of the general concept of this disclosure will be set forth in part in the description which follows, and in part will be clear from the description or may be learned by practice of the general concept of this disclosure. Attached Figure Description

[0031] These and / or other aspects and advantages of this disclosure will become clearer and more readily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 This is a flowchart illustrating a table splicing method according to an exemplary embodiment of the present disclosure;

[0033] Figure 2 This is a block diagram illustrating a table splicing device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0034] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the invention as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0035] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0036] The following reference Figure 1 and Figure 2 A detailed description of the table splicing method and apparatus according to exemplary embodiments of the present disclosure is provided.

[0037] Figure 1This is a flowchart illustrating a table splicing method according to an exemplary embodiment of the present disclosure.

[0038] Reference Figure 1 In step S101, for any sub-table corresponding to the main table, based on the connection key between the main table and the sub-table, the first tag of each data object in the main table can be concatenated to the sub-table to obtain a sub-table containing the second tag.

[0039] According to an exemplary embodiment of this disclosure, the master table and the sub-tables have a one-to-many table relationship based on a join key. In one possible implementation, the master table can be associated with multiple sub-tables, and any master table data object can be associated with multiple sub-table data objects. Here, a master table data object refers to a row or column of data in the master table; a sub-table data object is a row or column of data in the sub-table. The master table and sub-tables can be associated through a join key, which refers to the content of a cell in the table, existing in both the master and sub-tables. For example, the master table can contain user data from user 1 to user N, and there can be multiple sub-tables, for example, N sub-tables. Sub-table i can be a transaction data table or behavior data table for user i (where i ranges from 1 to N). Then, the join key between the master table and sub-table i can be understood as the content of the cell containing user i, meaning there is an association between the user identifiers in the master and sub-tables.

[0040] Here, for the main table and sub-table as tabular data, each row of data in the table can be used as a main table data object or a sub-table data object. In this case, for any join key, the first label corresponding to that join key can be appended to at least one row of data in the sub-table corresponding to that join key. Alternatively, each column of data in the table can be used as a main table data object or a sub-table data object. In this case, for any join key, the first label corresponding to that join key can be appended to at least one column of data in the sub-table corresponding to that join key. However, this disclosure does not limit this, and those skilled in the art can set it according to the actual application scenario. Furthermore, the main table data object can include multiple data items, so the first label can be one of the multiple data items. That is, the first label can be customized by those skilled in the art according to actual business needs. For example, when the main table is user transaction data, the transaction type data item can be used as the first label, or the transaction method or transaction amount can be used as the first label. Custom selection is possible in the actual scenario, but this disclosure is not limited to this. By passing the customized first label to the sub-table, the information obtained after processing the sub-table can better meet the actual business needs.

[0041] It should be noted that the tabular data used as the main and secondary tables in this disclosure includes, but is not limited to, data from the following scenarios: equipment parameter data in automatic control scenarios, such as parameters of equipment like mining units, wind turbine generators, and air conditioning systems; interactive data in intelligent question-and-answer scenarios, such as user behavior data; tabular data in business decision-making scenarios in the fintech, healthcare, and municipal sectors, including: marketing data (e.g., coupon data, ad click data, user data, etc.), transaction data, credit data, and commodity price data in the fintech sector; disease diagnosis data and personal health data in the healthcare sector; social security data, environmental facility data, industrial economic data, and transportation facility data in the municipal sector; recommendation data in recommendation business scenarios, such as news data, advertising data, music data, consultation data, and financial product data; search history data in search scenarios, such as web search history, image search history, text search history, and video search history; and behavioral data in abnormal behavior detection scenarios, such as residential electricity data, network traffic data, and operation log data. It should be understood that the data in the above scenarios can be displayed in tabular form or converted into tabular data and then applied to the methods shown in the embodiments of this disclosure, and are only used as examples. This disclosure does not impose any limitations on them.

[0042] Next, in step S102, a preset label classification model can be used to process the sub-table containing the second label to obtain the prediction results of each sub-table data object.

[0043] According to exemplary embodiments of this disclosure, a first label of any primary table data object can represent the category to which the primary table data object belongs. Correspondingly, a second label of any secondary table data object can also represent the category to which the secondary table data object belongs. Thus, the category to which each secondary table data object belongs can be predicted using a label classification model. In one possible implementation, probability can be used to characterize whether a secondary table data object belongs to a certain category. In this case, the probability of each secondary table data object belonging to a certain category can be predicted using a label classification model, so the prediction result of any secondary table data object can represent the probability that the predicted secondary table object belongs to a certain category. As mentioned above, the first label can be customized by those skilled in the art according to actual business needs. In other words, the category of the primary table data object can be defined by those skilled in the art according to the actual application scenario of the data. As an example, when the main table is a user information table and is used for credit rating, the main table data objects can be the basic public information of the corresponding user, the join key can be the user identifier, and the sub-tables can include at least one of the user's transaction information table, user's operation information table, user's consultation information table, and user's social evaluation information table. In this case, the data item in the main table used to represent the corresponding user's credit rating can be used as the first label, so that the first label can represent the credit rating category of the corresponding main table data object. In one possible implementation, the credit rating category of the main table data object can include excellent, good, poor, etc. Correspondingly, the prediction result of each sub-table data object can represent the probability of which level the corresponding user's credit rating belongs to, but this disclosure is not limited to this. By using a preset label classification model to process the sub-table containing the second label, the prediction result of each sub-table data object can accurately characterize which category each sub-table data object belongs to.

[0044] According to another exemplary embodiment of this disclosure, in the case of binary classification, the first label of any main table data object can indicate whether the main table data object belongs to the target category. Correspondingly, the second label of any sub-table data object can also indicate whether the sub-table data object belongs to the target category. Thus, the label classification model can predict whether each sub-table data object belongs to the target category. In one possible implementation, the probability of a sub-table data object belonging to the target category can be characterized by probability. In this case, the label classification model can predict the probability that each sub-table data object belongs to the target category, so the prediction result of any sub-table data object can represent the probability that the predicted sub-table object belongs to the target category. Here, the target category can be defined by those skilled in the art based on the actual application scenario of the data. As an example, in a scenario where the main table is a user information table and is applied to a credit card fraud scenario, the main table data objects can be the basic public information of the corresponding user, the join key can be the user identifier, and the sub-tables can include at least one of the user's transaction information table, user's operation information table, user's consultation information table, and user's social evaluation information table. In this case, the data item in the main table that represents the corresponding user's credit card default status can be used as the first label, so the target category can be credit card default status, which can be used to indicate whether the corresponding user has had credit card default behavior in the past. Correspondingly, the prediction results of each sub-table data object can represent the probability of the corresponding user having credit card default behavior in the future, but this disclosure is not limited to this. By using a preset label classification model to process the sub-tables containing the second label, the prediction results of each sub-table data object can accurately characterize whether each sub-table data object belongs to the target category.

[0045] Next, in step S103, the prediction results of each sub-table data object in the sub-table can be aggregated to obtain the sub-table statistics corresponding to the join key. Then, in step S104, the sub-table statistics can be concatenated to the main table based on the join key. Here, for any join key, the prediction results of each sub-table data object corresponding to that join key can be aggregated to obtain the sub-table statistics corresponding to the prediction results of that join key; then, the sub-table statistics corresponding to the prediction results of that join key can be concatenated to the main table data object corresponding to that join key in the main table. Further, the sub-table statistics may include, but are not limited to, at least one of the following: maximum value, minimum value, median, mean, variance, and count. In other words, when aggregating the prediction results of each sub-table data object, the maximum value, minimum value, median, mean, variance, and count can be obtained from the prediction results corresponding to the same join key, thereby realizing the aggregation of the prediction results of each sub-table data object and obtaining the sub-table statistics corresponding to the join key. However, this disclosure is not limited to this, and those skilled in the art can choose the aggregation method according to the actual situation to obtain the sub-table statistics that meet the business requirements. By aggregating the prediction results of each sub-table data object along the join key, the resulting sub-table statistics become more valuable, making it easier to rationally combine the integrated information into the main table.

[0046] In summary, the embodiments of this disclosure integrate the information of the secondary table through model training and then concatenate the integrated secondary table data into the primary table, thereby enabling the primary table to better utilize the secondary table information in subsequent applications.

[0047] In at least one embodiment of this disclosure, when processing a sub-table containing a second label using a preset label classification model, the prediction results of each sub-table data object can be obtained by cross-training the label classification model. In one possible implementation, the sub-table containing the second label can be used as a first training sample set. Here, each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table. Further, the sample features used for training in the first training samples do not include connection keys, so the information contained in the connection keys does not participate in the training of the label classification model. However, this disclosure does not limit this, and those skilled in the art can determine whether the sample features used for training include connection keys according to the actual situation. Next, the label classification model can be cross-trained using the first training sample set to predict each first training sample in the first training sample set through cross-training, obtaining the prediction result of each first training sample in the first training sample set, which is then used as the prediction result of each sub-table data object in the corresponding sub-table. As an example, the label classification model may include, but is not limited to, at least one of machine learning or deep learning models such as random forest model, support vector machine model, and Wide and Deep model, but this disclosure is not limited thereto. Those skilled in the art can use a suitable model for training according to the actual situation. By training the label classification model using a sub-table containing second labels, the label classification model can learn the distribution of each first training sample based on the second label corresponding to each first training sample, thereby enabling the label classification model to accurately predict the data objects in each sub-table.

[0048] As an example, for the aforementioned cross-training, the first training sample set can be divided into a first number of training subsets, where first training samples with the same connection bond are assigned to the same training subset. Then, during each training iteration, a second number of training subsets from the first number of training subsets can be used to train the first model, where the second number is less than the first number. For example, in the case of five-fold cross-training, the first number is 5 and the second number is 4, but this disclosure is not limited to this; the specific values ​​of the first and second numbers can be set by those skilled in the art according to actual circumstances. Furthermore, the second number of training subsets used in any given training iteration is not exactly the same as the second number of training subsets used in other training iterations. In other words, during each training iteration, a portion of the training subsets can be excluded, and the excluded training subsets are not exactly the same each time. Further still, for any given training iteration, the label classification model obtained from that training iteration can be used to predict each first training sample in the remaining training subsets of the first number of training subsets not used during that training iteration, obtaining the prediction result for each first training sample in the remaining training subsets. Here, the remaining training subsets are training subsets other than the second number of training subsets. By predicting the training subset excluded during training after each training iteration, we can avoid overlap between the training samples and the prediction samples, thus improving the reliability of the prediction results. In this way, we can ultimately obtain the prediction results for all sub-table data objects.

[0049] In at least one embodiment of this disclosure, the main table can be a second training sample set used to train the target business model, and the sub-table can be an extended sample set. Here, the main table data object can be a second training sample in the second training sample set, and the sub-table data object can be an extended sample in the extended sample set. The extended samples can be used to expand the information of the second training samples with the same join key. As an example, when the main table is a user information table, the main table data object is the basic public information of the corresponding user, that is, the second training sample is the user's basic public information. In this case, the sub-table can be a user transaction information table, including multiple transaction information of the user, so that each transaction information can be used as an extended sample to expand the basic public information of the same user, but this disclosure is not limited to this. After expanding the information of the main table through the sub-table, the trained target business model can learn more feature distributions, thereby having better performance.

[0050] According to at least one embodiment of this disclosure, after concatenating the statistical information from the secondary table to the primary table, the target business model can be trained based on the concatenated primary table to obtain a trained target business model. Here, the trained target business model can be used to perform tasks associated with the primary table. Further, the tasks associated with the primary table can include, but are not limited to, identification tasks, prediction tasks, classification tasks, decision-making tasks, etc., and will not be exhaustively listed here. As an example, when the training sample set is user data, the trained target business model can be used to assess whether a user is likely to default on a credit card in the future, or it can be used to assess the user's credit / risk level, but this disclosure is not limited to these.

[0051] It should be understood that the target business model is not necessarily related to the label classification model that processes the sub-table. That is, the target business model and the label classification model that processes the sub-table can be the same model or different models. Those skilled in the art can determine the appropriate model as the target business model based on the actual situation. Training the target business model using the main table obtained by concatenating the statistical information from the sub-table can efficiently and fully utilize the information in the sub-table, enabling the trained target business model to learn more feature distributions and thus better meet the needs of the target business.

[0052] It should be noted that the target business model involved in this disclosure embodiment can be used to predict credit ratings, fraudulent transactions, ad click-through rates, product prices, etc., and can be specifically determined by those skilled in the art based on the target business applied to the main table. In other words, the above-mentioned target business model aims to predict issues related to objects or events in relevant scenarios. For example, it can be used to predict credit ratings, fraudulent transactions, ad click-through rates, product prices, etc., so that the prediction results can be directly used as a basis for decision-making or further combined with other rules to become a basis for decision-making.

[0053] Furthermore, the target business model disclosed herein can be used in scenarios including but not limited to the following:

[0054] Automatic control scenarios include: prediction of mine group regulation operations, prediction of wind turbine generator group regulation operations, and prediction of air conditioning system regulation operations; specifically, for mine groups, a set of regulation operations with high mining efficiency can be predicted; for wind turbine generator groups, a set of regulation operations with high power generation efficiency can be predicted; and for air conditioning systems, a set of regulation operations that meet demand while saving energy can be predicted.

[0055] Intelligent question-and-answer scenarios include: chatbots and intelligent customer service.

[0056] Business decision-making scenarios include those in the fintech, healthcare, and municipal sectors. Fintech scenarios include marketing (such as coupon usage prediction, ad click behavior prediction, and user profiling) and customer acquisition, anti-fraud, anti-money laundering, underwriting and credit scoring, and commodity price prediction. Healthcare scenarios include disease screening and prevention, personalized health management, and assisted diagnosis. Municipal scenarios include social governance and regulatory enforcement, resource, environmental, and facility management, industrial development and economic analysis, public services and livelihood security, and smart cities (allocation and management of various urban resources such as public transportation, ride-hailing services, and shared bicycles).

[0057] Recommended business scenarios include: news, advertisements, music, information, videos, and financial products (such as wealth management and insurance).

[0058] Search scenarios include: web search, image search, text search, video search, etc.

[0059] Abnormal behavior detection scenarios include: detection of abnormal electricity consumption behavior of State Grid customers, detection of malicious network traffic, and detection of abnormal behavior in operation logs.

[0060] In addition, the target business model involved in the embodiments of this disclosure can also be used in scenarios such as privacy computing, multi-party secure computing, federated learning, anonymous query, and secure (privacy) intersection.

[0061] To facilitate understanding of the above embodiments, user data will be used as an example for description, in conjunction with Tables 1 to 6 below.

[0062] As examples, Table 1 shows a main table example and Table 2 shows a secondary table example, both representing user data. Table 1 is a user information table, including information on user ID, gender, occupation, education level, marital status, household registration type, and first tag. Each row of data in Table 1 (i.e., a user's basic information) represents a main table data object. Table 2 is a user transaction information table (only the transaction information corresponding to the user ID "f34cf3be330734cac1cdc5503d06be5b" is shown here), including information on user ID, timestamp, transaction type, transaction amount, and salary income tag. Each row of data in Table 2 (i.e., one transaction record) represents a secondary table data object. Here, the same user can correspond to multiple transaction records. It should be understood that Tables 1 and 2 are merely examples, and those skilled in the art can determine the data types, data dimensions, and data volume in the main and secondary tables according to actual circumstances. This disclosure does not impose any limitations in this regard.

[0063] Table 1 Main Representation Examples

[0064]

[0065] Table 2 Sub-examples

[0066]

[0067] For Tables 1 and 2 above, the connection key between the main table and the sub-table is the user identifier. Therefore, based on the user identifier, the first tag in the main table can be concatenated to the sub-table to obtain a sub-table containing the second tag. Table 3 below shows an example of a sub-table containing the second tag.

[0068] Table 3 contains examples of sub-labels with a second tag.

[0069]

[0070] Next, the sub-table containing the second label can be used as the first training sample set. Using cross-training, the label classification model is trained using this first training sample set. This cross-training is then used to predict each first training sample in the first training sample set, yielding the prediction result for each first training sample. This prediction result serves as the prediction result for the corresponding sub-table data objects. For example, a five-fold cross-training method can be used. The first training sample set is evenly divided into five parts, with one part excluded each time. The model is trained using the other four parts (e.g., but not limited to training a random forest model each time), and the trained label classification model is used to predict the excluded dataset. In this five-fold cross-training process, a total of five training and prediction cycles are performed. Finally, at the end of the five-fold cross-training, the prediction results for all sub-table data objects in the sub-table can be obtained. Table 4 below shows the prediction results for each sub-table data object.

[0071] Table 4 shows examples of prediction results for data objects in each sub-table.

[0072]

[0073] Next, the prediction results of each data object in the sub-table can be aggregated to obtain the sub-table statistics corresponding to the join key, including maximum, minimum, median, mean, variance, and count. Table 5 below shows an example of the sub-table statistics corresponding to the join key.

[0074] Example of statistical information in Table 5 (sub-table)

[0075]

[0076] Next, based on the user identifier, the statistical information from the sub-table can be concatenated into the main table. Table 6 below shows an example of the concatenated main table.

[0077] Table 6 Examples of the main representation after splicing

[0078]

[0079] The table splicing method according to the exemplary embodiments of this disclosure can integrate the information in the sub-table by passing the labels of the main table to the sub-table and processing the sub-table using a preset classification model, and then splicing the integrated information into the main table. This allows for efficient and full utilization of the information in the sub-table when performing data analysis using the spliced ​​main table in the future.

[0080] Figure 2 This is a block diagram illustrating a table splicing device according to an exemplary embodiment of the present disclosure.

[0081] Reference Figure 2 The table splicing device 200 according to an exemplary embodiment of the present disclosure may include a tag transmission unit 210, a sub-table prediction unit 220, an information aggregation unit 230, and an information splicing unit 240.

[0082] The tag passing unit 210 can, for any sub-table corresponding to the main table, concatenate the first tag of each data object in the main table to the sub-table based on the connection key between the main table and the sub-table, to obtain a sub-table containing the second tag.

[0083] The sub-table prediction unit 220 can use a preset label classification model to process the sub-table containing the second label and obtain the prediction results of each sub-table data object in the sub-table.

[0084] The information aggregation unit 230 can aggregate the prediction results of each sub-table data object in the sub-table to obtain the sub-table statistical information corresponding to the join key.

[0085] The information splicing unit 240 can splice the statistical information of the sub-table to the main table based on the connection key.

[0086] According to an exemplary embodiment of this disclosure, the first label of any main table data object can indicate whether the main table data object belongs to the target category, and the prediction result of any sub-table data object can indicate the probability that the sub-table data object belongs to the target category.

[0087] According to an exemplary embodiment of this disclosure, the master table data object may include multiple data items. Here, the first label may be one of the multiple data items.

[0088] According to an exemplary embodiment of this disclosure, the statistical information in the sub-table may include at least one of the following: maximum value, minimum value, median, mean, variance, and count.

[0089] According to an exemplary embodiment of this disclosure, the sub-table prediction unit 220 can use a sub-table containing the second label as a first training sample set; it uses the first training sample set to perform cross-training on the label classification model, so as to predict each first training sample in the first training sample set through cross-training, and obtain the prediction result of each first training sample in the first training sample set, which is used as the prediction result of each sub-table data object in the corresponding sub-table. Here, each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table.

[0090] According to an exemplary embodiment of this disclosure, the sample features used for training in the first training sample may not include connection keys.

[0091] According to an exemplary embodiment of this disclosure, the sub-table prediction unit 220 can divide the first training sample set into a first number of training subsets; during each training iteration, the first model is trained using a second number of training subsets from the first number of training subsets. Here, first training samples with the same connection key are assigned to the same training subset. Further, the second number is less than the first number. Even further, the second number of training subsets used in any given training iteration is not exactly the same as the second number of training subsets used in other training iterations.

[0092] According to an exemplary embodiment of this disclosure, the sub-table prediction unit 220 can also, for any given training iteration, use the label classification model obtained from that training iteration to predict each first training sample in the remaining training subsets of the first number of training subsets that were not used during that training iteration, thereby obtaining a prediction result for each first training sample in the remaining training subsets. Here, the remaining training subsets are training subsets other than the second number of training subsets.

[0093] According to an exemplary embodiment of this disclosure, the main table is a second training sample set used to train the target business model, and the sub-table is an extended sample set. Here, the data objects in the main table are the second training samples in the second training sample set, and the data objects in the sub-table are the extended samples in the extended sample set. The extended samples are used to expand the information of the second training samples that have the same connection key.

[0094] According to an exemplary embodiment of this disclosure, the table splicing apparatus 200 may further include a model training unit (not shown), which can train a target business model based on the spliced ​​main table to obtain a trained target business model. Here, the trained target business model can be used to perform tasks associated with the main table.

[0095] According to an exemplary embodiment of this disclosure, the main table may be a user information table, the main table data object may be the basic public information of the corresponding user, and the connection key may be a user identifier. Here, the sub-table may include at least one of the user's transaction information table, user's operation information table, user's consultation information table, and user's social evaluation information table.

[0096] The table splicing method and apparatus according to the exemplary embodiments of this disclosure can integrate information in the sub-table by passing the tags of the main table to the sub-table and processing the sub-table using a preset classification model, and then splicing the integrated information into the main table. This allows for efficient and full utilization of the information in the sub-table when performing data analysis using the spliced ​​main table in the future.

[0097] The above has been referred to Figure 1 and Figure 2 A table splicing method and apparatus according to exemplary embodiments of the present disclosure are described.

[0098] Figure 2 The individual units in the illustrated table splicing device can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, pure software code, or a module combining software and hardware. Furthermore, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, client, or server).

[0099] In addition, refer to Figure 1 The described table concatenation method can be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium may be provided that stores instructions, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform the table concatenation method according to the present disclosure.

[0100] The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the computer program can also be used to perform additional steps beyond those described above, or to perform more specific processing while performing the above steps. The details of these additional steps and further processing are already described in the reference... Figure 1 The relevant methods were mentioned in the description of the process, so they will not be repeated here to avoid repetition.

[0101] It should be noted that each unit in the table splicing apparatus according to the exemplary embodiments of this disclosure can rely entirely on the operation of a computer program to realize its corresponding function. That is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., a lib library) to realize its corresponding function.

[0102] on the other hand, Figure 2 The units shown can also be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment used to perform the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that the processor can perform the corresponding operation by reading and running the corresponding program code or code segment.

[0103] For example, exemplary embodiments of this disclosure can also be implemented as a computing device including a storage component and a processor, wherein the storage component stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the processor, the table concatenation method according to exemplary embodiments of this disclosure is executed.

[0104] Specifically, the computing device can be deployed on a server or client, or on node devices in a distributed network environment. Furthermore, the computing device can be a PC, tablet, personal digital assistant, smartphone, web application, or other device capable of executing the aforementioned set of instructions.

[0105] Here, the computing device is not necessarily a single computing device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The computing device can also be part of an integrated control system or system manager, or can be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0106] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0107] Some operations described in the table splicing method according to exemplary embodiments of this disclosure can be implemented in software, some operations can be implemented in hardware, and some operations can be implemented in a combination of software and hardware.

[0108] The processor can execute instructions or code stored in one of the storage components, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0109] Storage components can be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, storage components can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. Storage components and the processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the storage component.

[0110] In addition, the computing device may include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be interconnected via a bus and / or network.

[0111] The table splicing method according to exemplary embodiments of this disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logic device or operate according to non-precise boundaries.

[0112] Therefore, refer to Figure 1 The described table concatenation method can be implemented by a system comprising at least one computing device and at least one storage device containing at least one storage instruction.

[0113] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a table concatenation method according to an exemplary embodiment of the present disclosure, and a storage device stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one computing device, a reference is executed. Figure 1 The table concatenation method described.

[0114] The foregoing has described various exemplary embodiments of this disclosure. It should be understood that the foregoing description is exemplary only and not exhaustive, and this disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A table splicing method, characterized in that, include: For any sub-table corresponding to the main table, based on the join key between the main table and the sub-table, the first tag of each data object in the main table is concatenated to the sub-table to obtain a sub-table containing the second tag; The sub-table containing the second label is processed using a preset label classification model to obtain the prediction results of each sub-table data object; The prediction results of each data object in the sub-table are aggregated to obtain the sub-table statistics corresponding to the join key; Based on the connection key, the statistical information of the secondary table is concatenated into the primary table.

2. The table splicing method as described in claim 1, characterized in that, The first label of any main table data object indicates whether the main table data object belongs to the target category, and the prediction result of any secondary table data object indicates the probability that the secondary table data object belongs to the target category.

3. The table splicing method as described in claim 1, characterized in that, The main table data object includes multiple data items, wherein the first label is one of the multiple data items.

4. The table splicing method as described in claim 1, characterized in that, The statistical information in the sub-table includes at least one of the following: maximum value, minimum value, median, mean, variance, and count.

5. The table splicing method as described in claim 1, characterized in that, The process of using a preset label classification model to process the sub-table containing the second label to obtain the prediction results of each sub-table data object includes: The sub-table containing the second label is used as the first training sample set, wherein each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table. The label classification model is cross-trained using the first training sample set to predict each first training sample in the first training sample set, thereby obtaining the prediction result of each first training sample in the first training sample set, which is used as the prediction result of each sub-table data object in the corresponding sub-table.

6. The table splicing method as described in claim 5, characterized in that, The sample features used for training in the first training sample do not include connection bonds.

7. The table splicing method as described in claim 5, characterized in that, The step of cross-training the label classification model using the first training sample set includes: The first training sample set is divided into a first number of training subsets, wherein the first training samples with the same connection key are assigned to the same training subset. During each training session, the label classification model is trained using a second set of training subsets from the first set of training subsets, wherein the second set is less than the first set. In this case, the second number of training subsets used in any one training session are not exactly the same as the second number of training subsets used in other training sessions.

8. The table splicing method as described in claim 7, characterized in that, The step of predicting each first training sample in the first training sample set through the cross-training to obtain the prediction result for each first training sample in the first training sample set includes: For any given training iteration, using the label classification model obtained from that training iteration, predict each first training sample in the remaining training subsets of the first number of training subsets that were not used during that training iteration, and obtain the prediction result for each first training sample in the remaining training subsets, wherein the remaining training subsets are training subsets other than the second number of training subsets.

9. The table splicing method according to any one of claims 1 to 8, characterized in that, The main table is a second training sample set used to train the target business model, and the sub-table is an extended sample set. The main table data object is the second training sample in the second training sample set, and the sub-table data object is the extended sample in the extended sample set. The extended samples are used to expand the information of the second training samples with the same connection key.

10. The table splicing method according to any one of claims 1 to 8, characterized in that, Also includes: Based on the concatenated main table, the target business model is trained to obtain a trained target business model, wherein the trained target business model is used to execute tasks associated with the main table.

11. The table splicing method according to any one of claims 1 to 8, characterized in that, The main table is a user information table, and the data objects in the main table are the basic public information of the corresponding users. The connection key is the user identifier. The secondary tables include at least one of the user's transaction information table, user's operation information table, user's consultation information table, and user's social evaluation information table.

12. A table splicing device, characterized in that, include: The tag passing unit is configured to, for any sub-table corresponding to the main table, concatenate the first tag of each data object in the main table to the sub-table based on the connection key between the main table and the sub-table, to obtain a sub-table containing the second tag; The sub-table prediction unit is configured to process the sub-table containing the second label using a preset label classification model to obtain the prediction results of each sub-table data object in the sub-table. The information aggregation unit is configured to aggregate the prediction results of each data object in the sub-table to obtain the sub-table statistical information corresponding to the join key. The information splicing unit is configured to splice the statistical information of the sub-table into the main table based on the connection key.

13. The table splicing device as described in claim 12, characterized in that, The first label of any main table data object indicates whether the main table data object belongs to the target category, and the prediction result of any secondary table data object indicates the probability that the secondary table data object belongs to the target category.

14. The table splicing device as described in claim 12, characterized in that, The main table data object includes multiple data items, wherein the first label is one of the multiple data items.

15. The table splicing device as described in claim 12, characterized in that, The statistical information in the sub-table includes at least one of the following: maximum value, minimum value, median, mean, variance, and count.

16. The table splicing device as described in claim 12, characterized in that, The sub-table prediction unit is configured as follows: The sub-table containing the second label is used as the first training sample set, wherein each first training sample in the first training sample set corresponds to a corresponding sub-table data object in the sub-table. The label classification model is cross-trained using the first training sample set to predict each first training sample in the first training sample set, thereby obtaining the prediction result of each first training sample in the first training sample set, which is used as the prediction result of each sub-table data object in the corresponding sub-table.

17. The table splicing device as described in claim 16, characterized in that, The sample features used for training in the first training sample do not include connection bonds.

18. The table splicing device as described in claim 16, characterized in that, The sub-table prediction unit is also configured to: The first training sample set is divided into a first number of training subsets, wherein the first training samples with the same connection key are assigned to the same training subset. During each training session, the label classification model is trained using a second set of training subsets from the first set of training subsets, wherein the second set is less than the first set. In this case, the second number of training subsets used in any one training session are not exactly the same as the second number of training subsets used in other training sessions.

19. The table splicing device as described in claim 18, characterized in that, The sub-table prediction unit is also configured to: For any given training iteration, using the label classification model obtained from that training iteration, predict each first training sample in the remaining training subsets of the first number of training subsets that were not used during that training iteration, and obtain the prediction result for each first training sample in the remaining training subsets, wherein the remaining training subsets are training subsets other than the second number of training subsets.

20. The table splicing device as described in any one of claims 12 to 19, characterized in that, The main table is a second training sample set used to train the target business model, and the sub-table is an extended sample set. The main table data object is the second training sample in the second training sample set, and the sub-table data object is the extended sample in the extended sample set. The extended samples are used to expand the information of the second training samples with the same connection key.

21. The table splicing device as described in any one of claims 12 to 19, characterized in that, Also includes: The model training unit is configured to train the target business model based on the concatenated main table to obtain a trained target business model, wherein the trained target business model is used to execute tasks associated with the main table.

22. The table splicing device as described in any one of claims 12 to 19, characterized in that, The main table is a user information table, and the data objects in the main table are the basic public information of the corresponding users. The connection key is the user identifier. The secondary tables include at least one of the user's transaction information table, user's operation information table, user's consultation information table, and user's social evaluation information table.

23. A computer-readable storage medium for storing instructions, characterized in that, When the instruction is executed by at least one computing device, it causes the at least one computing device to perform the table concatenation method as described in any one of claims 1 to 11.

24. A system comprising at least one computing device and at least one storage device for storing instructions, characterized in that, When the instruction is executed by the at least one computing device, it causes the at least one computing device to perform the table concatenation method as described in any one of claims 1 to 11.

Images