Feature data generation method, apparatus, device, and medium

Abstract

This application discloses a method, apparatus, device, and medium for generating feature data. This application relates to the fields of artificial intelligence and big data technology. The method includes: obtaining model usage requirement information from participating parties; determining requirement data parameters based on the model usage requirement information; matching at least one target data table from different dimensions of data tables to be matched based on different fields to be matched in the requirement data parameters; and generating target feature data based on feature data in the target data table. This approach, by introducing data tables to be matched from different dimensions to determine at least one target data table, improves the comprehensiveness of the determined target data table, thereby improving the comprehensiveness of the subsequent generation of target feature data based on the feature data in the target data table, and contributing to improving the accuracy of subsequent data processing results based on the target feature data.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and big data technology, and in particular to a method, apparatus, device and medium for generating feature data. Background Technology

[0002] Feature data is involved in the training and use of models. In existing technologies, feature data is selected or generated specifically based on factors such as the model's category or function. However, feature data generated from a single dimension is not comprehensive enough, which affects the results of its use. Summary of the Invention

[0003] This application provides a feature data generation method, apparatus, device, and medium to improve the comprehensiveness of feature data.

[0004] In a first aspect, embodiments of this application provide a method for generating feature data, including:

[0005] Obtain information on the model usage requirements of the participants;

[0006] Based on the model's usage requirements, determine the required data parameters;

[0007] Based on the different fields to be matched in the requirements data parameters, match at least one target data table from the data tables to be matched in different dimensions;

[0008] Generate target feature data based on the feature data in the target data table.

[0009] Secondly, embodiments of this application also provide a feature data generation apparatus, comprising:

[0010] The requirement information acquisition module is used to acquire the model usage requirements of the participants;

[0011] The requirement data parameter determination module is used to determine the requirement data parameters based on the requirement information used in the model.

[0012] The target data table matching module is used to match at least one target data table from different dimensions of the data tables to be matched based on different fields to be matched in the required data parameters.

[0013] The target feature data generation module is used to generate target feature data based on the feature data in the target data table.

[0014] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the feature data generation methods provided in the first aspect of this application.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the feature data generation methods provided in the first aspect of this application.

[0016] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements any of the feature data generation methods provided in the first aspect of this application.

[0017] This application embodiment obtains model usage requirement information from participating parties; determines requirement data parameters based on the model usage requirement information; matches at least one target data table from different dimensions of target data tables based on different fields to be matched in the requirement data parameters; and generates target feature data based on the feature data in the target data table. The above scheme, by introducing target data tables from different dimensions to determine at least one target data table, improves the comprehensiveness of the determined target data table, thereby improving the richness and comprehensiveness of the target feature data generated subsequently based on the feature data in the target data table; simultaneously, it helps improve the accuracy of subsequent data processing results based on the target feature data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a feature data generation method provided in an embodiment of this application;

[0020] Figure 2 This is a flowchart of another feature data generation method provided in the embodiments of this application;

[0021] Figure 3 This is a flowchart of another feature data generation method provided in the embodiments of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a feature data generation device provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device that implements a feature data generation method according to an embodiment of this application. Detailed Implementation

[0024] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. The acquisition, storage, use, and processing of data such as model usage requirements, requirement data parameters, participation frequency, contribution level, and historical usage frequency in the technical solution of this application all comply with relevant national laws and regulations.

[0026] The feature data generation methods and apparatuses provided in this application are applicable to application scenarios where feature data is generated from data acquired from multiple dimensions. The feature data generation methods provided in this application can be executed by a feature data generation apparatus, which can be implemented in software and / or hardware and specifically configured in an electronic device with certain computing and storage capabilities.

[0027] To facilitate understanding, the method for generating feature data will be explained in detail first.

[0028] See Figure 1 The feature data generation method shown includes:

[0029] S110. Obtain information on the model usage requirements of the participants.

[0030] Here, "participant" refers to the end party that can provide model usage requirement information needed during the feature data generation process. Model usage requirement information refers to the conditional requirements used to indicate the generated feature data. Specifically, model usage requirement information can be used to indicate at least one of the following: the type of feature data to be generated and the content of the feature data.

[0031] This application does not limit the method for obtaining the model usage requirements information of the participants in its embodiments; the method can be set by technical personnel based on experience. In an optional embodiment, the model usage requirements information of the participants can be obtained manually.

[0032] To improve the efficiency of obtaining model usage requirement information, this application also provides a method for automatically obtaining model usage requirement information. In another optional embodiment, model description data of the models required by the participants can be obtained; model information of preset feature fields can be extracted from the model description data to obtain model usage requirement information.

[0033] Here, model description data refers to data used to describe the model from at least one dimension. This application embodiment does not limit the content of the model description data, and it can be set by those skilled in the art based on experience or multiple experiments. For example, model description data may be the model's basic parameter data; or, model description data may also be data used to describe the model, fed back by participating parties based on their needs for model use.

[0034] In this context, preset feature fields refer to fields pre-set for feature extraction. This application embodiment does not impose any limitations on preset feature fields, which can be set by technical personnel based on experience. For example, preset feature fields may include model name, model category, participant category, model application scenario, and data granularity.

[0035] For example, the participant dimension can include categories such as private, micro and small enterprises, or non-retail. Application scenarios for the model can include resource transfers or credit cards. Data granularity can include debt, contracts, and customers.

[0036] Specifically, the process involves obtaining model description data from the participants regarding the demand model; extracting model information from the model description data based on preset feature fields to obtain model usage requirement information.

[0037] Understandably, by introducing preset feature fields, model information is extracted from the model description data to obtain model usage requirement information, thus achieving automated acquisition of model usage requirement information and improving the efficiency of acquiring model usage requirement information.

[0038] S120. Determine the required data parameters based on the model usage requirements information.

[0039] Here, the demand data parameters refer to the parameters corresponding to the demand information used by the model. This application embodiment does not limit the configuration method of the demand data parameters; they can be set by technical personnel based on experience. For example, technical personnel can dynamically set or adjust them according to the needs of the scenario.

[0040] For example, the required data parameters may include data granularity, granularity name, participant category, application scenario category, physical layer table identifier, physical layer table primary key field name, and physical layer table foreign key field name. The granularity name may include both the English and Chinese names. The physical layer table identifier may include the Chinese and English names of the physical layer table, and the physical layer table data date field name.

[0041] It should be noted that data granularity, participant category, and application scenario category are the main granularity of any model; the foreign key field name in the physical layer table is the cross-granularity association feature.

[0042] Specifically, based on the model usage requirement information, the requirement data parameters corresponding to each piece of information in the model usage requirement information are determined from the pre-set requirement data parameters.

[0043] S130. Based on the different fields to be matched in the required data parameters, match at least one target data table from the data tables to be matched in different dimensions.

[0044] In this context, the field to be matched refers to a field that can be used for feature data matching. This application embodiment does not impose any limitations on the field to be matched, and it can be set by a technician based on experience. The data table to be matched refers to the physical layer table used to store feature data. The target data table refers to the physical layer table that includes the field to be matched.

[0045] This application does not specifically limit the method for determining the target data table based on the field to be matched. In one optional embodiment, the field to be matched includes a primary key field; correspondingly, matching at least one target data table from different data tables to be matched based on different fields to be matched in the required data parameters includes: matching at least one target data table from different data tables to be matched based on the primary key field.

[0046] In another alternative embodiment, the field to be matched includes a foreign key field; correspondingly, matching at least one target data table from different data tables to be matched based on different fields to be matched in the required data parameters includes: matching at least one target data table from different data tables to be matched based on the foreign key field.

[0047] In another alternative embodiment, the fields to be matched include a primary key field and a foreign key field; accordingly, matching at least one target data table from different data tables to be matched based on different fields to be matched in the required data parameters includes: matching at least one target data table from different data tables to be matched based on the primary key field; and matching at least one target data table from different data tables to be matched based on the foreign key field.

[0048] The primary key field can be the name of the primary key field in the table to be matched. The foreign key field can be the name of the foreign key field in the table to be matched. This application does not impose any limitations on the specific content of the primary key field and foreign key field; they can be set by those skilled in the art based on experience.

[0049] Specifically, the primary key field is matched with the primary key field in the data table to be matched to obtain at least one target data table; and the foreign key field is matched with the primary key field in the data table to be matched to obtain at least one target data table; the target data tables obtained by the above two methods are merged to obtain the target data table corresponding to the field to be matched.

[0050] It should be noted that the primary key field and foreign key field in this application embodiment are two different dimensions of data. The target data table is determined based on the primary key field and foreign key field, which realizes the determination of the target data table based on cross-granularity feature data, realizes the association of feature data of different dimensions, and improves the accuracy of the determined target data table.

[0051] Understandably, by introducing primary key and foreign key fields, the target data table is determined jointly based on the primary key and foreign key fields. This avoids omissions when determining the target data table based on a single field to be matched, improves the comprehensiveness of the determined target data table, and in turn improves the comprehensiveness of the target feature data determined based on the target data table.

[0052] S140. Generate target feature data based on the feature data in the target data table.

[0053] Here, target feature data refers to feature data generated based on feature data in the target data table. For example, target feature data can be all feature data in the target data table; or, target feature data can be at least a portion of the feature data in the target data table.

[0054] It should be noted that the target feature data generated in the embodiments of this application can be used to train or use the model to improve the accuracy of model training or model prediction.

[0055] This application embodiment obtains model usage requirement information from participating parties; determines requirement data parameters based on the model usage requirement information; matches at least one target data table from different dimensions of target data tables based on different fields to be matched in the requirement data parameters; and generates target feature data based on the feature data in the target data table. The above scheme, by introducing target data tables from different dimensions to determine at least one target data table, improves the comprehensiveness of the determined target data table, thereby improving the richness and comprehensiveness of the target feature data generated subsequently based on the feature data in the target data table; simultaneously, it helps improve the accuracy of subsequent data processing results based on the target feature data.

[0056] Based on the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the mechanism for determining the target data table is optimized and improved.

[0057] Furthermore, the operation of "matching at least one target data table from different dimensions of the data tables to be matched based on different fields to be matched in the demand data parameters" is refined to "matching different fields to be matched in the demand data parameters with the matchable fields in the corresponding feature field information tables of different data tables to be matched; and taking the data table to which the successfully matched field belongs as the target data table," thereby improving the mechanism for determining the target data table. It should be noted that parts not detailed in this embodiment can be found in the descriptions of other embodiments.

[0058] See Figure 2 The feature data generation method shown includes:

[0059] S210. Obtain information on the model usage requirements of the participants.

[0060] S220. Determine the required data parameters based on the model usage requirements information.

[0061] S230. Match the different fields to be matched in the requirement data parameters with the matching fields in the corresponding feature field information table of the different data tables to be matched.

[0062] The feature field information table refers to the physical layer table used to store feature fields. Specifically, the feature field information table can include the feature, feature name, physical layer table identifier, feature field type, physical layer table data date field name, and physical layer table primary key field name. The feature name can include both the Chinese and English names of the feature. The physical layer table identifier can include the English, Chinese, and physical layer table data date field names.

[0063] Here, a matchable field refers to a feature field that can be used for field matching. For example, a matchable field can be the primary key field name of a physical layer table.

[0064] Specifically, identify the different fields to be matched in the required data parameters, as well as the matching fields in the corresponding feature field information table of the data table to be matched; match the fields to be matched with the matching fields, and determine the target data table based on the matching results.

[0065] S240. Use the table of data to be matched, which contains the matched fields that have been successfully matched, as the target table.

[0066] Specifically, if the field to be matched successfully matches the matching field, the table to which the matching matching field belongs will be used as the target table; if the field to be matched fails to match the matching field, the table to which the matching matching field belongs will be prohibited from being used as the target table.

[0067] It should be noted that the embodiments of this application do not limit the method for determining whether a match is successful or unsuccessful; it can be set by a technician based on experience. For example, the determination can be based on the similarity score. Optionally, if the similarity score of the matching result is greater than a set threshold, the match is considered successful; otherwise, the match fails. The embodiments of this application do not limit the size of the set threshold; it can be set by a technician based on experience, or determined repeatedly through numerous experiments.

[0068] S250. Generate target feature data based on the feature data in the target data table.

[0069] In an alternative embodiment, a technician can generate target feature data from the feature data of the target data table as needed or under preset conditions.

[0070] To improve the accuracy of the generated target feature data, in another optional embodiment, the feature field information table also includes feature identification information; correspondingly, generating target feature data based on the feature data in the target data table includes: selecting target feature data from the target data table based on the feature identification information.

[0071] In this context, feature identification information refers to identity information used to indicate feature data. This application embodiment does not impose any limitations on feature identification information, which can be set by those skilled in the art based on experience. For example, feature identification information may include at least one of the following: feature, feature name, physical layer table identifier, and feature field type. The feature name may include the English name and Chinese name of the feature. The physical layer table identifier may include the English name, Chinese name, and date field name of the physical layer table data.

[0072] It is understandable that by introducing feature identification information and selecting target feature data based on the feature identification information, the target feature data can be determined from multiple dimensions, thereby improving the accuracy of the selected target feature data.

[0073] In an optional embodiment, selecting target feature data from the target data table based on feature identification information includes: directly filtering the feature data in the target feature table based on feature identification information to obtain the target feature data.

[0074] To avoid omissions, the table structure of the target data table can be identified in advance, and then the target feature data can be determined based on the feature identification information. In another optional embodiment, selecting target feature data from the target data table based on the feature identification information includes: identifying the table structure of the target data table to obtain different candidate feature fields; selecting the target feature field from each candidate feature field based on the feature identification information; and obtaining the target feature data under the target feature field.

[0075] Candidate feature fields refer to the feature fields stored in the target data table. Target feature fields refer to feature fields that are associated with feature identification information.

[0076] Specifically, the table structure of the target data table is identified to determine all candidate feature fields in the target data table; target feature fields that match the feature identification information are selected from each candidate feature field; and target feature data under the target feature fields are obtained.

[0077] Understandably, by introducing candidate feature fields, a comprehensive identification of the target feature data table is achieved, avoiding the omission of target feature fields, improving the comprehensiveness of the identified target feature fields, and thus improving the comprehensiveness of the target feature data.

[0078] The feature data generation scheme provided in this application refines the operation of matching at least one target data table from different dimensions of matchable data tables based on different matchable fields in the requirement data parameters. This is further refined into matching different matchable fields in the requirement data parameters with matchable fields in the corresponding feature field information tables of different matchable data tables. The matchable data table to which the successfully matched matchable field belongs is used as the target data table, thus improving the target data table determination mechanism. By introducing matchable fields and matching matchable fields between matchable fields, the scheme determines the target data table based on the matching results, avoiding inaccuracies in the determined target data table and improving its accuracy.

[0079] Based on the above technical solutions, this application also provides an optional embodiment. In this optional embodiment, the generation mechanism of target feature data has been optimized and improved.

[0080] Furthermore, the operation of "generating target feature data based on feature data in the target data table" is refined to "obtaining feature evaluation data of feature fields belonging to different feature data in the target data table; combining feature data under at least one feature field in the target data table based on the feature evaluation data to obtain target feature data," thereby improving the target feature data generation mechanism. It should be noted that parts not detailed in this embodiment can be found in other embodiments.

[0081] See Figure 3 The feature data generation method shown includes:

[0082] S310. Obtain information on the model usage requirements of the participants.

[0083] S320. Determine the required data parameters based on the model usage requirements information.

[0084] S330. Based on the different fields to be matched in the required data parameters, match at least one target data table from the data tables to be matched in different dimensions.

[0085] S340. Obtain the feature evaluation data of the feature fields to which different feature data belong in the target data table.

[0086] Feature evaluation data refers to data used to quantify the importance of feature fields. This application does not limit the method for determining feature evaluation data; it can be set by a technician based on experience.

[0087] In one optional embodiment, the feature evaluation data is determined as follows: the frequency of participation of the feature fields to which different feature data belong in the target data table during model training is obtained; and the feature evaluation data of the corresponding feature fields is determined based on the participation frequency.

[0088] The frequency of participation refers to the number of times a feature field participates during model training.

[0089] In another optional embodiment, the feature evaluation data is determined in the following way: the contribution of the feature fields to which different feature data belong in the target data table during model training is obtained; and the feature evaluation data of the corresponding feature fields is determined according to the contribution.

[0090] The degree of contribution refers to the extent to which the feature fields play a role in parameter adjustment during model training.

[0091] In another alternative embodiment, the feature evaluation data is determined as follows: the frequency of participation and degree of contribution of the feature fields to which different feature data belong in the target data table during model training are obtained; and the feature evaluation data of the corresponding feature fields are determined based on the frequency of participation and degree of contribution.

[0092] Understandably, by introducing participation frequency and / or contribution level to determine feature evaluation data, the accuracy of determining feature evaluation data is improved.

[0093] In this embodiment of the application, when determining feature evaluation data based on contribution level, there may be situations where the same feature field has at least two contribution levels. To improve the accuracy of the determined feature evaluation data, in an optional embodiment, if the number of models training the same feature field is at least two, then determining the feature evaluation data of the corresponding feature field based on contribution level includes: sorting the at least two contribution levels and selecting the larger contribution level as the basis for determining the feature evaluation data of the feature field. Preferably, the largest contribution level is selected as the basis for determining the feature evaluation data of the feature field.

[0094] To improve the accuracy of the determined feature evaluation data, in another optional embodiment, contribution weights can be introduced to determine the feature evaluation data. Specifically, if at least two models participate in the training of the same feature field, the feature evaluation data for the corresponding feature field is determined according to the degree of contribution. This includes: determining the contribution weight of the same feature field during training of different models based on the historical usage frequency of different models; and weighting the degree of contribution under the same feature field according to the contribution weights to obtain the feature evaluation data for the corresponding feature field.

[0095] The contribution weight quantifies the degree of contribution of a feature field during model training. Historical usage frequency refers to the number of times the model has been used in the past.

[0096] It should be noted that the weighting method in this application embodiment is not limited in any way, and can be set by those skilled in the art based on experience. For example, the contribution levels under the same feature field can be weighted directly according to the contribution weight to obtain the feature evaluation data under the corresponding feature field; or, at least one contribution level can be deleted according to the contribution weight, and the remaining contribution levels can be weighted according to the corresponding contribution weight to obtain the feature evaluation data under the corresponding feature field.

[0097] Understandably, by introducing contribution weights, the situation where the feature evaluation data cannot be accurately determined when the contribution of the same feature field is too large is avoided, thus improving the accuracy of determining the feature evaluation data.

[0098] S350. Based on the feature evaluation data, combine the feature data under at least one feature field in the target data table to obtain the target feature data.

[0099] The embodiments of this application do not limit the combination method in any way, and can be set by technicians based on experience.

[0100] In this embodiment of the application, feature data under at least one feature field in the target data table can be filtered according to the frequency of use or the proportion of use of feature data; the filtered feature data are combined to obtain target feature data.

[0101] In this embodiment of the application, feature data can also be filtered based on the data to be evaluated. Optionally, based on feature evaluation data, feature data under at least one feature field in the target data table can be combined to obtain target feature data, including: filtering feature data under at least one feature field in the target data table based on the data to be evaluated; and combining the filtered feature data to obtain target feature data.

[0102] Understandably, by filtering feature data based on the data to be evaluated, target feature data can be obtained, thus avoiding the impact of unimportant feature data on subsequent processing and reducing resource waste.

[0103] In one optional embodiment, based on feature evaluation data, feature data under at least one feature field in the target data table are combined to obtain target feature data, including: sorting at least one feature field in the target feature table according to the data to be evaluated; and combining feature data under at least one feature field in the target feature table according to the sorting result to obtain target feature data.

[0104] Understandably, by sorting the feature fields according to the data to be evaluated and then combining the feature data under the feature fields to obtain the target feature data, the deletion of the required feature data is avoided, thus improving the comprehensiveness of the target feature data. At the same time, the obtained target feature data can be filtered according to the user's needs, making it more convenient.

[0105] In one optional embodiment, based on feature evaluation data, feature data under at least one feature field in the target data table are combined to obtain target feature data, including: filtering at least one feature field in the target feature table based on the data to be evaluated; sorting the filtered feature data according to the data to be evaluated; and combining the feature data under at least one feature field in the target feature table according to the sorting result to obtain target feature data.

[0106] It is understandable that by filtering and / or sorting the feature fields, the target feature data is obtained, avoiding the influence of unimportant feature data on the target feature data, improving the accuracy of the determined target feature data, and improving the applicability of the target feature data.

[0107] This application embodiment refines the process of generating target feature data based on feature data in the target data table into obtaining feature evaluation data for feature fields belonging to different feature data in the target data table; and combining feature data under at least one feature field in the target data table based on the feature evaluation data to obtain the target feature data, thereby improving the target feature data generation mechanism. The above scheme, by introducing feature evaluation data to determine the target feature data, quantifies the importance of feature fields, avoids the influence of irrelevant feature data on the target feature data, and improves the accuracy of the determined target feature data.

[0108] As an implementation of the above-mentioned feature data generation methods, this application also provides an optional embodiment of an execution device for implementing the feature data generation methods.

[0109] See Figure 4 The illustrated feature data generation device includes: a demand information acquisition module 410, a demand data parameter determination module 420, a target data table matching module 430, and a target feature data generation module 440. Among them,

[0110] The requirement information acquisition module 410 is used to acquire the model usage requirement information of the participants;

[0111] The requirement data parameter determination module 420 is used to determine the requirement data parameters based on the requirement information used in the model.

[0112] The target data table matching module 430 is used to match at least one target data table from different dimensions of the data tables to be matched based on different fields to be matched in the required data parameters.

[0113] The target feature data generation module 440 is used to generate target feature data based on the feature data in the target data table.

[0114] This application embodiment obtains model usage requirement information from participating parties through a requirement information acquisition module; determines requirement data parameters based on the model usage requirement information through a requirement data parameter determination module; matches at least one target data table from different dimensions of data tables to be matched based on different fields to be matched in the requirement data parameters; and generates target feature data based on the feature data in the target data table through a target feature data generation module. This solution, by introducing data tables to be matched from different dimensions and determining at least one target data table, improves the comprehensiveness of the determined target data table, thereby enhancing the richness and comprehensiveness of the target feature data generated subsequently based on the feature data in the target data table. Simultaneously, it helps improve the accuracy of subsequent data processing results based on the target feature data.

[0115] Optionally, the target data table matching module 430 includes:

[0116] The field matching unit is used to match different fields to be matched in the requirements data parameters with the matching fields in the corresponding feature field information table of different data tables to be matched.

[0117] The target data table determination unit is used to identify the target data table as the data table to which the successfully matched fields belong.

[0118] Optionally, the feature field information table may also include feature identification information;

[0119] Correspondingly, the target feature data generation module 440 includes:

[0120] The target feature data selection unit is used to select target feature data from the target data table based on feature identification information.

[0121] Optional, the target feature data selection unit is specifically used for:

[0122] Identify the table structure of the target data table to obtain different candidate feature fields;

[0123] Based on the feature identification information, the target feature field is selected from each candidate feature field;

[0124] Retrieve target feature data under the target feature field.

[0125] Optionally, the fields to be matched include primary key fields and foreign key fields;

[0126] Correspondingly, the target data table matching module 430 includes:

[0127] The first target data table matching unit is used to match at least one target data table from different data tables to be matched based on the primary key field; and,

[0128] The second target data table matching unit is used to match at least one target data table from different data tables to be matched based on the foreign key field.

[0129] Optionally, the target feature data generation module 440 includes:

[0130] The feature evaluation data acquisition unit is used to acquire feature evaluation data of the feature fields to which different feature data belong in the target data table.

[0131] The target feature data determination unit is used to combine feature data under at least one feature field in the target data table based on feature evaluation data to obtain target feature data.

[0132] Optional, the target feature data determination unit is specifically used for:

[0133] Based on the data to be evaluated, filter the feature data under at least one feature field in the target data table;

[0134] The selected feature data are combined to obtain the target feature data.

[0135] Optional, the target feature data determination unit is specifically used for:

[0136] Based on the data to be evaluated, sort at least one feature field in the target feature table;

[0137] Based on the sorting results, the feature data under at least one feature field in the target feature table are combined to obtain the target feature data.

[0138] Optionally, the feature evaluation data may be determined using the following apparatus:

[0139] The participation frequency acquisition unit is used to obtain the participation frequency of the feature fields to which different feature data belong in the target data table during model training;

[0140] The first feature evaluation data determination unit is used to determine the feature evaluation data of the corresponding feature fields based on the participation frequency.

[0141] Optionally, the feature evaluation data may be determined using the following apparatus:

[0142] The contribution level acquisition unit is used to obtain the contribution level of the feature fields to which different feature data belong in the target data table during model training.

[0143] The second feature evaluation data determination unit is used to determine the feature evaluation data of the corresponding feature fields based on the degree of contribution.

[0144] Optionally, if at least two models participate in training with the same feature field, the second feature evaluation data determination unit is specifically used for:

[0145] Based on the historical usage frequency of different models, determine the contribution weight of the same feature field during the training of different models;

[0146] Based on the contribution weight, the degree of contribution under the same feature field is weighted to obtain the feature evaluation data under the corresponding feature field.

[0147] Optionally, the demand information acquisition module 410 includes:

[0148] The model description data acquisition unit is used to acquire the model description data required by the participants.

[0149] The requirement information determination unit is used to extract model information from preset feature fields of the model description data to obtain model usage requirement information.

[0150] The feature data generation apparatus provided in this application embodiment can execute the feature data generation method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing each feature data generation method.

[0151] The feature data generation apparatus provided in this application embodiment can execute the feature data generation method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing each feature data generation method.

[0152] Figure 5 This is a schematic diagram of the structure of an electronic device that implements a feature data generation method according to an embodiment of this application. Figure 5 The electronic device 512 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0153] like Figure 5 As shown, electronic device 512 is represented in the form of a general-purpose computing device. The components of electronic device 512 may include, but are not limited to: one or more processors or processing units 516, system memory 528, and bus 518 connecting different system components (including system memory 528 and processing unit 516).

[0154] Bus 518 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0155] Electronic device 512 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 512, including volatile and non-volatile media, removable and non-removable media.

[0156] System memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. Electronic device 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 534 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 518 via one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0157] A program / utility 540 having a set (at least one) of program modules 542 may be stored, for example, in memory 528. Such program modules 542 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 542 typically perform the functions and / or methods described in the embodiments of this application.

[0158] Electronic device 512 can also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), and with one or more devices that enable a user to interact with electronic device 512, and / or with any device that enables electronic device 512 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 522. Furthermore, electronic device 512 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 520. As shown, network adapter 520 communicates with other modules of electronic device 512 via bus 518. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0159] The processing unit 516 executes various functional applications and data processing by running at least one of the multiple programs stored in the system memory 528, such as implementing the feature data generation method provided in the embodiments of this application.

[0160] This application also provides a computer-readable storage medium storing a computer program (or computer-executable instructions) thereon, which, when executed by a processor, is used to perform the feature data generation method provided in this application.

[0161] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0162] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0163] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0164] Computer program code for performing the operations of the embodiments of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0165] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the feature data generation method provided in any embodiment of this application.

[0166] In the implementation of the computer program product, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0167] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A feature data generation method characterized by comprising: include: Obtain information on the model usage requirements of the participants; Based on the required information from the model, determine the required data parameters; Based on the different fields to be matched in the required data parameters, at least one target data table is matched from the data tables to be matched in different dimensions. Generate target feature data based on the feature data in the target data table; The step of matching at least one target data table from different dimensions of the target data tables based on different fields to be matched in the required data parameters includes: The different fields to be matched in the required data parameters are matched with the matching fields in the corresponding feature field information tables of different required data tables; wherein, the required data parameters include data granularity, granularity name, participant category, application scenario category, physical layer table identifier, physical layer table primary key field name, and physical layer table foreign key field name; the feature field information table includes feature, feature name, physical layer table identifier, feature field type, physical layer table data date field name, and physical layer table primary key field name; The table containing the matched fields is used as the target table. The step of using the matching data table to which the successfully matched field belongs as the target data table includes: if the similarity of the matching result is greater than a set threshold, then the matching is determined to be successful; otherwise, the matching fails. The feature field information table also includes feature identification information; The step of generating target feature data based on the feature data in the target data table includes: Identify the table structure of the target data table to obtain different candidate feature fields; Based on the feature identification information, the target feature field is selected from each candidate feature field; Obtain the target feature data under the target feature field; The step of generating target feature data based on the feature data in the target data table includes: Obtain feature evaluation data of the feature fields to which different feature data belong in the target data table; wherein, the feature evaluation data is data used to quantify the importance of the feature fields; Based on the feature evaluation data, feature data under at least one feature field in the target data table are combined to obtain the target feature data; The feature evaluation data is determined in the following way: Obtain the contribution of the feature fields to which different feature data belong in the target data table during model training; Based on the degree of contribution, the feature evaluation data of the corresponding feature fields are determined; If at least two models participate in training with the same feature field, then determining the feature evaluation data for the corresponding feature field based on the degree of contribution includes: Based on the historical usage frequency of different models, determine the contribution weight of the same feature field during the training of different models; Based on the contribution weights, the contribution levels under the same feature field are weighted to obtain the feature evaluation data under the corresponding feature field.

2. The method according to claim 1, characterized in that, The fields to be matched include primary key fields and foreign key fields; Accordingly, the step of matching at least one target data table from different target data tables based on different fields to be matched in the demand data parameters includes: Based on the primary key field, match at least one target data table from different tables to be matched; and, Based on the foreign key field, match at least one target data table from different data tables to be matched.

3. The method according to claim 1, characterized in that, The step of combining feature data from at least one feature field in the target data table to obtain the target feature data based on the feature evaluation data includes: Based on the feature evaluation data, feature data under at least one feature field in the target data table is filtered; The selected feature data are combined to obtain the target feature data.

4. The method according to claim 1, characterized in that, The step of combining feature data from at least one feature field in the target data table to obtain the target feature data based on the feature evaluation data includes: Based on the feature evaluation data, sort at least one feature field in the target data table; Based on the sorting results, the feature data under at least one feature field in the target data table are combined to obtain the target feature data.

5. The method according to claim 1, characterized in that, The feature evaluation data is determined in the following way: Obtain the feature fields to which different feature data belong in the target data table, and their participation frequency during model training; Based on the frequency of participation, the feature evaluation data for the corresponding feature fields are determined.

6. The method according to any one of claims 1-2, characterized in that, The process of obtaining the model usage requirements information of the participants includes: Obtain the model description data required by the participants; The model description data is processed by extracting model information from preset feature fields to obtain the model usage requirements information.

7. A feature data generation device, characterized in that, include: The requirement information acquisition module is used to acquire the model usage requirements of the participants; The demand data parameter determination module is used to determine the demand data parameters based on the demand information used in the model. The target data table matching module is used to match at least one target data table from different dimensions of the data tables to be matched based on different fields to be matched in the required data parameters. The target feature data generation module is used to generate target feature data based on the feature data in the target data table; The target data table matching module includes: The field matching unit is used to match different fields to be matched in the requirement data parameters with the matching fields in the corresponding feature field information tables of different data tables; wherein, the requirement data parameters include data granularity, granularity name, participant category, application scenario category, physical layer table identifier, physical layer table primary key field name, and physical layer table foreign key field name; the feature field information table includes feature, feature name, physical layer table identifier, feature field type, physical layer table data date field name, and physical layer table primary key field name; The target data table determination unit is used to determine the target data table as the data table to which the successfully matched fields belong. The target data table determination unit is specifically used to determine that if the similarity of the matching result is greater than a set threshold, the matching is successful; otherwise, the matching fails. The feature field information table also includes feature identification information; The target feature data generation module includes: The target feature data selection unit is used to identify the table structure of the target data table and obtain different candidate feature fields; Based on the feature identification information, the target feature field is selected from each candidate feature field; Obtain the target feature data under the target feature field; The target feature data generation module includes: The feature evaluation data acquisition unit is used to acquire feature evaluation data of feature fields to which different feature data belong in the target data table; wherein, the feature evaluation data is data used to quantify the importance of feature fields; The target feature data determination unit is used to combine feature data under at least one feature field in the target data table according to the feature evaluation data to obtain the target feature data. The feature evaluation data is determined using the following device: The contribution degree acquisition unit is used to acquire the contribution degree of the feature fields to which different feature data belong in the target data table during model training. The second feature evaluation data determination unit is used to determine the feature evaluation data of the corresponding feature field according to the degree of contribution. If at least two models participate in training with the same feature field, then the second feature evaluation data determination unit is specifically used for: Based on the historical usage frequency of different models, determine the contribution weight of the same feature field during the training of different models; Based on the contribution weight, the degree of contribution under the same feature field is weighted to obtain the feature evaluation data under the corresponding feature field.

8. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the feature data generation method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the feature data generation method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the feature data generation method as described in any one of claims 1-6.

Images