Model construction system, method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a model construction system, method, electronic device and storage medium. A model grouping unit is configured to, in response to a model selection instruction for a model database, obtain a main model and a plurality of candidate models corresponding to the main model, combine the main model with each candidate model to obtain at least two model groups, a feature grouping unit is configured to obtain a first model attribute and a first model feature of the main model in each model group, a second model attribute and a second model feature of the candidate model, and group the first model feature and the second model feature according to the first model attribute and the second model attribute to obtain a corresponding feature set, the feature set including a first feature set and a second feature set, and a model construction unit is configured to fuse the first feature set and the second feature set corresponding to each model group to obtain a fused model corresponding to each model group.

Description

Technical Field

[0001] This invention relates to the field of model data processing technology, and in particular to a model building system, a model building method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In existing technologies, developers often update a single model to a new model based on actual needs, or assemble the original model with other models to build a new model. In the specific construction process, it mainly relies on developers to write SQL (Structured Query Language) statements in advance to query the field information of each model, and then use a text editor to write SQL statements to generate the new model. Since the model information results tab and the field information tab are on different interfaces, when extracting field information of the new model from the original model, developers not only need to switch between the model information results tab and the field information tab repeatedly, which makes the model construction time long and the work efficiency low, but also the manual extraction of fields is prone to the risk of field extraction errors, resulting in inaccurate model construction. Summary of the Invention

[0003] The present invention provides a model building system, method, electronic device, and computer-readable storage medium to solve or partially solve the problems of existing model building processes that rely on developers to manually extract and filter model data, which is prone to errors in field extraction, low accuracy of the built model, and long time for manual model building, resulting in low work efficiency for developers.

[0004] This invention discloses a model building system, which includes at least a model database, a model grouping unit communicatively connected to the model database, a feature grouping unit communicatively connected to the model grouping unit, and a model building unit. The model database includes several different models.

[0005] The model grouping unit is configured to respond to a model selection instruction for the model database, obtain a main model corresponding to the model selection instruction and several candidate models corresponding to the main model, and combine the main model with each of the candidate models to obtain at least two groups of model groups.

[0006] The feature grouping unit is used to obtain the first model attribute and first model feature of the main model in each model group, and the second model attribute and second model feature of the candidate model, and to group the first model feature and the second model feature according to the first model attribute and the second model attribute to obtain a corresponding feature set, wherein the feature set includes a first feature set and a second feature set.

[0007] The model building unit is used to perform model fusion based on the first feature set and the second feature set corresponding to each model group to obtain a fused model corresponding to each model group.

[0008] Optionally, the first model attribute includes a first field type, and the second model attribute includes a second field type;

[0009] The feature grouping unit is used to obtain the first field type and first model feature of the main model in each model group, as well as the second field type and second model feature of the candidate model;

[0010] Based on the first field type and the second field type, the first model features and the second model features are grouped to obtain the corresponding feature sets.

[0011] Optionally, the first model feature includes main model field information, the second model feature includes candidate field information, and the first feature set includes a set of identical non-numeric features and a set of dissimilar non-numeric features.

[0012] The feature grouping unit is used to group the first model features and the second model features if the first field type and the second field type are non-numeric, and the main model field information is equal to the candidate model field information, to obtain a corresponding set of identical non-numeric features or a set of dissimilar non-numeric features.

[0013] And / or, if the first field type and the second field type are non-numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0014] Optionally, the main model field information includes the main model field key, and the alternative field information includes the alternative model field key;

[0015] The feature grouping unit is used to calculate the first hash value corresponding to the first model feature and the second hash value corresponding to the second model feature in the model group respectively using a hash algorithm if the first field type and the second field type are non-numeric and the main model field key is equal to the candidate model field key.

[0016] The first correlation coefficient between the first hash value and the second hash value is calculated using the Jaccard coefficient formula, and the first correlation coefficient is weighted and averaged with the corresponding preset correlation weight to obtain the first similarity score.

[0017] Group the first model features and the second model features whose first similarity scores are greater than the first preset threshold into the same set of non-numeric features, or group the first model features and the second model features whose first similarity scores are less than or equal to the first preset threshold into the different set of non-numeric features;

[0018] The formula for the Jaccard coefficient is as follows:

[0019]

[0020] In the Jaccard coefficient formula, X is the first hash value and Y is the second hash value;

[0021] The first model feature includes the main model name, the main model dimension, and the main model field annotation, while the second model feature includes the alternative model name, the alternative model dimension, and the alternative model field annotation.

[0022] Optionally, the feature grouping unit is used to calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model group respectively if the first field type and the second field type are non-numeric and the main model field key is not equal to the candidate model field key;

[0023] The second correlation coefficient between the first word vector and the second word vector is calculated using the cosine correlation formula, and the second correlation coefficient is weighted and averaged with the corresponding preset correlation weight to obtain the second similarity score.

[0024] Group the first model features and the second model features whose second similarity scores are greater than the second preset threshold into the same set of non-numeric features, or group the first model features and the second model features whose second similarity scores are less than or equal to the second preset threshold into the different set of non-numeric features;

[0025] The cosine correlation formula is as follows:

[0026]

[0027] In the cosine correlation formula, A represents the first word vector, and B represents the second word vector.

[0028] Optionally, the first model feature includes main model field information, the second model feature includes candidate field information, and the second feature set includes a set of identical numeric features and a set of dissimilar numeric features;

[0029] The feature grouping unit is used to group the first model features and the second model features if the first field type and the second field type are numeric, and the main model field information is equal to the candidate model field information, so as to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features.

[0030] And / or, if the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features.

[0031] Optionally, the main model field information includes the main model field key, and the alternative field information includes the alternative model field key;

[0032] The feature grouping unit is used to calculate the first matching coefficient between the first model feature and the second model feature in the model grouping if the first field type and the second field type are numeric and the main model field key is equal to the candidate model field key;

[0033] The first matching coefficient is calculated by weighting each of the first matching coefficients with the corresponding preset matching weights to obtain the first matching score.

[0034] Group the first model features and the second model features whose first matching score is greater than the first preset threshold into the same set of digital features, or group the first model features and the second model features whose first matching score is less than or equal to the first preset threshold into the different set of digital features;

[0035] The first model features include the main model name, main model indicators, and main model calculation methods, while the second model features include the alternative model name, alternative model indicators, and alternative model calculation methods.

[0036] Optionally, the feature grouping unit is configured to use the Fuzzy PartialRatio algorithm to calculate a second matching coefficient between the first model feature and the second model feature in the model group if the first field type and the second field type are numeric and the main model field information is not equal to the candidate model field information;

[0037] The second matching score is obtained by weighting each of the second matching coefficients with the corresponding preset matching weights and calculating the weighted average.

[0038] The first model feature and the second model feature whose second matching score is greater than the first preset threshold are grouped into the same set of digital features, or the first model feature and the second model feature whose second matching score is less than or equal to the first preset threshold are grouped into the different set of digital features.

[0039] Optionally, the model building unit is used to visualize the first feature set and the second feature set corresponding to each of the model groups, and in response to feature modification instructions and / or feature confirmation instructions for the first feature set and the second feature set, generate a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set;

[0040] The first target feature set and the second target feature set are used to perform model fusion to obtain a fusion model corresponding to each model group.

[0041] This invention also discloses a model building method, comprising:

[0042] In response to a model selection instruction for a model database, a main model corresponding to the model selection instruction and several candidate models corresponding to the main model are obtained, and the main model is combined with each of the candidate models to obtain at least two groups of models.

[0043] Obtain the first model attribute and first model feature of the main model in each model group, and the second model attribute and second model feature of the candidate model. Then, group the first model feature and the second model feature according to the first model attribute and the second model attribute to obtain the corresponding feature set, which includes a first feature set and a second feature set.

[0044] Model fusion is performed based on the first feature set and the second feature set corresponding to each model group to obtain a fused model corresponding to each model group.

[0045] Optionally, the first model attribute includes a first field type, and the second model attribute includes a second field type. The step of obtaining the first model attribute and first model feature of the main model in each group of models, and the second model attribute and second model feature of the candidate models, and grouping the first model feature and second model feature according to the first model attribute and the second model attribute to obtain a corresponding feature set, includes:

[0046] Obtain the first field type and first model feature of the main model in each model group, as well as the second field type and second model feature of the candidate models;

[0047] Based on the first field type and the second field type, the first model features and the second model features are grouped to obtain the corresponding feature sets.

[0048] Optionally, the first model feature includes main model field information, the second model feature includes candidate field information, the first feature set includes a set of identical non-numeric features and a set of dissimilar non-numeric features, and the step of grouping the first model feature and the second model feature according to the first field type and the second field type to obtain the corresponding feature set includes:

[0049] If the first field type and the second field type are non-numeric, and the main model field information is equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0050] And / or, if the first field type and the second field type are non-numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0051] Optionally, the main model field information includes a main model field key, the candidate field information includes a candidate model field key, and if the first field type and the second field type are non-numeric, and the main model field information is equal to the candidate model field information, then the first model features and the second model features are grouped to obtain corresponding sets of identical or dissimilar non-numeric features, including:

[0052] If the first field type and the second field type are non-numeric, and the main model field key is equal to the candidate model field key, then a hash algorithm is used to calculate the first hash value corresponding to the first model feature in the model group and the second hash value corresponding to the second model feature.

[0053] The first correlation coefficient between the first hash value and the second hash value is calculated using the Jaccard coefficient formula, and the first correlation coefficient is weighted and averaged with the corresponding preset correlation weight to obtain the first similarity score.

[0054] Group the first model features and the second model features whose first similarity scores are greater than the first preset threshold into the same set of non-numeric features, and / or group the first model features and the second model features whose first similarity scores are less than or equal to the first preset threshold into the different set of non-numeric features;

[0055] The formula for the Jaccard coefficient is as follows:

[0056]

[0057] In the Jaccard coefficient formula, X is the first hash value and Y is the second hash value;

[0058] The first model feature includes the main model name, the main model dimension, and the main model field annotation, while the second model feature includes the alternative model name, the alternative model dimension, and the alternative model field annotation.

[0059] Optionally, if the first field type and the second field type are both non-numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain corresponding sets of identical or dissimilar non-numeric features, including:

[0060] If the first field type and the second field type are non-numeric, and the main model field key is not equal to the candidate model field key, then the word2vec algorithm is used to calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model group respectively.

[0061] The second correlation coefficient between the first word vector and the second word vector is calculated using the cosine correlation formula, and each second correlation coefficient is weighted and averaged with its corresponding preset correlation weight to obtain a second similarity score. First model features and second model features with second similarity scores greater than a second preset threshold are grouped into the same set of non-numeric features, and / or, first model features and second model features with second similarity scores less than or equal to the second preset threshold are grouped into the different set of non-numeric features. The cosine correlation formula is as follows: In the cosine correlation formula, A represents the first word vector, and B represents the second word vector. Optionally, the first model feature includes main model field information, the second model feature includes candidate field information, and the second feature set includes a set of identical numeric features and a set of dissimilar numeric features. The step of grouping the first model feature and the second model feature according to the first field type and the second field type to obtain the corresponding feature set includes: if the first field type and the second field type are numeric, and the main model field information is equal to the candidate model field information, then grouping the first model feature and the second model feature to obtain the corresponding set of identical numeric features or the set of dissimilar numeric features; and / or, if the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, then grouping the first model feature and the second model feature to obtain the corresponding set of identical numeric features or the set of dissimilar numeric features. Optionally, the main model field information includes a main model field key, and the alternative field information includes an alternative model field key. If the first field type and the second field type are numeric, and the main model field information is equal to the alternative model field information, then the first model feature and the second model feature are grouped to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features. This includes: if the first field type and the second field type are numeric, and the main model field key is equal to the alternative model field key, then the FuzzyRatio algorithm is used to calculate a first matching coefficient between the first model feature and the second model feature in the model group; and a weighted average is calculated using each of the first matching coefficients and the corresponding preset matching weights to obtain a first matching score.

[0062] Group the first model features and the second model features whose first matching score is greater than the first preset threshold into the same set of digital features, and / or group the first model features and the second model features whose first matching score is less than or equal to the first preset threshold into the different set of digital features;

[0063] The first model features include the main model name, main model indicators, and main model calculation methods, while the second model features include the alternative model name, alternative model indicators, and alternative model calculation methods.

[0064] Optionally, if the first field type and the second field type are both numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features, including:

[0065] If the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, then the Fuzzy Partial Ratio algorithm is used to calculate the second matching coefficient between the first model feature and the second model feature in the model group;

[0066] The second matching score is obtained by weighting each of the second matching coefficients with the corresponding preset matching weights and calculating the weighted average.

[0067] Group the first model features and the second model features whose second matching score is greater than the first preset threshold into the same set of digital features, and / or group the first model features and the second model features whose second matching score is less than or equal to the first preset threshold into the different set of digital features.

[0068] Optionally, the step of performing model fusion based on the first feature set and the second feature set corresponding to each of the model groups to obtain a fused model corresponding to each model group includes:

[0069] The first feature set and the second feature set corresponding to each model group are visualized. In response to feature modification instructions and / or feature confirmation instructions for the first feature set and the second feature set, a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set are generated.

[0070] The first target feature set and the second target feature set are used to perform model fusion to obtain a fusion model corresponding to each model group.

[0071] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0072] The memory is used to store computer programs;

[0073] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0074] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0075] The embodiments of the present invention have the following advantages:

[0076] In this embodiment of the invention, the model building system includes at least a model database, a model grouping unit communicatively connected to the model database, a feature grouping unit communicatively connected to the model grouping unit, and a model building unit. The model database includes several different models. The model grouping unit, in response to a model selection instruction for the model database, acquires a main model corresponding to the model selection instruction and several candidate models corresponding to the main model. It then combines the main model with each candidate model to obtain at least two sets of model groups. The feature grouping unit acquires a first model attribute and a first model feature of the main model in each model group, as well as a second model attribute and a second model feature of the candidate models. It then performs feature grouping on the first and second model features based on the first and second model attributes to obtain a corresponding feature set. The feature set includes... The first and second feature sets, the model building unit, are used to fuse models according to the first and second feature sets corresponding to each model group, to obtain the fused model corresponding to each model group. On the one hand, it automatically determines the main model and each candidate model by responding to the model selection instruction, and randomly groups the main model and each candidate model to obtain the model combination, so as to simplify the model building process. On the other hand, it groups the model features of the main model and the candidate models according to the first model attribute and the second model attribute, and adopts a more suitable feature grouping method for different model attributes. Furthermore, it divides different feature sets according to the feature relationship between the main model and the candidate models, thereby realizing the automatic extraction and classification of a large number of model features, which greatly reduces the workload of developers and thus significantly improves the efficiency and accuracy of model building. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of a model building system provided in an embodiment of the present invention;

[0078] Figure 2 This is a flowchart of the automated pre-modeling steps provided in an embodiment of the present invention;

[0079] Figure 3This is a flowchart of the steps for automated data warehouse modeling based on visualization technology provided in this embodiment of the invention;

[0080] Figure 4 This is a flowchart of the steps of a model construction method provided in an embodiment of the present invention;

[0081] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0082] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] Data warehouse modeling technology has emerged and developed rapidly in recent years. A data warehouse (DW or DWH) is a strategic collection that provides all types of data support for decision-making processes at all levels of an enterprise. It is essentially created for analytical reporting and decision support purposes, serving as a structured data environment for decision support systems and online analytical applications. Through a data warehouse, various multidimensional models can be built according to different data needs, forming data marts available to different user groups. Therefore, modeling a data warehouse can provide effective decision-making references for enterprise decision-makers. Data modeling is the most critical technical step in the data warehouse creation process. However, existing data warehouse modeling techniques often rely on manual data extraction, analysis, and model building. This leads to low developer efficiency and the risk of errors in field extraction during manual data extraction, resulting in inaccurate model construction.

[0084] One of the core inventive points of this invention is the application of a model building system, which includes at least a model database, a model grouping unit communicatively connected to the model database, a feature grouping unit communicatively connected to the model grouping unit, and a model building unit. The model database includes several different models. The model grouping unit, in response to a model selection instruction for the model database, acquires a main model corresponding to the model selection instruction and several candidate models corresponding to the main model. It then combines the main model with each candidate model to obtain at least two sets of model groups. The feature grouping unit acquires the first model attribute and first model feature of the main model in each set of model groups, as well as the second model attribute and second model feature of the candidate models. It then performs feature grouping on the first model feature and second model feature according to the first model attribute and second model attribute to obtain corresponding features. The feature set, including a first feature set and a second feature set, is a model building unit used to perform model fusion based on the first and second feature sets corresponding to each model group, obtaining a fused model corresponding to each model group. On the one hand, it automatically determines the main model and each candidate model by responding to model selection instructions, and randomly groups the main model with each candidate model to obtain model combinations, thereby simplifying the model building process. On the other hand, it groups the model features of the main model and candidate models according to the first and second model attributes, and adopts a more suitable feature grouping method for different model attributes. Furthermore, it divides different feature sets according to the feature relationship between the main model and candidate models, realizing the automatic extraction and classification of a large number of model features, greatly reducing the workload of developers, and thus significantly improving the efficiency and accuracy of model building.

[0085] Reference Figure 1 The diagram illustrates a model building system provided in an embodiment of the present invention. The model building system includes at least a model database, a model grouping unit communicatively connected to the model database, a feature grouping unit communicatively connected to the model grouping unit, and a model building unit. The model database includes several different models, and may specifically include the following steps:

[0086] The model grouping unit is used to respond to the model selection command for the model database, obtain the main model corresponding to the model selection command and several candidate models corresponding to the main model, and combine the main model with each candidate model to obtain at least two groups of model groups.

[0087] The feature grouping unit is used to obtain the first model attribute and first model feature of the main model in each model group, as well as the second model attribute and second model feature of the candidate model, and to group the first model feature and second model feature according to the first model attribute and second model attribute to obtain the corresponding feature set, which includes the first feature set and the second feature set.

[0088] The model building unit is used to fuse models based on the first feature set and the second feature set corresponding to each model group to obtain a fused model corresponding to each model group.

[0089] In this embodiment of the invention, the primary model and the alternative models are data models selected from the model list. The criteria for selecting the primary model and the alternative models are: the model that responds to the model selection command in the first instance is designated as the primary model, and the remaining models are designated as alternative models. For example, dragging an existing model from the model list in the model database to the workspace, the first dragged model is marked as the primary model T0, and the remaining models are marked as alternative models T1, T2, ..., TN, respectively. A data model is a reflection and abstraction of real-world phenomena; it is a mapping of data relationships, representing the relationships between business operations and depicting them graphically, rather than a vague relationship in the mind. The model database is a database that stores multiple data models. The primary model and the alternative models can be selected from the model database. The model grouping unit, which communicates with the model database, is a unit used to group multiple data models.

[0090] For example, if three models are selected from the model database as the main model T0, the alternative model T1, and the alternative model T2, then the main model T0 is combined with the alternative models T1 and T2 respectively to obtain two model groups: model group S1 {main model T0, alternative model T1} and model group S2 {main model T0, alternative model T2}. The feature grouping unit, which communicates with the model grouping unit, is a unit that classifies the first model features of the main model and the second model features of each alternative model using the first model attribute and the second model attribute, and generates the corresponding feature set. The model building unit is a unit that fuses the main model and the alternative models based on the feature set to build a fused model corresponding to the model group. The fused model is a new model generated based on the main model and the alternative models. For example, if the main model is an order table and the alternative models are user tables, then the fused model based on the order table and the user table can be a user order table.

[0091] Optionally, model attributes can be field types of the data model. For example, field types can be numeric fields, such as int (32-bit integer), bigint (64-bit integer), float (32-bit single-precision floating-point number), double (64-bit single-precision floating-point number), etc., as well as non-numeric fields. These can include a first model attribute (first field type) corresponding to the main model and a second model attribute (second field type) corresponding to the candidate models. Model features can be model information, field information, and corresponding indicator / dimensional information of the data model. Model features can be extracted when the main model and candidate models are selected in the model database. They can include a first model feature corresponding to the main model and a second model feature corresponding to the candidate models. The feature set can be the result set generated after grouping the first and second model features. It can include the first feature set and the second feature set between the main model and the candidate models.

[0092] Specifically, the first model attribute includes the first field type, the second model attribute includes the second field type, and the feature grouping unit can be used to obtain the first field type and first model feature of the main model in each model group, as well as the second field type and second model feature of the candidate model. Based on the first field type and the second field type, the first model feature and the second model feature are grouped to obtain the corresponding feature set.

[0093] In a specific implementation, the first model feature includes main model field information, the second model feature includes candidate field information, and the first feature set includes a set of identical non-numeric features and a set of dissimilar non-numeric features. The feature grouping unit can be used to group the first model feature and the second model feature if the first field type and the second field type are non-numeric, and the main model field information is equal to the candidate model field information, to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features, and / or, if the first field type and the second field type are non-numeric, and the main model field information is not equal to the candidate model field information, to group the first model feature and the second model feature to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0094] Optionally, the set of identical non-numeric features can be a result set used to record all equal first model features of the main model and second model features of the candidate models when the first and second model attributes are non-numeric. The set of dissimilar non-numeric features can be a result set used to record all unequal first model features of the main model and second model features of the candidate models when the first and second model attributes are non-numeric. For example, the set of identical non-numeric features S1 is: {field1: main model.field1 = modelN.field, field2: main model.field2 = modelN.field, ...}, and the set of dissimilar non-numeric features S2 is: {main model.fieldx, main model.fieldy, ..., modelN.fieldz}.

[0095] As an example, the main model field information includes the main model field key, and the alternative field information includes the alternative model field key. The feature grouping unit can be used to calculate the first hash value corresponding to the first model feature and the second hash value corresponding to the second model feature in the model grouping if the first field type and the second field type are non-numeric, and the main model field key and the alternative model field key are equal. Then, a hash algorithm is used to calculate the first hash value corresponding to the first model feature and the second hash value corresponding to the second model feature in the model grouping. The Jaccard coefficient formula is used to calculate the first correlation coefficient between the first hash value and the second hash value. Each first correlation coefficient is then weighted and averaged with its corresponding preset correlation weight to obtain a first similarity score. The first model features and second model features with a first similarity score greater than a first preset threshold are grouped into the same set of non-numeric features, or the first model features and second model features with a first similarity score less than or equal to the first preset threshold are grouped into different sets of non-numeric features. The Jaccard coefficient formula is:

[0096]

[0097] In the Jaccard coefficient formula, X is the first hash value and Y is the second hash value;

[0098] The first model feature includes the main model name, main model dimensions, and main model field annotations, while the second model feature includes the alternative model name, alternative model dimensions, and alternative model field annotations.

[0099] Optionally, the correlation coefficient can be a coefficient representing the correlation between the first model feature and the second model feature. It can include a first correlation coefficient corresponding to non-numeric fields and a second correlation coefficient corresponding to numeric fields. The values ​​of the first and second correlation coefficients are both in the range of [0, 1]. The larger the correlation coefficient, the higher the correlation between the first model feature and the second model feature. The main model field information can be the field key of the main model, and the candidate model field information can be the field key of the candidate model. The field key is the primary key field of the model. By comparing the field keys, it is determined whether the field information between the main model and the candidate model is the same. The main model name can be the model name in Chinese and English, and the candidate model name can be the model name in Chinese and English. The main model dimension and the candidate model dimension can be the content representing a certain feature of a thing or phenomenon, such as user age, user gender, user height, user code, order code, etc. The dimension content changes for different fields. The field annotation can be the content annotated in the field by technical personnel according to actual needs. The preset relevance weights can be weight coefficients set by relevant technical personnel for non-numerical model features based on actual needs or the importance of model features. The larger the preset relevance weight, the higher the importance of the model feature. The similarity score is the coefficient value obtained by weighting the various relevance coefficients and the preset relevance weights.

[0100] In one example, the pre-set relevance weights for the dimensions corresponding to the main model field and the candidate model field are 0.6; the pre-set relevance weights for the model's English name + field key corresponding to the main model and the candidate model are 0.2; and the pre-set relevance weights for the field comments corresponding to the main model field and the candidate model field are 0.2. The first preset threshold is a coefficient threshold set under the premise that the first model attribute and the second model attribute are equal to the main model field key and the candidate model field key. For example, if the first preset threshold is 0.5, the first model features and second model features with a first similarity score greater than 0.5 are classified into the same set of non-numeric features, and the first model features and second model features with a first similarity score less than or equal to 0.5 are classified into the different set of non-numeric features.

[0101] Hash algorithms (also known as hashing algorithms or message digest algorithms) are a fundamental technology used for information storage and retrieval. They are file construction methods based on hash functions, enabling fast random access to records. Specifically, they map any given key of arbitrary length to a hash value of fixed length. Hash values ​​are generally used for authentication, encryption, and indexing. In this invention, hash algorithms can be used to calculate the first hash value corresponding to the first model feature and the second hash value corresponding to the second model feature in the model group. For example, hash algorithms can be used to calculate the first hash value corresponding to the main model name plus the field key and the second hash value corresponding to the candidate model name plus the field key. These first and second hash values ​​are then used as inputs to the Jaccard coefficient formula to obtain the first correlation coefficient. Similarly, hash algorithms can be used to calculate the first hash value corresponding to the main model dimension, the second hash value corresponding to the candidate model dimension, and the first hash value corresponding to the main model field annotation and the second hash value corresponding to the candidate model field annotation. The first and second hash values ​​under the same model feature are then input into the Jaccard coefficient formula to obtain the first correlation coefficient.

[0102] The Jaccard Index, also known as the Jaccard similarity coefficient, is used to compare the differences and similarities between two samples. It can calculate the similarity of asymmetric binary attribute objects. The sets are distinct, meaning that any two elements in a set are different objects. By judging whether the value of any element in set A is equal to that in set B, a similarity exists. The higher the Jaccard Index, the higher the similarity between the two samples. After quantizing the first model feature of the main model and the second model feature of the candidate model into fixed-length hash values, the correlation between the first model feature and the second model feature can be obtained using the Jaccard Index formula.

[0103] As another example, the feature grouping unit can be used to calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model grouping if the first field type and the second field type are non-numeric, and the main model field key and the candidate model field key are not equal. Then, the word2vec algorithm is used to calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model grouping. The cosine correlation formula is used to calculate the second correlation coefficient between the first word vector and the second word vector. Each second correlation coefficient is then weighted and averaged with its corresponding preset correlation weight to obtain a second similarity score. The first model feature and the second model feature with a second similarity score greater than a second preset threshold are grouped into the same set of non-numeric features, or the first model feature and the second model feature with a second similarity score less than or equal to the second preset threshold are grouped into different sets of non-numeric features. The cosine correlation formula is:

[0104]

[0105] In the cosine correlation formula, A is the first word vector and B is the second word vector.

[0106] Optionally, the second preset threshold is a similarity threshold set under the premise that the first model attribute and the second model attribute are not equal to the key of the main model field and the key of the candidate model field. For example, the second preset threshold is 0.8. The first model feature and the second model feature with a second similarity score greater than 0.5 are classified into the same set of non-numeric features, and the first model feature and the second model feature with a second similarity score less than or equal to 0.8 are classified into the different set of non-numeric features.

[0107] Word2vec is a relevance model used to generate word vectors. These models are shallow, two-layer neural networks trained to reconstruct linguistic word text. The network represents words and needs to guess the input words in adjacent positions. Under the bag-of-words assumption in word2vec, the order of words is unimportant. After training, the word2vec model maps each word to a vector, representing the relationship between words. This vector is the hidden layer of the neural network. Therefore, the word2vec algorithm can calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model group, respectively. The word vectors are used to measure the relevance between the first and second model features. Cosine correlation, also known as cosine similarity, evaluates the similarity between two vectors by calculating the cosine of the angle between them. Cosine similarity plots vectors in a vector space based on their coordinates, such as the most common two-dimensional space, and measures the similarity between two vectors by measuring the cosine of the angle between them.

[0108] For example, from the data warehouse model, two original models are selected: the main model {order table} and the alternative model {user table}. Based on application requirements, a fusion model {user order table} is constructed between the main model and the alternative model. From the perspective of field dimensions, the ID in the user table corresponds to the user_id in the order table (order.user_id = user.id), and the order code corresponds to order.id. Other non-numeric model features are not equal. By adopting a logical processing method for non-numeric features, we can obtain the set of identical non-numeric features S1 = {order.user_id: order.user_id = user.id}, and the set of dissimilar non-numeric features S2 = {order.id, order.order_time, user.name, user.email}. From the perspective of metrics, the actual amount received corresponds to order.amount. By adopting a logical processing method for numeric features, we can obtain the set of identical numeric features S3 = {}, and the set of dissimilar numeric features S4 = {order.amount: sum}.

[0109] ID(varchar) Name(varchar) Email(varchar) U1 Zhang San zhangsan@163.com U2 Li Si lisi@163.com

[0110] User table

[0111]

[0112] Order form

[0113] In the specific implementation, the first model feature includes main model field information, the second model feature includes candidate field information, the second feature set includes a set of identical numeric features and a set of dissimilar numeric features, and the feature grouping unit is used to group the first model feature and the second model feature if the first field type and the second field type are numeric, and the main model field information is equal to the candidate model field information, to obtain the corresponding set of identical numeric features or set of dissimilar numeric features, and / or if the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, to group the first model feature and the second model feature to obtain the corresponding set of identical numeric features or set of dissimilar numeric features.

[0114] As an example, the feature grouping unit is used to calculate the first matching coefficient between the first model feature and the second model feature in the model grouping if the first field type and the second field type are numeric, and the main model field key and the candidate model field key are equal. The first matching coefficient is then weighted and averaged with the corresponding preset matching weight to obtain the first matching score. The first model feature and the second model feature with the first matching score greater than the first preset threshold are grouped into the same set of numeric features, or the first model feature and the second model feature with the first matching score less than or equal to the first preset threshold are grouped into different sets of numeric features. The first model feature also includes the main model name, the main model index, and the main model calculation method, and the second model feature also includes the candidate model name, the candidate model index, and the candidate model calculation method.

[0115] As another example, the feature grouping unit is used to calculate the second matching coefficient between the first model feature and the second model feature in the model grouping if the first field type and the second field type are numeric and the main model field information and the candidate model field information are not equal. The second matching coefficient is then calculated by weighted averaging of each second matching coefficient with the corresponding preset matching weight to obtain the second matching score. The first model feature and the second model feature with the second matching score greater than the first preset threshold are grouped into the same set of numeric features, or the first model feature and the second model feature with the second matching score less than or equal to the first preset threshold are grouped into different sets of numeric features.

[0116] Optionally, the main model indicators can be the digital content of the main model, and the alternative model indicators can be the digital content of the alternative models, such as population, income, number of users, user ID, and actual order amount received. They can also include indicator aggregation methods, such as sum (summation), count (return all rows), avg (average), max (maximum value), and min (minimum value), etc. The calculation scope can be the calculation method and unit of measurement. Both the main model and the alternative models have corresponding calculation scopes.

[0117] The Fuzzy Ratio algorithm (matching algorithm) and the Fuzzy Partial Ratio algorithm (incomplete matching algorithm) can perform matching based on the best substring to calculate the matching degree between the first model feature of the main model and the second model feature of the candidate model. The matching coefficient can be a value representing the matching degree between the first and second model features, and can include both the first and second matching coefficients. The preset matching weight can be a weight coefficient set by relevant technical personnel for numerical model features based on actual needs or the importance of the model features. The larger the preset matching weight, the higher the importance of the model feature. The matching degree score is the coefficient value obtained by weighted averaging of each matching coefficient and the preset matching weight. For example, the preset matching weight for the indicators and aggregation methods corresponding to the main model fields and candidate model fields is 0.6; the preset matching weight for the model name + field key corresponding to the main model and candidate models is 0.1; and the preset matching weight for the calculation caliber corresponding to the main model fields and candidate model fields is 0.3.

[0118] Reference Figure 2 The flowchart illustrates an automated pre-modeling process. First, a set `table_ids` is created using the model IDs of models T0, T1, T2…TN. Then, based on `table_ids`, model information (model name in English and Chinese, model comments, etc.), field information (field name, field key, field type, field comment, field length), and corresponding metrics and dimension information (metric / dimensional name, key, type, comment, calculation method) for {T0, T1, T2…TN} are retrieved from the model database. Next, the selected models are grouped into G = [{T0, T1}, {T0, T2}…{T0, TN}]. The two models in each group of dataset G are traversed, and the model information, field information, and corresponding metrics and dimension information for both models are extracted. Based on whether the field type is numeric, the data is divided into two parts for parallel processing. The parallel processing process specifically includes the following steps:

[0119] (1) When the field type of the main model and the field of model N is a non-numeric field group, iterate through the fields of the main model and compare them with all fields of model N to determine whether the field keys are equal:

[0120] a. If they are equal, use Algorithm 1 to calculate the equality. The specific calculation process is as follows: First, extract the dimensions (weight 0.6), model name + field key (weight 0.2), and comments (weight 0.2) corresponding to the main model field and the model N field. Use the Hash algorithm to calculate the hash values ​​X (main model) and Y (model N) respectively. Then, use the Jaccard coefficient Jaccard(X, Y) = (X∩Y) / (X∪Y) to calculate the difference between the two. First, calculate the intersection of X and Y, then calculate the union of X and Y, and finally calculate the coefficient value θ. After weighted averaging, if it is greater than 0.5, it is considered equal. Then, group the main model field and the model N field into the set of non-numeric features that are the same, otherwise group them into the set of non-numeric features that are different.

[0121] b. If they are not equal, use Algorithm 2 to calculate the relevance. The specific calculation process is as follows: First, extract the dimensions (weight 0.6) corresponding to the main model field and the model N field, the English name of the model + field key (weight 0.2), and the annotation (weight 0.2). Use the word2vec algorithm to calculate the word vectors A and B respectively. Then, use the cosine relevance sim(A, B) = cos(θ) = (A×B) / (|A||B|) to calculate the relevance θ between the two. After weighted averaging, if it is greater than 0.8, it is considered to be closely related. Then, group the main model field and the model N field into the set of non-numeric features that are the same, otherwise group them into the set of non-numeric features that are different.

[0122] c. The final result set of equal fields S1 (set of identical non-numeric features) = {field1: main model.field1 = model N.field, field2: main model.field2 = model N.field, ...} and the set of all unequal fields S2 (set of dissimilar non-numeric features) = {main model.fieldx, main model.fieldy, ..., model N.fieldz};

[0123] (2) When the field type of the main model and the field of model N is a numeric field group, iterate through the fields of the main model and compare them with all fields of model N to determine whether the field keys are equal:

[0124] a. If they are equal, use Algorithm 3. The specific calculation process is as follows: First, extract the indicators and aggregation methods (e.g., sum, count, avg, max, etc.) corresponding to the main model field and the model N field (weight 0.6), the model's English name + field key (weight 0.1), and the scope (weight 0.3). Use the Fuzzy ratio (matching algorithm) to calculate the matching degree θ. If the weighted average is greater than 0.5, they are considered equal. Then, group the main model field and the model N field into the set of identical numerical features. Otherwise, group them into the set of dissimilar numerical features.

[0125] b. If they are not equal, use Algorithm 4: First, extract the indicators and aggregation methods (weight 0.6), model English name + field key (weight 0.1), and caliber (weight 0.3) corresponding to the main model field and the model N field. Calculate the matching degree θ using the fuzzy partial ratio algorithm. If the weighted average is greater than 0.8, they are considered equal. Group the main model field and the model N field into the set of identical numerical features. Otherwise, group them into the set of dissimilar numerical features.

[0126] c. Finally, the result set S3 (set of identical numeric features) is obtained where the fields are equal (taking the main model) = {main model.field1: sum, main model.field2: avg, main model.field3: count...} and the result set S4 (set of dissimilar numeric features) is obtained where the fields are unequal (taking the main model) = {main model.field4: sum, model N.field5: avg, model N.field6: count...}.

[0127] (3) Merge the output result sets (feature sets) S1, S2, S3, S4.

[0128] In one example of the present invention, the model building unit can be used to visualize the first feature set and the second feature set corresponding to each model group, and in response to feature modification instructions and / or feature confirmation instructions for the first feature set and the second feature set, generate a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set, and perform model fusion using the first target feature set and the second target feature set to obtain a fused model corresponding to each model group.

[0129] Reference Figure 3 This diagram illustrates the steps involved in automated data warehouse modeling based on visualization technology, including the following steps:

[0130] (1) Model list: Drag an existing model from the model list to the work area. The first dragged model is marked as the main model T0, and the remaining models are marked as model T1, model T2, ... model TN respectively;

[0131] (2) Automated pre-modeling of the selected model;

[0132] (3) Visualization of the execution statement: The result sets of S1, S2, S3, and S4 are visualized at the bottom of the workspace. Then, the result set of the fields of S1 is displayed in the model connection area of ​​the workspace, and the query statement is generated and displayed in the preview result area at the bottom of the workspace. The query statement is as follows:

[0133] Select fields S1, S2, S3 [aggregation method (main model field)], and S4 [aggregation method (main model + model N field)].

[0134] From mainmodel left join model1 on S1[mainmodel.field1 = model1.field and mainmodel.field2 = model1.field...]

[0135] LEFT JOIN Model2 ON S1[MainModel.Field1 = Model2.Field AND MainModel.Field2 = Model2.Field...]

[0136] LEFT JOIN ModelN ON S1[MainModel.Field1 = ModelN.Field AND MainModel.Field2 = ModelN.Field...]

[0137] Where main model.time field = {date variable} and model N.time field = {date variable}

[0138] Group by S1 field, S2 field

[0139] (4) Manual confirmation: The result sets of S1, S2, S3, and S4 are to be manually confirmed or modified. The confirmed or modified result sets are then written back to S1, S2, S3, and S4.

[0140] (5) Result Preview: Preview the results again to confirm whether the results are correct;

[0141] (6) Model Confirmation: After confirming that there are no errors, a new model is generated. The model statement is as follows:

[0142] Insert overwrite table new model PARTITION(time field = {date variable})

[0143] Select fields S1, S2, S3 [aggregation method (main model field)], and S4 [aggregation method (main model + model N field)].

[0144] From mainmodel left join model1 on S1[mainmodel.field1 = model1.field and mainmodel.field2 = model1.field...]

[0145] LEFT JOIN Model2 ON S1[MainModel.Field1 = Model2.Field AND MainModel.Field2 = Model2.Field...]

[0146] left join modelN on S1[mainmodel.field1 = modelN.field and mainmodel.field2 = modelN.field...]

[0147] Where main model.time field = {date variable} and model N.time field = {date variable}

[0148] Group by S1 field, S2 field

[0149] (7) Write the new model to the data warehouse: Execute the model statement and mount the schedule. The model generation is complete.

[0150] In the process of automated data warehouse modeling based on visualization technology, firstly, all fields of the model can be analyzed through automated pre-construction, directly outputting the model corresponding to the equal fields, saving a significant amount of development time and greatly improving the efficiency of data developers. Secondly, by visualizing the result set and supporting developers to confirm or modify it in the visualization interface, it facilitates human-computer interaction, improves the accuracy of model construction, and reduces the risk of field extraction errors.

[0151] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that, under the guidance of the ideas in the embodiments of the present invention, those skilled in the art can make settings according to actual circumstances, and the present invention does not limit such settings.

[0152] In this embodiment of the invention, the model building system includes at least a model database, a model grouping unit communicatively connected to the model database, a feature grouping unit communicatively connected to the model grouping unit, and a model building unit. The model database includes several different models. The model grouping unit, in response to a model selection instruction for the model database, acquires a main model corresponding to the model selection instruction and several candidate models corresponding to the main model. It then combines the main model with each candidate model to obtain at least two sets of model groups. The feature grouping unit acquires a first model attribute and a first model feature of the main model in each model group, as well as a second model attribute and a second model feature of the candidate models. It then performs feature grouping on the first and second model features based on the first and second model attributes to obtain a corresponding feature set. The feature set includes... The first and second feature sets, the model building unit, are used to fuse models according to the first and second feature sets corresponding to each model group, to obtain the fused model corresponding to each model group. On the one hand, it automatically determines the main model and each candidate model by responding to the model selection instruction, and randomly groups the main model and each candidate model to obtain the model combination, so as to simplify the model building process. On the other hand, it groups the model features of the main model and the candidate models according to the first model attribute and the second model attribute, and adopts a more suitable feature grouping method for different model attributes. Furthermore, it divides different feature sets according to the feature relationship between the main model and the candidate models, thereby realizing the automatic extraction and classification of a large number of model features, which greatly reduces the workload of developers and thus significantly improves the efficiency and accuracy of model building.

[0153] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0154] Reference Figure 4 The diagram illustrates a flowchart of a model construction method provided in an embodiment of the present invention, which may specifically include the following modules:

[0155] Step 401: In response to the model selection instruction for the model database, obtain the main model corresponding to the model selection instruction and several candidate models corresponding to the main model, and combine the main model with each of the candidate models to obtain at least two groups of models.

[0156] Step 402: Obtain the first model attribute and first model feature of the main model in each model group, and the second model attribute and second model feature of the candidate model. Then, group the first model feature and the second model feature according to the first model attribute and the second model attribute to obtain the corresponding feature set. The feature set includes the first feature set and the second feature set.

[0157] Step 403: Perform model fusion based on the first feature set and the second feature set corresponding to each model group to obtain a fused model corresponding to each model group.

[0158] In one optional embodiment, the first model feature includes main model field information, the second model feature includes candidate field information, the first feature set includes a set of identical non-numeric features and a set of dissimilar non-numeric features, and the step of grouping the first model feature and the second model feature according to the first field type and the second field type to obtain the corresponding feature set includes:

[0159] If the first field type and the second field type are non-numeric, and the main model field information is equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0160] And / or, if the first field type and the second field type are non-numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical non-numeric features or the set of dissimilar non-numeric features.

[0161] In one optional embodiment, the main model field information includes a main model field key, the alternative field information includes an alternative model field key, and if the first field type and the second field type are non-numeric, and the main model field information is equal to the alternative model field information, then the first model features and the second model features are grouped to obtain corresponding sets of identical or dissimilar non-numeric features, including:

[0162] If the first field type and the second field type are non-numeric, and the main model field key is equal to the candidate model field key, then a hash algorithm is used to calculate the first hash value corresponding to the first model feature in the model group and the second hash value corresponding to the second model feature.

[0163] The first correlation coefficient between the first hash value and the second hash value is calculated using the Jaccard coefficient formula, and the first correlation coefficient is weighted and averaged with the corresponding preset correlation weight to obtain the first similarity score.

[0164] Group the first model features and the second model features whose first similarity scores are greater than the first preset threshold into the same set of non-numeric features, and / or group the first model features and the second model features whose first similarity scores are less than or equal to the first preset threshold into the different set of non-numeric features;

[0165] The formula for the Jaccard coefficient is as follows:

[0166]

[0167] In the Jaccard coefficient formula, X is the first hash value and Y is the second hash value;

[0168] The first model feature includes the main model name, the main model dimension, and the main model field annotation, while the second model feature includes the alternative model name, the alternative model dimension, and the alternative model field annotation.

[0169] In one optional embodiment, if the first field type and the second field type are both non-numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain corresponding sets of identical or dissimilar non-numeric features, including:

[0170] If the first field type and the second field type are non-numeric, and the main model field key is not equal to the candidate model field key, then the word2vec algorithm is used to calculate the first word vector corresponding to the first model feature and the second word vector corresponding to the second model feature in the model group respectively.

[0171] The second correlation coefficient between the first word vector and the second word vector is calculated using the cosine correlation formula, and the second correlation coefficient is weighted and averaged with the corresponding preset correlation weight to obtain the second similarity score.

[0172] Group the first model features and the second model features whose second similarity scores are greater than the second preset threshold into the same set of non-numeric features, and / or group the first model features and the second model features whose second similarity scores are less than or equal to the second preset threshold into the different set of non-numeric features;

[0173] The cosine correlation formula is as follows:

[0174]

[0175] In the cosine correlation formula, A represents the first word vector, and B represents the second word vector.

[0176] In one optional embodiment, the first model feature includes main model field information, the second model feature includes candidate field information, and the second feature set includes a set of identical numeric features and a set of dissimilar numeric features. The step of grouping the first model feature and the second model feature according to the first field type and the second field type to obtain the corresponding feature set includes:

[0177] If the first field type and the second field type are numeric, and the main model field information is equal to the candidate model field information, then the first model features and the second model features are grouped to obtain the corresponding set of identical numeric features or the set of dissimilar numeric features.

[0178] And / or, if the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, then the first model features and the second model features are grouped to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features.

[0179] In one optional embodiment, the main model field information includes a main model field key, the alternative field information includes an alternative model field key, and if the first field type and the second field type are numeric, and the main model field information is equal to the alternative model field information, then the first model features and the second model features are grouped to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features, including:

[0180] If the first field type and the second field type are numeric, and the main model field key is equal to the candidate model field key, then the Fuzzy Ratio algorithm is used to calculate the first matching coefficient between the first model feature and the second model feature in the model group;

[0181] The first matching coefficient is calculated by weighting each of the first matching coefficients with the corresponding preset matching weights to obtain the first matching score.

[0182] Group the first model features and the second model features whose first matching score is greater than the first preset threshold into the same set of digital features, and / or group the first model features and the second model features whose first matching score is less than or equal to the first preset threshold into the different set of digital features;

[0183] The first model features include the main model name, main model indicators, and main model calculation methods, while the second model features include the alternative model name, alternative model indicators, and alternative model calculation methods.

[0184] In one optional embodiment, if the first field type and the second field type are both numeric, and the main model field information is not equal to the candidate model field information, then feature grouping is performed on the first model features and the second model features to obtain a corresponding set of identical numeric features or a set of dissimilar numeric features, including:

[0185] If the first field type and the second field type are numeric, and the main model field information is not equal to the candidate model field information, then the Fuzzy Partial Ratio algorithm is used to calculate the second matching coefficient between the first model feature and the second model feature in the model group;

[0186] The second matching score is obtained by weighting each of the second matching coefficients with the corresponding preset matching weights and calculating the weighted average.

[0187] Group the first model features and the second model features whose second matching score is greater than the first preset threshold into the same set of digital features, and / or group the first model features and the second model features whose second matching score is less than or equal to the first preset threshold into the different set of digital features.

[0188] In one optional embodiment, the step of performing model fusion based on the first feature set and the second feature set corresponding to each of the model groups to obtain a fused model corresponding to each of the model groups includes:

[0189] The first feature set and the second feature set corresponding to each model group are visualized. In response to feature modification instructions and / or feature confirmation instructions for the first feature set and the second feature set, a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set are generated.

[0190] The first target feature set and the second target feature set are used to perform model fusion to obtain a fusion model corresponding to each model group.

[0191] As the method embodiments are basically similar to the system embodiments, the description is relatively simple, and relevant parts can be found in the description of the method embodiments.

[0192] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described model construction method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0193] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described model construction method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0194] Figure 5 A structural block diagram of an electronic device for implementing various embodiments of the present invention.

[0195] The electronic device 500 includes, but is not limited to, components such as: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will understand that... Figure 5 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. In embodiments of the present invention, the electronic device includes, but is not limited to, mobile phones, tablet computers, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.

[0196] It should be understood that, in this embodiment of the invention, the radio frequency unit 501 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 510; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 501 can also communicate with networks and other devices through a wireless communication system.

[0197] The electronic device provides users with wireless broadband internet access through the network module 502, such as helping users send and receive emails, browse web pages, and access streaming media.

[0198] The audio output unit 503 can convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into audio signals and output them as sound. Furthermore, the audio output unit 503 can also provide audio output related to specific functions performed by the electronic device 500 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, and a receiver, etc.

[0199] Input unit 504 is used to receive audio or video signals. Input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 506. The image frames processed by GPU 5041 can be stored in memory 509 (or other storage medium) or transmitted via radio frequency unit 501 or network module 502. Microphone 5042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 501 in telephone call mode.

[0200] The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 5061 according to the ambient light level, and the proximity sensor can turn off the display panel 5061 and / or backlight when the electronic device 500 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 505 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.

[0201] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0202] User input unit 507 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 507 includes a touch panel 5071 and other input devices 5072. Touch panel 5071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 5071). Touch panel 5071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 510, which receives and executes commands from the processor 510. In addition, touch panel 5071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. Besides touch panel 5071, user input unit 507 may also include other input devices 5072. Specifically, other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0203] Furthermore, the touch panel 5071 can cover the display panel 5061. When the touch panel 5071 detects a touch operation on or near it, it transmits the information to the processor 510 to determine the type of touch event. Subsequently, the processor 510 provides corresponding visual output on the display panel 5061 based on the type of touch event. Although in Figure 5 In this embodiment, the touch panel 5071 and the display panel 5061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.

[0204] Interface unit 508 serves as an interface for connecting external devices to electronic device 500. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 508 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 500, or it can be used to transmit data between electronic device 500 and external devices.

[0205] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0206] The processor 510 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 509, and by calling data stored in the memory 509, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 510.

[0207] The electronic device 500 may also include a power supply 511 (such as a battery) for supplying power to various components. Preferably, the power supply 511 can be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0208] In addition, the electronic device 500 includes some functional modules not shown, which will not be described in detail here.

[0209] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0211] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0212] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0213] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0214] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0217] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A model building system, characterized by, The model construction system comprises at least a model database, a model grouping unit in communication connection with the model database, a feature grouping unit in communication connection with the model grouping unit, and a model construction unit, the model database comprising a plurality of different models; wherein The model grouping unit is configured to, in response to a model selection instruction for the model database, acquire a main model and a plurality of alternative models corresponding to the main model of the model selection instruction, combine the main model with each of the alternative models to obtain at least two groups of model groups; The feature grouping unit is configured to acquire a first model attribute and a first model feature of the main model in each of the model groups, and a second model attribute and a second model feature of the alternative model, and perform feature grouping on the first model feature and the second model feature according to the first model attribute and the second model attribute to obtain a corresponding feature set, the feature set comprising a first feature set and a second feature set; The model construction unit is configured to perform model fusion according to the first feature set and the second feature set corresponding to each of the model groups to obtain a fusion model corresponding to each of the model groups; The first model attribute comprises a first field type, and the second model attribute comprises a second field type; The feature grouping unit is configured to acquire a first field type and a first model feature of the main model in each of the model groups, and a second field type and a second model feature of the alternative model; The feature grouping unit is configured to acquire a first field type and a first model feature of the main model in each of the model groups, and a second field type and a second model feature of the alternative model; The model construction unit is configured to perform visual display on the first feature set and the second feature set corresponding to each of the model groups, and in response to a feature modification instruction and / or a feature confirmation instruction for the first feature set and the second feature set, generate a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set; The model construction unit is configured to perform model fusion using the first target feature set and the second target feature set to obtain a fusion model corresponding to each of the model groups.

2. The system of claim 1, wherein, The first model feature comprises main model field information, the second model feature comprises alternative field information, and the first feature set comprises a non-digital feature same set and a non-digital feature different set; The feature grouping unit is configured to, if the first field type and the second field type are non-digital, and the main model field information is equal to the alternative model field information, perform feature grouping on the first model feature and the second model feature to obtain a corresponding non-digital feature same set or a non-digital feature different set. And / or, if the first field type and the second field type are the non-numeric type, and the main model field information is not equal to the alternative model field information, the first model feature and the second model feature are grouped to obtain a corresponding non-numeric feature same set or a non-numeric feature different set.

3. The system of claim 2, wherein, The main model field information includes a main model field key, and the alternative field information includes an alternative model field key. The feature grouping unit is configured to, if the first field type and the second field type are the non-numeric type, and the main model field key is equal to the alternative model field key, calculate a first hash value corresponding to the first model feature and a second hash value corresponding to the second model feature in the model group by using a hash algorithm respectively. A first correlation coefficient between the first hash value and the second hash value is calculated by using a Jaccard coefficient formula respectively, and each first correlation coefficient is weighted and averaged with a corresponding preset correlation weight to obtain a first similarity score. The first model feature and the second model feature with the first similarity score greater than a first preset threshold are grouped into the non-numeric feature same set, or the first model feature and the second model feature with the first similarity score less than or equal to the first preset threshold are grouped into the non-numeric feature different set. The Jaccard coefficient formula is: X in the Jaccard coefficient formula is the first hash value, and Y is the second hash value. The first model feature includes a main model name, a main model dimension, and a main model field annotation, and the second model feature includes an alternative model name, an alternative model dimension, and an alternative model field annotation.

4. The system of claim 3, wherein, The feature grouping unit is configured to, if the first field type and the second field type are the non-numeric type, and the main model field key is not equal to the alternative model field key, calculate a first word vector corresponding to the first model feature and a second word vector corresponding to the second model feature in the model group by using a word2vec algorithm respectively. A second correlation coefficient between the first word vector and the second word vector is calculated by using a cosine correlation formula respectively, and each second correlation coefficient is weighted and averaged with a corresponding preset correlation weight to obtain a second similarity score. The first model feature and the second model feature with the second similarity score greater than a second preset threshold are grouped into the non-numeric feature same set, or the first model feature and the second model feature with the second similarity score less than or equal to the second preset threshold are grouped into the non-numeric feature different set. The cosine correlation formula is: A in the cosine correlation formula is the first word vector, and B is the second word vector.

5. The system of claim 1, wherein, The first model feature includes main model field information, the second model feature includes alternative field information, the second feature set includes a same set of numerical features and a different set of numerical features; The feature grouping unit is configured to, if the first field type and the second field type are numerical types and the main model field information is equal to the alternative model field information, group the first model feature and the second model feature to obtain a same set of numerical features or a different set of numerical features corresponding thereto. And / or, if the first field type and the second field type are numerical types and the main model field information is not equal to the alternative model field information, group the first model feature and the second model feature to obtain a same set of numerical features or a different set of numerical features corresponding thereto.

6. The system of claim 5, wherein, The main model field information includes a main model field key, and the alternative field information includes an alternative model field key. The feature grouping unit is configured to, if the first field type and the second field type are numerical types and the main model field key is equal to the alternative model field key, calculate a first matching coefficient between the first model feature and the second model feature in the model group by using a Fuzzy Ratio algorithm. Each first matching coefficient is weighted and averaged with a corresponding preset matching weight to obtain a first matching degree score. The first model feature and the second model feature with the first matching degree score greater than a first preset threshold are grouped into the same set of numerical features, or the first model feature and the second model feature with the first matching degree score less than or equal to the first preset threshold are grouped into the different set of numerical features. The first model feature includes a main model name, a main model index, and a main model calculation range, and the second model feature includes an alternative model name, an alternative model index, and an alternative model calculation range.

7. The system of claim 6, wherein The feature grouping unit is configured to, if the first field type and the second field type are numerical types and the main model field information is not equal to the alternative model field information, calculate a second matching coefficient between the first model feature and the second model feature in the model group by using a Fuzzy Partial Ratio algorithm. Each second matching coefficient is weighted and averaged with a corresponding preset matching weight to obtain a second matching degree score. The first model feature and the second model feature with the second matching degree score greater than a first preset threshold are grouped into the same set of numerical features, or the first model feature and the second model feature with the second matching degree score less than or equal to the first preset threshold are grouped into the different set of numerical features.

8. A model building method characterized by, The main model field information includes a main model field key, and the alternative field information includes an alternative model field key. In response to a model selection instruction for a model database, a main model and a plurality of candidate models corresponding to the main model are obtained, the main model is combined with each of the candidate models to obtain at least two model groups; A first model attribute and a first model feature of the main model in each of the model groups and a second model attribute and a second model feature of the candidate models are obtained, and the first model feature and the second model feature are grouped according to the first model attribute and the second model attribute to obtain a corresponding feature set, the feature set including a first feature set and a second feature set; According to the first feature set and the second feature set corresponding to each of the model groups, a fusion model corresponding to each of the model groups is obtained; The first model attribute includes a first field type, and the second model attribute includes a second field type; The first model attribute and the first model feature of the main model in each of the model groups and the second model attribute and the second model feature of the candidate models are obtained, and the first model feature and the second model feature are grouped according to the first model attribute and the second model attribute to obtain a corresponding feature set, including: The first field type and the first model feature of the main model in each of the model groups and the second field type and the second model feature of the candidate models are obtained; The first model feature and the second model feature are grouped according to the first field type and the second field type to obtain a corresponding feature set; According to the first feature set and the second feature set corresponding to each of the model groups, a fusion model corresponding to each of the model groups is obtained, including: The first feature set and the second feature set corresponding to each of the model groups are visually displayed, and in response to a feature modification instruction and / or a feature confirmation instruction for the first feature set and the second feature set, a first target feature set corresponding to the first feature set and a second target feature set corresponding to the second feature set are generated; The first target feature set and the second target feature set are used for model fusion to obtain a fusion model corresponding to each of the model groups.

9. An electronic device, comprising: A processor, a communication interface, a memory and a communication bus are included, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is used to store a computer program; The processor is used to execute the program stored on the memory to implement the method of claim 8.

10. A computer readable storage medium having instructions stored thereon, which when executed by one or more processors, cause the processors to perform the method of claim 8.

Images