Data modeling method and device, computer equipment and storage medium

Abstract

The invention discloses a data modeling method and device, computer equipment and a storage medium, relates to the technical field of artificial intelligence, and is applied to data modeling of a financial big data platform. The original data elements are processed in a standardized mode, and the type labels are automatically marked according to the preset classification rules; in combination with a feature coding technology, converting multi-dimensional data elements into uniform characterization representation; calculating by using a data field standard map and semantic similarity, automatically identifying an abnormal mode, and performing quantitative scoring through a pre-training review model; and finally, performing iterative optimization according to the review index score, and generating a detailed feedback report. According to the method, the modeling efficiency is greatly improved, automation and intelligentization of the field creation, type judgment and standard binding processes are realized, the data standard coverage rate and the modeling quality are effectively improved, the risk of manual review and script deployment errors is reduced, and the modeling efficiency is improved. And the modeling reliability and the operation and maintenance efficiency of the financial big data platform in a complex business scene are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a data modeling method and device, computer equipment and storage medium. BACKGROUND

[0002] Currently, the traditional modeling method of the property insurance big data platform has obvious shortcomings in dealing with complex business requirements. Many existing solutions rely on manual design and experience judgment, which is difficult to cope with large-scale data and fast iteration of business scenarios. This leads to long time consumption and errors in model design, and it is difficult to ensure the consistency of data standards. For example, when processing the pricing data of insurance products, the field naming and type designed manually often lack uniform standards, resulting in data redundancy or incompatibility, which in turn affects subsequent analysis and decision-making. More importantly, these solutions lack automated support in the data standard binding and review process, and manual verification is inefficient, making it difficult to adapt to the high-frequency data model update requirements.

[0003] In the field of data modeling, the determination of field naming and type is the basic link of data modeling, which directly affects the readability and compatibility of data tables. However, due to the involvement of various product lines and data sources in insurance business, field naming often lacks uniformity, leading to ambiguity in querying and analysis. For example, a field representing "premium amount" in a business system may be named "premium" or "fee" in different scenarios, or even have different formats, which increases the complexity of data integration. Moreover, the matching process between fields and data standards is highly dependent on human experience, and lacks automated mechanisms to ensure the accurate correspondence between fields and standards, which makes it difficult for data models to quickly adapt to new requirements in a fast iteration business environment.

[0004] In actual operation, the mapping between standards and fields often needs to be checked manually, which is time-consuming and prone to errors. For example, when processing claim settlement data, if the field is not accurately bound to the unified standard, it may cause statistical deviation of the claim settlement amount, thereby affecting the accuracy of risk assessment. This inefficient matching process not only increases the cost of human labor, but also makes it difficult for data models to quickly respond to complex business scenarios.

[0005] Therefore, how to realize the automated management of field naming and type in a dynamic and variable business scenario, and ensure the accurate and efficient binding of fields and data standards, has become a key problem for the property insurance big data platform to improve modeling efficiency and quality. SUMMARY

[0006] The purpose of the embodiments of the present application is to propose a data modeling method and device, computer equipment and storage medium, to realize the automated management of field naming and type in a dynamic and variable business scenario, and ensure the accurate and efficient binding of fields and data standards.

[0007] To solve the above technical problems, the embodiment of the application provides a data modeling method, which adopts the technical scheme as follows:

[0008] A data modeling method comprises:

[0009] Collecting a set of original data elements and a description of an associated business scenario, performing standardization processing on the set of original data elements to obtain a set of standardized data elements;

[0010] According to a preset data classification rule, determining a type label of each data element in the set of standardized data elements, wherein the type label comprises a numerical type label and a character type label;

[0011] Adding each type label to a data element corresponding to the set of standardized data elements;

[0012] Performing feature coding on data elements of each dimension in the set of standardized data elements to obtain a featureized data element representation;

[0013] According to the featureized data element representation, querying a preset data field standard graph, and adopting similarity calculation to match field annotations in the graph with standard item descriptions to obtain a recommended matching result;

[0014] Extracting an abnormal pattern from the recommended matching result and inputting the abnormal pattern into an evaluation model to obtain an evaluation index score;

[0015] According to the evaluation index score, iteratively optimizing data elements corresponding to the set of standardized data elements, and generating a feedback report of data modeling.

[0016] To solve the above technical problems, the embodiment of the application further provides a data modeling device, which adopts the technical scheme as follows:

[0017] A data modeling device comprises:

[0018] A standardization module configured to collect a set of original data elements and a description of an associated business scenario, perform standardization processing on the set of original data elements to obtain a set of standardized data elements;

[0019] A data label module configured to determine a type label of each data element in the set of standardized data elements according to a preset data classification rule, wherein the type label comprises a numerical type label and a character type label;

[0020] A label embedding module configured to add each type label to a data element corresponding to the set of standardized data elements;

[0021] The feature coding module is configured to code data elements of each dimension in the standardized data element set to obtain a feature data element representation.

[0022] The recommendation matching module is configured to query a preset data field standard graph according to the feature data element representation, and match field annotations in the graph with standard item descriptions by similarity calculation to obtain a recommendation matching result.

[0023] The index review module is configured to extract an abnormal pattern from the recommendation matching result and input the abnormal pattern into a review model to obtain a review index score.

[0024] The modeling feedback module is configured to iteratively optimize data elements corresponding to the standardized data element set according to the review index score, and generate a feedback report of data modeling.

[0025] To solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the technical scheme as follows:

[0026] A computer device includes a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the data modeling method according to any one of the above.

[0027] To solve the above technical problems, the embodiment of the present application further provides a computer readable storage medium, which adopts the technical scheme as follows:

[0028] A computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by a processor to realize the steps of the data modeling method according to any one of the above.

[0029] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0030] The application discloses a data modeling method and device, computer equipment and a storage medium, relates to the technical field of artificial intelligence, and is applied to data modeling of a financial big data platform. The application automatically labels a type tag according to a preset classification rule by standardizing original data elements, and the system can guarantee the consistency and standardization of field types; multi-dimensional data elements are converted into unified characteristic representation by combining feature coding technology; the accurate binding of fields and standard items is realized by using data field standard graph and semantic similarity calculation, the field naming is highly consistent with the business meaning, meanwhile, abnormal patterns are automatically identified and quantitatively scored by a pre-training review model to provide traceable review results for each field; finally, the review index score is iteratively optimized, and a detailed feedback report is generated to provide decision basis and improvement direction for a modeling team. The application not only greatly improves the modeling efficiency, realizes the automation and intelligentization of the field creation, type determination and standard binding process, but also effectively improves the data standard coverage rate and modeling quality, reduces the risk of manual review and script deployment errors, and significantly enhances the modeling reliability and operation efficiency of the financial big data platform in complex business scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the schemes in the application, the drawings needed in the description of the embodiments of the application will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0032] Figure 1 An exemplary system architecture diagram to which the application can be applied is shown;

[0033] Figure 2 A flowchart of one embodiment of the data modeling method according to the application is shown;

[0034] Figure 3 A flowchart of one embodiment of step S205 in the Figure 2

[0035] Figure 4 A structural schematic diagram of one embodiment of the data modeling device according to the application is shown;

[0036] Figure 5 A structural schematic diagram of one embodiment of the Figure 4

[0037] Figure 6 A structural schematic diagram of one embodiment of the computer equipment according to the application is shown. DETAILED DESCRIPTION

[0038] ​​Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terms used in the specification are intended to describe the particular embodiments and are not intended to limit the application; the terms "include" and "have" and their any variations used in the specification and the claims and the above description of drawings are intended to cover the non-exclusive inclusion; the terms "first", "second" and the like used in the specification and the claims and the above description of drawings are intended to distinguish different objects, not to describe a particular order.

[0039] Reference herein to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive or alternative embodiments. It is expressly understood that the embodiments described herein are combinable with each other.

[0040] In order to make the person skilled in the art better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings below.

[0041] As shown in Figure 1 The system architecture 100 can include a terminal device 101, a network 102 and a server 103, and the terminal device 101 can be a notebook computer 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0042] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0043] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing, in addition to the notebook computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an electronic book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer and a desktop computer, etc.

[0044] The server 103 can be a server providing various services, for example, a background server providing support for a page displayed on the terminal device 101.

[0045] It should be noted that the data modeling method provided by the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the data modeling apparatus is generally arranged in a server / terminal device.

[0046] It should be understood that, Figure 1 The number of terminal devices, networks and servers in the system is only illustrative, and the system can have any number of terminal devices, networks and servers according to the implementation needs.

[0047] With reference to Figure 2 , a flow chart of one embodiment of the data modeling method according to the present application is shown. The data modeling method includes the following steps:

[0048] S201, collecting a set of original data elements and an associated business scenario description, and performing standardization processing on the set of original data elements to obtain a set of standardized data elements;

[0049] Specifically, the set of original data elements is often derived from different business systems, databases, files or interfaces, and these data have great differences in format, coding method, field naming, value range, etc., which easily leads to inconsistency in the modeling process. Therefore, the first step of the standardization processing is to clean and pre-process the collected data, including removing redundant fields, merging fields with the same semantics, unifying time and date formats, correcting data units (such as RMB and USD, meters and kilometers, etc.). On this basis, comparison can be made through data dictionary and business rules to ensure that the basic definition of the data elements is consistent with the business semantics.

[0050] For example, for the user identity category field, the identity card number, passport number, and enterprise unified social credit code can be used as the primary key identifier, instead of relying on the serial number within the business system. Further standardization steps also involve the unification of field naming, such as using NLP technology to perform lexical analysis on Chinese field descriptions, extract the root, and convert to standardized English field naming in combination with the organization's data standard specification. In addition, to enhance the degree of automation of standardization, a "field semantic vector library" can be constructed, and a pre-trained language model can be used to vectorize the field description and business scenario description, thereby realizing cross-system and cross-field data semantic normalization.

[0051] It should be noted that,

[0052] Standardization can further integrate automated testing and verification functions, use machine learning algorithms to learn from historical data modeling cases, identify common standardization error patterns, and provide early warning and correction in the new data standardization process to reduce manual intervention and improve standardization efficiency and accuracy.

[0053] S202, according to a preset data classification rule, determining a type label of each data element in the standardized data element set, wherein the type label includes a numerical type label and a character type label;

[0054] Specifically, the determination of the data type label relies on the preset data classification rule library, which is usually constructed by historical experience and industry standards, including common numerical type, character type, date type, Boolean type, and geographic location type classifications. In actual operation, a combination of static rules and dynamic learning is needed to determine the data type of the field.

[0055] For example, for numerical data, it can be detected whether the field value is all numbers, whether there is a decimal point, and whether there are abnormal characters, while combining the business description of the field (such as "amount", "quantity", "ratio") to further confirm. For character type data, the character set, maximum length, and whether there is an encoding mode (such as an identity card number with a length of 18 digits and a mobile phone number with a length of 11 digits) need to be considered. In more complex cases, the same field may exhibit different data types in different business scenarios, at which point a machine learning model (such as a classifier based on decision tree or random forest) can be introduced to automatically give the most likely type prediction using field sample distribution characteristics, missing rate, and number of unique values.

[0056] S203, adding each type label to the data element corresponding to the standardized data element set;

[0057] Specifically, adding various types of labels to the data elements corresponding to the standardized data element set not only involves simple attribute addition, but also requires establishing a structured mapping relationship between the labels and the data elements. First, the system generates a record for each field in the metadata management platform, including the field name, field description, source system, subject domain, data type label, business use, and other information, and binds them through a unique identifier. This approach ensures that labels are not isolated, but become part of the data asset lifecycle management. Second, the storage form of the type label can use JSON or graph database structure to establish a node relationship between the data elements and the labels, making it easy to perform semantic queries and cross-domain analysis. Further, an automated verification mechanism can be built based on the labels, such as when a field is labeled as "amount type," the system will automatically check whether it contains non-numeric characters and whether it has a reasonable precision range to avoid precision loss during storage or calculation.

[0058] S204, encoding the data elements of each dimension in the standardized data element set to obtain a feature data element representation;

[0059] Specifically, the core goal of feature encoding is to convert standardized data elements into numerical representations that can be processed and modeled by computers. In technical implementation, various methods can be used: on the one hand, for categorical variables (such as type labels, field use), One-Hot Encoding, Label Encoding, or Embedding can be used to represent them; on the other hand, for numerical fields, their statistical features (mean, variance, distribution shape, missing rate, etc.) can be extracted as part of the feature vector. At the same time, natural language processing needs to be performed on text field descriptions, such as converting field annotations into semantic vectors through BERT, Word2Vec, etc. This can better capture the semantic similarity between fields. In terms of implementation, feature encoding is not just a single field operation, but also needs to consider multi-dimensional joint features, such as the combined features of "field name + type label + business scenario description."

[0060] In addition, dimension reduction and normalization need to be considered during the encoding process, such as using PCA or AutoEncoder for feature compression to ensure computational efficiency and reduce noise effects. Finally, the feature data element representation forms a structured high-dimensional vector space that can reflect the semantic features of the field and retain its business background information.

[0061] S205, according to the feature data element representation, querying the pre-set data field standard atlas, and using similarity calculation to match the field annotations in the atlas with the standard item descriptions to obtain a recommended matching result;

[0062] Specifically, based on the feature data element representation, similarity calculation can be performed in the preset data field standard atlas to obtain recommended matching results. The standard atlas is usually constructed in the form of a knowledge graph or a semantic network, and contains information such as the definition, alias, business meaning, value constraint, and the like of the standard field.

[0063] In terms of technical implementation, first, the field annotation and description information in the standard atlas need to be converted into a vectorized representation. A Transformer-based pre-trained language model (such as BERT or RoBERTa) can be used to generate semantic embeddings. Then, cosine similarity, Euclidean distance, dot product similarity, and the like can be used to calculate the semantic proximity between the current data element and the standard item. In order to improve the accuracy of the matching, multi-modal features can also be introduced, such as the naming rules (English abbreviations and roots), historical usage frequency, and upstream and downstream dependency relationships of the field, which can be used as additional information for similarity weighting. During the matching process, the system usually outputs a Top-N candidate result, and provides a similarity score and explanatory information (such as why the matching is performed and which feature dimensions have high scores) for each candidate.

[0064] In addition, a feedback learning mechanism can be used to continuously optimize the matching effect. For example, the user's final selection can be recorded as a supervision signal to update the vector retrieval model. This mechanism enables the system to achieve efficient and accurate field standard matching in large-scale scenarios.

[0065] S206, extracting an abnormal pattern from the recommended matching result and inputting it into the review model to obtain a review index score;

[0066] Specifically, extracting an abnormal pattern from the recommended matching result is an important step to ensure the quality of modeling. Abnormal patterns mainly include: a large difference between field annotations and recommended standards, inconsistency between field types and recommended standards, multiple fields mapped to the same standard item (redundancy), and lack of necessary business annotations for fields. In terms of technical implementation, anomaly detection algorithms can be used to identify these situations, such as clustering-based methods (such as DBSCAN and K-Means) to detect semantically isolated fields, or rule engines to define common violation patterns (such as not setting the Decimal precision for amount fields). These abnormal patterns are then input into the review model. The review model is usually a multi-dimensional scoring system, including naming compliance, type rationality, standard matching degree, and dependency integrity. The model can use weighted scoring, hierarchical scoring, or even a machine learning-based regression model to generate a comprehensive score. Finally, the system generates a detailed review index score for each data element or data table, and points out specific problem items, so that business personnel and data governance teams can make targeted corrections.

[0067] For example, in the process of data modeling for a client's claim data of an insurance company, the system collects a field named "fee_total", which records the total amount of the claim in the original system. After standardization, the system finds that it has high semantic similarity with "claim_amount_total" in the standard field library through similarity matching, and the similarity score of the recommended result reaches 0.82. However, in the further abnormality detection process, the system finds that the data type of this field is defined as VARCHAR(50), while according to the recommended standard field, it should be Decimal(18,4). At this time, the rule engine identifies that this is a typical abnormal pattern of "amount type field not stored in numerical type". At the same time, the clustering algorithm also detects another field "total_fee", which has almost the same semantics as "fee_total", but is stored separately in the data table, forming semantic redundancy. The review model inputs these two abnormalities into the review matrix together, deducts points in naming compliance because "fee_total" is not generated according to the naming specification; deducts points again in type rationality because its field type is seriously inconsistent with the business meaning; and also reduces the standard matching degree because the system cannot confirm which of "fee_total" and "total_fee" is the final effective field. Finally, the review model gives the comprehensive score of the data table only 65 points, and points out in the feedback report that the type of "fee_total" needs to be modified to Decimal(18,4), and suggests merging or deleting the redundant field "total_fee".

[0068] S207, iteratively optimize the data elements corresponding to the set of standardized data elements according to the review index score, and generate a feedback report for data modeling.

[0069] Specifically, iterating optimization of data elements according to the evaluation index score means that the system needs to have the ability of automatic correction and interactive improvement. The first step of optimization is to generate improvement suggestions according to low-score items, for example, when the field name is not standardized, the system will automatically recommend a field name that meets the naming standard; when the field type does not match the business scenario, a reasonable data type replacement suggestion will be given; when the field lacks explanation, a preliminary annotation draft will be generated based on similar standard fields. Secondly, the optimization process should have iterative characteristics, that is, after each modification, the system will re-execute standardization, feature encoding and similarity matching to ensure that the correction result can pass the next round of evaluation. Technically, an active learning mechanism can be used to continuously feed back the user's modifications and confirmation results to the model, thereby improving the system's self-learning ability. Finally, the system will generate a complete data modeling feedback report, which includes the overall evaluation score, problem list, improvement suggestions, comparison before and after optimization, and quantitative indicators such as standard coverage rate and compliance rate. The report form can support PDF, HTML or dashboard display to allow different roles (data engineers, governance teams, management) to review.

[0070] In the above embodiment, during the current customer claim data table modeling evaluation, the system's comprehensive evaluation score is 72, which is in the "needs improvement" interval. The specific problem list includes: ① The data type of the field fee_total is defined as VARCHAR(50), which does not match the business meaning and should be corrected to Decimal(18,4); ② The fields total_fee and fee_total have semantic redundancy and should be merged to retain one field; ③ The table-level annotation is missing and needs to be supplemented with business explanation; ④ Some field names do not follow the

subject domain_business root_system abbreviation

[0071] Further, the step of determining a type label of each data element in the standardized data element set according to a preset data classification rule specifically includes:

[0072] The preset classification rule is used to extract features of each data element in the standardized data element set to obtain an attribute description of each data element;

[0073] For the attribute description of each data element, if it meets the regular expression of the numerical type feature, it is determined as a numerical type label, and the corresponding numerical type identifier is generated;

[0074] For the attribute description of each data element, if it meets the regular expression of the character type feature, it is determined as a character type label, and the corresponding character type identifier is generated.

[0075] In this embodiment, the classification rule is not limited to simple regular expression matching, but also combines multi-dimensional feature extraction methods. For example, when processing numerical data, the system will comprehensively detect the value range of the field, whether there is a decimal point, whether negative values are allowed, the discreteness and uniqueness of the numerical value, and other statistical characteristics, so as to improve the accuracy of classification; At the same time, for character data, it will combine character length, coding rules, common patterns (such as 11 digits of mobile phone number, 18 digits of ID card number, and "@" symbol in email) for identification. In addition, in order to deal with the fuzzy situation in complex business scenarios, this scheme introduces an auxiliary classification model based on machine learning to make a secondary discrimination on the boundary field. For example, the postal code field may be marked as numerical type or character type, which can be automatically adjusted by training samples and business scenario description. Finally, each data element will be bound with a clear type label and stored in the metadata management system.

[0076] Through the above steps, the accuracy of data element type determination can be significantly improved, and manual intervention can be reduced, so that the construction of standardized data model is more efficient and reliable.

[0077] Further, the step of feature coding each dimension of data elements in the standardized data element set to obtain a feature data element representation, specifically includes:

[0078] Identify each dimension of data elements in the standardized data element set;

[0079] According to the attribute characteristics of each dimension of data elements, a matching coding algorithm is used for coding conversion to obtain data element features of each dimension;

[0080] Combining the data element features of each dimension, a feature data element representation is generated.

[0081] In the embodiment, the feature encoding is not only a simple conversion of a single attribute of a data element, but also a comprehensive representation combined with multi-dimensional information. Specifically, the system first identifies each dimension in the standardized data element set, including field name, data type, length constraint, value range, missing rate, unique value proportion, business annotation, etc. Then, according to different attribute characteristics, appropriate encoding methods are selected: for categorical features such as data type and label, One-Hot Encoding or Embedding can be used; for numerical features such as value distribution and unique value number, normalization, standardization or binning processing can be used; for text features (field annotation, business description), word vector models (such as Word2Vec, BERT, etc.) can be used to convert them into semantic embedding representation. Finally, the system concatenates or combines the above multi-dimensional feature vectors to form a unified high-dimensional feature representation, which contains both structural attributes and semantic information.

[0082] Through the above steps, comprehensive numerical expression of data elements can be realized, and semantic and structural features are unified modeled, thereby improving the accuracy of standard matching and anomaly detection.

[0083] Further, referring to Figure 3 , according to the feature data element representation, querying the preset data field standard graph, and using similarity calculation to match the field annotation and the standard item description in the graph to obtain the recommended matching result, the steps specifically include:

[0084] S301, according to the feature data element representation, querying in the data field standard graph, and calculating the similarity between the feature data element representation and the standard item description to obtain a similarity score;

[0085] S302, if the similarity score is greater than a preset similarity threshold, the standard item description is selected to generate a preliminary matching result set;

[0086] S303, determining the field annotation matched with the standard item description, extracting the keywords in the field annotation to obtain a keyword set;

[0087] S304, comparing the keyword set with the semantic information of the standard item description to calculate the weighted similarity of the keywords and the standard item description;

[0088] S305, optimizing the preliminary matching result set according to the weighted similarity to obtain an optimized matching result set;

[0089] S306, taking the optimized matching result set as the recommended matching result.

[0090] In this embodiment, the matching process does not only rely on a single similarity calculation, but introduces a multi-level, multi-dimensional semantic matching mechanism to ensure the accuracy and explainability of the matching results. Specifically, in S301, the system first performs semantic vectorization processing on the featureized data element representation and the standard item description in the standard field atlas, converts both into high-dimensional embedding vectors using a deep language model (such as BERT, RoBERTa), and then calculates the similarity score using methods such as cosine similarity and dot product similarity. In the S302 stage, the system will filter out potential candidates based on a pre-set threshold (e.g. 0.75 or 0.8) to form a preliminary matching result set, in order to avoid low-relevance fields being recommended by mistake. In S303, the system further performs word segmentation and keyword extraction on the annotations of the matching fields, such as using TF-IDF, TextRank or deep semantic parsing methods to obtain the key semantic words in the field annotations and form a keyword set. After entering the S304 stage, the system will perform semantic alignment on the keywords and the core words in the standard item description, and introduce a weighting mechanism, such as giving higher weights to business core words (such as "amount", "customer", "policy") and lower weights to stop words or modifying words. In this way, noise words can be filtered out and the accuracy of semantic alignment can be improved. In S305, the system will reorder the preliminary matching result set according to the weighted similarity to form an optimized candidate set, which not only retains the optimal matching item but also gives several suboptimal candidates for further confirmation by humans or systems. Finally in S306, the system outputs the optimized result set as the recommended matching results, and attaches similarity scores, keyword alignment conditions and recommendation reason explanations to each recommended result to ensure the transparency and traceability of the matching results.

[0091] In the technical solution of the present application, the descriptions of the to-be-matched field and the standard field are both converted into semantic vectors, and the similarity scores of the two are calculated, and a threshold is set as a filtering condition. Only the candidate that exceeds the threshold is considered to have sufficient relevance and enters the preliminary matching result set. The core idea of this is to use the numerical similarity to quantify the semantic closeness, and to control the quality of the results through the threshold, avoiding noise fields or irrelevant fields from entering the subsequent steps. For example, suppose there is a to-be-matched field "fee_total", the system calculates its similarity with "claim_amount_total" in the standard field library as 0.83, with "customer_name" as 0.21, and with "policy_number" as 0.34. If the system threshold is set to 0.75, only "claim_amount_total" will be selected into the preliminary result set, and "customer_name" and "policy_number" will be directly filtered out, thereby reducing invalid candidates and ensuring that the optimization step focuses on high-relevance fields.

[0092] Through the above steps, more refined and intelligent field matching can be achieved at the semantic level, effectively reducing the cases of false matching and missing matching, thereby improving the accuracy and reliability of data standardization binding.

[0093] Further, according to the feature data element representation, the similarity between the feature data element representation and the standard item description is calculated in the data field standard graph, and the similarity score is obtained.

[0094] Through the graph query operation, the target standard item description corresponding to the feature data element representation is located in the data field standard graph.

[0095] The cosine similarity algorithm is used to calculate the similarity between the feature data element representation and the target standard item description, and the similarity score is obtained.

[0096] In the present embodiment, the similarity calculation step not only relies on a single cosine similarity calculation, but also combines knowledge graph structure and semantic vectorization technology to enhance the accuracy of matching. Specifically, first, through graph query operation, the subject domain, hierarchical relationship and upstream and downstream dependency relationship of the field are used to quickly locate the target standard item description in the data field standard graph, ensuring accurate search range. Then, the characteristic data element representation and the target standard item description are respectively converted into semantic vectors. This process can use embedding models based on deep learning (such as Word2Vec, BERT, etc.), so that semantic similarity can be effectively quantified. Next, the cosine similarity algorithm is used to calculate the vectors to obtain a similarity score, which is used to measure the closeness of the two in the semantic space. On this basis, business rules can also be combined for weighted correction, such as increasing the weight of core words in the fields of amount, date, customer, etc. to further improve the business rationality of the matching. Finally, the system will generate a set of candidate standard fields with similarity scores for each data element to be matched.

[0097] Through the above steps, efficient and accurate field similarity calculation can be achieved at the semantic level.

[0098] Further, the step of comparing the semantic information of the keyword set and the standard item description to calculate the weighted similarity of the keywords and the standard item description, specifically includes:

[0099] The semantic information of the keyword set is analyzed to obtain the keyword semantic representation;

[0100] The semantic information of the standard item description is analyzed to obtain the standard item semantic representation;

[0101] The cosine similarity between the keyword semantic representation and the standard item semantic representation is calculated to obtain an initial similarity value;

[0102] According to the importance degree of the keyword in the field annotation, a weighted coefficient is assigned to the initial similarity value to obtain a weighted similarity, wherein the importance degree is determined according to the appearance frequency, position and association strength of the keyword in the field annotation.

[0103] In this embodiment, the keyword and the standard item description weighted similarity calculation is mainly realized through the joint modeling of semantic analysis and weight allocation. First, the system will perform semantic analysis on the keyword set extracted from the field annotation. This process can obtain the context information of the keywords through natural language processing techniques such as word segmentation, part-of-speech tagging, and dependency syntax analysis, and use deep embedding models such as Word2Vec, BERT, and ERNIE to convert the keywords into semantic vector representations, thereby capturing their potential semantic features. At the same time, the system will perform the same semantic analysis on the standard item description to generate a semantic vector representation of the standard item, ensuring that both are in the same semantic space. Next, the cosine similarity method is used to calculate the initial similarity value of the keyword vector and the standard item description vector, obtaining the closeness of the two in the semantic space. On this basis, the system will assign different weight coefficients according to the importance of the keywords in the field annotation, for example: if the keyword is located at the beginning of the annotation or in the title position, it will be given a higher weight; if the keyword appears frequently in the annotation or has a strong dependency relationship with other core keywords (such as "amount" and "settlement"), its weight value will also increase. Finally, by combining the initial similarity value and the weighting coefficient, a weighted similarity score is generated.

[0104] For example, assuming the data field annotation is "settlement amount total (total cost)", the system will first extract the keyword set

settlement, amount, total, total cost

[0105] Through the above steps, the influence of key business words on the matching result can be effectively highlighted, making the correspondence between the field and the standard item more consistent with business logic, and improving the precision and reliability of field standardization binding.

[0106] Further, the step of extracting abnormal patterns from the recommended matching results and inputting them into the review model to obtain the review index score includes:

[0107] Identify abnormal patterns from the recommended matching results that do not conform to the preset business logic rules;

[0108] Input the abnormal patterns as input data to the pre-trained review model;

[0109] Each abnormal pattern is evaluated one by one by the business rules and review criteria built in the review model;

[0110] The review results of each abnormal pattern are output, including whether it passes the review and the corresponding review index score.

[0111] In this embodiment, the review model not only relies on simple rule matching, but also combines the multi-dimensional analysis capabilities of machine learning and knowledge graph to identify and evaluate abnormal patterns. Specifically, the system first extracts fields that violate the pre-set business logic from the recommended matching results, such as field type and business scenario mismatch, field naming and standard specification inconsistency, field mapping to multiple standard items or no matching item, etc. These abnormal patterns are converted into structured feature vectors as input to the pre-trained review model. The review model integrates a large number of business rules, field specifications and historical review experience, which can automatically evaluate each abnormal pattern, including scoring in multiple dimensions such as naming specification, type matching, standard binding correctness, and field level attribution. The model outputs the review results of each abnormal pattern, including whether it passes the review, the score of each dimension, and improvement suggestions, providing a clear optimization direction for data modeling. At the same time, the model can also record the review history.

[0112] Through the above steps, abnormal patterns in data modeling can be automatically identified and quantified, providing objective and quantifiable indicators for data review, thereby improving review efficiency and accuracy.

[0113] In the above embodiment, the present application discloses a data modeling method, which is applied to the data modeling of a financial big data platform and relates to the field of artificial intelligence technology. The present application standardizes the original data elements and automatically labels type tags according to pre-set classification rules, so that the system can ensure the consistency and standardization of field types; combines feature encoding technology to convert multi-dimensional data elements into unified feature representation; uses data field standard graph and semantic similarity calculation to realize accurate binding of fields and standard items, ensure high consistency between field naming and business meaning, automatically identify abnormal patterns and quantitatively score through a pre-trained review model, provide traceable review results for each field; finally, iterate and optimize according to the review index score and generate a detailed feedback report to provide decision basis and improvement direction for the modeling team. The present application not only greatly improves the modeling efficiency, realizes automation and intelligentization of field creation, type determination and standard binding process, but also effectively improves the data standard coverage rate and modeling quality, reduces the risk of manual review and script deployment errors, significantly enhances the modeling reliability and operation efficiency of the financial big data platform in complex business scenarios.

[0114] In this embodiment, the electronic device (e.g. Figure 1 The server shown in the figure can receive instructions or obtain data through wired or wireless connections. It should be noted that the wireless connection can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (ultra wideband) connection, and other now known or future developed wireless connection.

[0115] It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned original data information, the above-mentioned original data information can also be stored in a node of a block chain.

[0116] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, a series of data blocks associated using cryptographic methods, each containing a batch of network transaction information for verifying the validity (anti-fraud) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0117] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.

[0118] Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by computer readable instructions instructing related hardware, which can be stored in a computer readable storage medium. The program can include the processes of the above-mentioned embodiments when executed, wherein the storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM) and the like.

[0120] It should be understood that, although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise explicitly stated herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0121] Further referring to Figure 4 , as an implementation of the method shown in the above Figure 2 , the present application provides an embodiment of a data modeling device, which corresponds to the method embodiment shown in Figure 2 , and the device can be specifically applied to various electronic devices.

[0122] As shown in Figure 4 , the data modeling device 400 described in the embodiment includes:

[0123] The standardization module 401 is configured to collect a set of original data elements and an associated business scenario description, and perform standardization processing on the set of original data elements to obtain a set of standardized data elements.

[0124] The data label module 402 is configured to determine a type label of each data element in the set of standardized data elements according to a preset data classification rule, wherein the type label includes a numerical type label and a character type label.

[0125] The label embedding module 403 is configured to add each type label to a data element corresponding to the set of standardized data elements.

[0126] The feature encoding module 404 is configured to perform feature encoding on data elements of each dimension in the set of standardized data elements to obtain a featureized data element representation.

[0127] The recommendation matching module 405 is configured to query a preset data field standard graph according to the featureized data element representation, and match field annotations in the graph with standard item descriptions using similarity calculation to obtain a recommendation matching result.

[0128] The index review module 406 is configured to extract an abnormal pattern from the recommendation matching result and input it to a review model to obtain a review index score.

[0129] The modeling feedback module 407 is configured to iteratively optimize the data elements in the standardized data element set according to the evaluation index score, and generate a feedback report of data modeling.

[0130] Further, the data labeling module 402 specifically includes:

[0131] The feature extraction unit is configured to extract features of each data element in the standardized data element set according to a preset classification rule, and obtain an attribute description of each data element.

[0132] The first determination unit is configured to determine, for the attribute description of each data element, whether the regular expression of the numerical type feature is met, and if so, determine that the data element is of the numerical type label, and generate a corresponding numerical type identifier.

[0133] The second determination unit is configured to determine, for the attribute description of each data element, whether the regular expression of the character type feature is met, and if so, determine that the data element is of the character type label, and generate a corresponding character type identifier.

[0134] Further, the feature encoding module 404 specifically includes:

[0135] The element recognition unit is configured to recognize the data elements of each dimension in the standardized data element set.

[0136] The encoding conversion unit is configured to convert the data elements of each dimension according to the attribute features of the data elements of each dimension, and obtain the feature of the data element of each dimension.

[0137] The feature combination unit is configured to combine the features of the data elements of each dimension, and generate a feature data element representation.

[0138] Further, please refer to Figure 5 The recommendation matching module 405 specifically includes:

[0139] The graph query unit 501 is configured to query the data field standard graph according to the feature data element representation, calculate the similarity between the feature data element representation and the standard item description, and obtain a similarity score.

[0140] The preliminary matching unit 502 is configured to select the standard item description to generate a preliminary matching result set if the similarity score is greater than a preset similarity threshold.

[0141] The keyword extraction unit 503 is configured to determine the field annotation matched with the standard item description, extract keywords in the field annotation, and obtain a keyword set.

[0142] The semantic analysis unit 504 is configured to compare the semantic information of the keyword set and the standard item description, and calculate the weighted similarity between the keywords and the standard item description.

[0143] The result optimization unit 505 is configured to optimize the preliminary matching result set according to the weighted similarity to obtain an optimized matching result set.

[0144] The recommendation matching unit 506 is configured to take the optimized matching result set as a recommended matching result.

[0145] Further, the atlas query unit 501 specifically includes:

[0146] The description positioning subunit is configured to locate a target standard item description corresponding to the feature data element representation in the data field standard atlas through an atlas query operation.

[0147] The similarity calculation subunit is configured to calculate the similarity between the feature data element representation and the target standard item description by using a cosine similarity algorithm to obtain a similarity score.

[0148] Further, the semantic analysis unit 504 specifically includes:

[0149] The first semantic analysis subunit is configured to analyze the semantic information of the keyword set to obtain a keyword semantic representation.

[0150] The second semantic analysis subunit is configured to analyze the semantic information of the standard item description to obtain a standard item semantic representation.

[0151] The cosine similarity subunit is configured to calculate the cosine similarity between the keyword semantic representation and the standard item semantic representation to obtain an initial similarity value.

[0152] The weighting assignment subunit is configured to assign a weighting coefficient to the initial similarity value according to the importance degree of the keyword in the field annotation to obtain a weighted similarity, wherein the importance degree is determined according to the appearance frequency, position and association strength with other keywords of the keyword in the field annotation.

[0153] Further, the index review module 406 specifically includes:

[0154] The anomaly identification unit is configured to identify an abnormal pattern that does not conform to a preset business logic rule from the recommended matching result.

[0155] The input data unit is configured to pass the abnormal pattern to a pre-trained review model as input data.

[0156] The review evaluation unit is configured to evaluate each abnormal pattern one by one through the business rules and review standards built in the review model.

[0157] The index score output unit is configured to output a review result of each abnormal pattern, wherein the review result includes whether it passes the review and a corresponding review index score.

[0158] In the above embodiment, the application discloses a data modeling device, relates to the technical field of artificial intelligence, and is applied to data modeling of a financial big data platform. The application can guarantee the consistency and standardization of field types by standardizing original data elements and automatically labeling type tags according to preset classification rules, can convert multi-dimensional data elements into unified characteristic representations by combining feature coding technology, can realize accurate binding of fields and standard items by using data field standard graph and semantic similarity calculation, can ensure that field naming and business meaning are highly consistent, can automatically identify abnormal patterns and perform quantitative scoring through a pre-training review model to provide traceable review results for each field, and finally, can generate detailed feedback reports for modeling teams to provide decision-making basis and improvement directions. The application not only greatly improves modeling efficiency, realizes automation and intelligentization of the field creation, type determination and standard binding process, but also effectively improves data standard coverage and modeling quality, reduces the risk of manual review and script deployment errors, and significantly enhances the modeling reliability and operation efficiency of the financial big data platform in complex business scenarios.

[0159] To solve the above technical problems, the application embodiment further provides a computer device. For details, please refer to Figure 6 , Figure 6 The basic structure block diagram of the computer device of the embodiment is shown in the figure.

[0160] The computer device 6 includes a memory 61, a processor 62 and a network interface 63 which are connected to each other through a system bus. It should be pointed out that only the computer device 6 with the memory 61, the processor 62 and the network interface 63 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device here is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0161] The computer device can be a desktop computer, a notebook computer, a palm computer and a cloud server, etc. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad or a voice control device, etc.

[0162] The memory 61 includes at least one type of readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or a memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 6. Of course, the memory 61 can also include both the internal storage unit and the external storage device of the computer device 6. In this embodiment, the memory 61 is generally used to store an operating system and various application software installed on the computer device 6, such as computer readable instructions of the data modeling method, etc. In addition, the memory 61 can also be used to temporarily store various data that have been output or will be output.

[0163] The processor 62 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer readable instructions or process data stored in the memory 61, such as computer readable instructions of the data modeling method.

[0164] The network interface 63 can include a wireless network interface or a wired network interface, which is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0165] The present application also provides an embodiment, i.e., to provide a computer device, which includes a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to implement the steps of the data modeling method as described above, i.e., to implement:

[0166] A data modeling method includes:

[0167] Collecting a set of original data elements and an associated business scenario description, performing standardization processing on the set of original data elements to obtain a set of standardized data elements;

[0168] determining a type label of each data element in the standardized data element set according to a preset data classification rule, wherein the type label comprises a numerical type label and a character type label;

[0169] adding each type label to a corresponding data element of the standardized data element set;

[0170] performing feature coding on data elements of each dimension in the standardized data element set to obtain a feature data element representation;

[0171] querying a preset data field standard graph according to the feature data element representation, and matching field annotations in the graph with standard item descriptions using similarity calculation to obtain a recommended matching result;

[0172] extracting an abnormal pattern from the recommended matching result and inputting it into an evaluation model to obtain an evaluation index score;

[0173] iteratively optimizing data elements corresponding to the standardized data element set according to the evaluation index score, and generating a feedback report for data modeling.

[0174] The application also provides another embodiment, namely providing a computer readable storage medium storing computer readable instructions executable by at least one processor to enable the at least one processor to perform the steps of the data modeling method as described above, namely to implement:

[0175] A data modeling method comprises:

[0176] collecting an original data element set and an associated business scenario description, performing standardization processing on the original data element set to obtain a standardized data element set;

[0177] determining a type label of each data element in the standardized data element set according to a preset data classification rule, wherein the type label comprises a numerical type label and a character type label;

[0178] adding each type label to a corresponding data element of the standardized data element set;

[0179] performing feature coding on data elements of each dimension in the standardized data element set to obtain a feature data element representation;

[0180] querying a preset data field standard graph according to the feature data element representation, and matching field annotations in the graph with standard item descriptions using similarity calculation to obtain a recommended matching result;

[0181] extracting an abnormal pattern from the recommended matching result and inputting it into an evaluation model to obtain an evaluation index score;

[0182] According to the evaluation index score, the data elements corresponding to the standardized data element set are iteratively optimized, and a feedback report of data modeling is generated.

[0183] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0184] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, small computers, mainframe computers, distributed computing environments including any of the above systems or devices. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0185] It should be noted that the non-company software tools or components appearing in various embodiments of the present application are only illustrative and do not represent actual use.

[0186] Obviously, the above-described embodiments are only a part of the embodiments of the present application, and not all embodiments. The preferred embodiments of the present application are given in the drawings, but do not limit the patent scope of the present application. The present application can be realized in many different forms, and on the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent replacements to some technical features. Any equivalent structure made by using the contents of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.

Claims

1. A data modeling method, characterized in that, include: Collect the original set of data elements and the associated business scenario description, and perform standardization processing on the original set of data elements to obtain a standardized set of data elements. According to the preset data classification rules, the type label of each data element in the standardized data element set is determined, wherein the type label includes numeric labels and character labels; Add each of the aforementioned type labels to the corresponding data elements of the standardized data element set; Feature encoding is performed on the data elements of each dimension in the standardized data element set to obtain a feature-based data element representation; Based on the characteristic data element representation, a preset data field standard map is queried, and the field annotations and standard item descriptions in the map are matched using similarity calculation to obtain recommended matching results; Anomalies are extracted from the recommended matching results and input into the review model to obtain review index scores. Based on the scores of the evaluation indicators, iteratively optimize the data elements corresponding to the standardized data element set, and generate a data modeling feedback report.

2. The data modeling method as described in claim 1, characterized in that, The step of determining the type label of each data element in the standardized data element set according to the preset data classification rules specifically includes: Feature extraction is performed on each data element in the standardized data element set using preset classification rules to obtain the attribute description of each data element; For each data element's attribute description, if it matches a regular expression with numeric characteristics, it is determined to be a numeric tag, and a corresponding numeric identifier is generated. For each data element's attribute description, if it matches a regular expression with character characteristics, it is determined to be a character tag, and a corresponding character identifier is generated.

3. The data modeling method as described in claim 1, characterized in that, The step of performing feature encoding on data elements of each dimension in the standardized data element set to obtain featured data element representations specifically includes: Identify data elements in each dimension of the standardized data element set; Based on the attribute characteristics of data elements in each dimension, a matching encoding algorithm is used for encoding conversion to obtain the data element characteristics in each dimension; The data element features from various dimensions are combined to generate the characteristic data element representation.

4. The data modeling method as described in claim 1, characterized in that, The step of querying a preset data field standard map based on the characteristic data element representation, and using similarity calculation to match field annotations and standard item descriptions in the map to obtain recommended matching results specifically includes: Based on the characteristic data element representation, a query is performed in the standard graph of the data field, and the similarity between the characteristic data element representation and the standard item description is calculated to obtain a similarity score; If the similarity score is greater than the preset similarity threshold, then a standard item description is selected to generate a preliminary matching result set; Identify field annotations that match the standard item description, extract keywords from the field annotations, and obtain a keyword set; By comparing the semantic information of the keyword set with that of the standard item description, the weighted similarity between the keywords and the standard item description is calculated. The preliminary matching result set is optimized based on the weighted similarity to obtain the optimized matching result set. The optimized matching result set is used as the recommended matching result.

5. The data modeling method as described in claim 4, characterized in that, The step of querying the standard graph of the data field based on the characteristic data element representation and calculating the similarity between the characteristic data element representation and the standard item description to obtain a similarity score specifically includes: Through the graph query operation, the target standard item description corresponding to the characteristic data element representation is located in the standard graph of the data field; The cosine similarity algorithm is used to calculate the similarity between the featured data element representation and the target standard item description, and the similarity score is obtained.

6. The data modeling method as described in claim 4, characterized in that, The step of comparing the semantic information of the keyword set with that of the standard item description, and calculating the weighted similarity between the keywords and the standard item description, specifically includes: The semantic information of the keyword set is parsed to obtain the semantic representation of the keywords; The semantic information described by the standard item is parsed to obtain the semantic representation of the standard item; Calculate the cosine similarity between the semantic representation of the keyword and the semantic representation of the standard item to obtain an initial similarity value; Based on the importance of the keywords in the field annotations, a weighted coefficient is assigned to the initial similarity value to obtain a weighted similarity. The importance is determined based on the frequency and position of the keywords in the field annotations and the strength of their association with other keywords.

7. The data modeling method as described in claim 1, characterized in that, The step of extracting abnormal patterns from the recommendation matching results and inputting them into the review model to obtain review index scores specifically includes: Identify abnormal patterns from the recommended matching results that do not conform to preset business logic rules; The abnormal pattern is used as input data and passed to the pre-trained review model; Each of the aforementioned abnormal patterns is evaluated one by one using the business rules and review criteria built into the review model. Output the review result for each of the aforementioned abnormal patterns, wherein the review result includes whether the review is passed and the corresponding review indicator score.

8. A data modeling apparatus, characterized in that, include: The standardization module is used to collect the original set of data elements and the associated business scenario description, and to perform standardization processing on the original set of data elements to obtain a standardized set of data elements. The data label module is used to determine the type label of each data element in the standardized data element set according to the preset data classification rules, wherein the type label includes numeric labels and character labels; A tag embedding module is used to add each of the aforementioned type tags to the data elements corresponding to the standardized data element set; The feature encoding module is used to perform feature encoding on data elements of each dimension in the standardized data element set to obtain a feature-based data element representation. The recommendation matching module is used to query a preset data field standard map based on the characteristic data element representation, and to use similarity calculation to match the field annotations and standard item descriptions in the map to obtain the recommendation matching result; The indicator review module is used to extract abnormal patterns from the recommendation matching results and input them into the review model to obtain the review indicator scores. The modeling feedback module is used to iteratively optimize the data elements corresponding to the standardized data element set based on the review index scores, and generate a data modeling feedback report.

9. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data modeling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data modeling method as described in any one of claims 1 to 7.

Images