Field rule matching recommendation method and device based on multi-dimensional feature fusion

Abstract

The invention relates to the technical field of data processing, and provides a field rule matching recommendation method and device based on multi-dimensional feature fusion. Obtaining user input information, wherein the user input information comprises a target data table and a current business target; performing feature extraction on fields in the target data table through a field feature extraction module to obtain field multi-dimensional features; performing attribute modeling on the data quality rules in the preset rule base through rule attribute modeling to obtain rule multi-dimensional attributes; calculating matching scores of the field multi-dimensional features and the rule multi-dimensional attributes through a multi-dimensional scoring model; and dynamically adjusting the weight of each dimension in the multi-dimensional scoring model according to the service target priority through a rule recommendation engine, and outputting Top-N recommendation rules based on the adjusted weight and the matching score. According to the scheme, the problems of low quality rule matching precision, poor efficiency and insufficient scene coverage of a traditional method can be solved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a field rule matching and recommendation method and apparatus based on multi-dimensional feature fusion. Background Technology

[0002] Data quality governance is a core component of enterprise digital transformation, and one of its key challenges is automatically recommending applicable quality rules for data fields. Traditional methods have the following drawbacks: (1) Rule matching relies on human experience: experts need to manually configure the mapping relationship between rules and fields, which is inefficient and prone to omissions.

[0003] (2) Simple matching logic: Most systems only perform simple matching based on field name or type, ignoring key factors such as semantic similarity and business target suitability.

[0004] (3) Insufficient scenario coverage: The existing rule base is mostly general rules and lacks the ability to adapt to specific business processes (such as financial auditing and supply chain management).

[0005] (4) Weak dynamic adjustment capability: unable to dynamically adjust the recommendation strategy based on the real-time business objectives input by the user (such as "prioritizing data accuracy").

[0006] Therefore, a method is needed to solve the problems of low accuracy, poor efficiency, and insufficient scenario coverage in traditional methods for quality rule matching. Summary of the Invention

[0007] This invention provides a field rule matching recommendation method and apparatus based on multi-dimensional feature fusion, which can solve the problems of low accuracy, poor efficiency and insufficient scene coverage of traditional methods.

[0008] According to one aspect of the present invention, a field rule matching recommendation method based on multi-dimensional feature fusion is provided, comprising: Obtain user input information, which includes the target data table and the current business objective; The field feature extraction module extracts features from the fields in the target data table to obtain multi-dimensional features of the fields; By performing attribute modeling on the data quality rules in the preset rule base, multi-dimensional attributes of the rules are obtained. The matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule is calculated using a multi-dimensional scoring model. The rule-based recommendation engine dynamically adjusts the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives, and outputs the Top-N recommendation rules based on the adjusted weights and matching scores.

[0009] Optionally, the multi-dimensional features of the field include textual semantic features, data type features, statistical features, and business process tags. The textual semantic features are obtained by converting the field description into a high-dimensional semantic vector using a BERT pre-trained model. The data type features are the data type identifiers of the field, including numeric, date, and string types. The statistical features include the missing rate, outlier ratio, mean, standard deviation, and number of unique values ​​for the field's sample values. The business process tags are the identifiers of the business processes to which the field belongs, including sales processes, supply chain management processes, and financial audit processes.

[0010] Optionally, the application logic of the statistical features includes: when the missing rate of a field is greater than a preset threshold, automatically marking the field as needing to be preferentially matched with non-empty check rules after the field features are extracted; when the proportion of outliers in a field is greater than a preset threshold, automatically marking the field as needing to be matched with outlier detection rules.

[0011] Optionally, the rule's multi-dimensional attributes include semantic features, action type, priority, specialization, and complexity; wherein, the semantic features are obtained by extracting core business semantics from the rule name; the action type is the data type identifier to which the rule applies, corresponding to the data type features of the field; the priority is the importance level of the rule in data quality governance; the specialization is the specificity level of the rule's applicable scenario; and the complexity is the level of technical difficulty and computational resource consumption in implementing the rule.

[0012] Optionally, the evaluation dimensions of the multi-dimensional scoring model include semantic similarity, generality, priority, specialization, complexity, and alignment with business objectives. The semantic similarity is obtained by calculating the cosine similarity between the field text semantic vector and the rule semantic feature vector, with a value range of [0,1]. The generality is obtained by normalizing the predefined rule action type and field data type matching matrix, with a value range of [0,1]. The priority is obtained by normalizing the rule's preset priority level and field statistical features after dynamic adjustment, with a value range of [0,1]. The specialization is obtained by reverse normalization, converting the specialization to a value range of [0.2,1.0]. The complexity is obtained by converting it to a cost coefficient and normalizing it to a value range of [0.2,1.0]. The business goal alignment is obtained by constructing a business process and rule mapping matrix and calculating it using TF-IDF text similarity, with a value range of [0,1]. The matching score is calculated using a weighted linear combination model. The calculation formula is: Sere=Σ(Wi×Di), where Wi is the weight of each evaluation dimension, the sum of the weights of all dimensions is 1, and Di is the normalized score of each evaluation dimension.

[0013] Optionally, the rule recommendation engine uses the hierarchical analysis method to achieve dynamic weight allocation for each evaluation dimension; In core business scenarios, the weight of semantic similarity is adjusted to 30%, and the weight of rule priority is adjusted to 25%; in general governance scenarios, the weight of type compatibility is adjusted to 20%, and the weight of implementation efficiency is adjusted to 10%. The output of the rule recommendation engine includes a list of Top-N recommendation rules sorted in descending order of matching score, and a radar chart of the dimensional scores of each recommendation rule; The rule recommendation engine supports dynamic model optimization, including training the weight coefficients of each evaluation dimension using historical matching data and updating the parameters of the multi-dimensional scoring model monthly. It also supports manual calibration of rule multi-dimensional attributes, including adjusting the priority, specialization, and complexity level of rules to adapt to changes in business needs.

[0014] Optionally, the method further includes: Edit the attributes of rules in the rule base, including modifying the semantic features, action type, priority, specialization degree, and complexity of the rules; Add data quality rules to the rule base and automatically complete multi-dimensional attribute modeling for the new rules; Semantic adaptation of multilingual field descriptions and multilingual rule descriptions is achieved by leveraging the multilingual capabilities of the BERT pre-trained model to realize semantic matching of fields and rules in different languages.

[0015] According to another aspect of the present invention, a field rule matching and recommendation device based on multi-dimensional feature fusion is provided, comprising: The acquisition unit is used to acquire user input information, which includes a target data table and the current business objective. The extraction unit is used to extract features from the fields in the target data table through the field feature extraction module to obtain multi-dimensional features of the fields; The modeling unit is used to perform attribute modeling on the data quality rules in the preset rule base through rule attribute modeling, so as to obtain multi-dimensional attributes of the rules. The calculation unit is used to calculate the matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule through a multi-dimensional scoring model; The output unit is used to dynamically adjust the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives through the rule recommendation engine, and output the Top-N recommendation rules based on the adjusted weights and matching scores.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the field rule matching recommendation method based on multi-dimensional feature fusion as described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the field rule matching recommendation method based on multi-dimensional feature fusion as described in any embodiment of the present invention.

[0018] This invention provides a field rule matching and recommendation method and apparatus based on multi-dimensional feature fusion. Users only need to input a data table and business objectives to automatically recommend Top-N applicable rules. The method integrates six dimensions: semantic similarity, generality, priority, rule specialization, complexity, and business objective adaptability to improve matching accuracy.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a field rule matching and recommendation method based on multi-dimensional feature fusion provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a field rule matching and recommendation device based on multi-dimensional feature fusion according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the field rule matching recommendation method based on multi-dimensional feature fusion according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] like Figure 1 As shown, this embodiment of the invention provides a field rule matching recommendation method based on multi-dimensional feature fusion, which may include the following steps: S110. Obtain user input information, which includes the target data table and the current business objective.

[0025] Users need to provide two key types of information: First, the target data table, which is the specific data table that needs to be treated for data quality governance. The table contains multiple fields to be matched with rules and is the only data source for subsequent field feature extraction. Second, the current business goal, which is the specific governance direction that the user expects to achieve through rule matching. This goal directly determines the weight adjustment logic of each dimension in the subsequent multi-dimensional scoring model to ensure that the recommendation results match the user's core needs.

[0026] S120. The field feature extraction module extracts features from the fields in the target data table to obtain multi-dimensional features of the fields.

[0027] The core objective of the field feature extraction module is to extract key information from the fields of the target data table that reflects its business attributes, data characteristics, and scenario affiliation, forming a complete feature system to provide field-side basis for accurate matching with rules.

[0028] In this embodiment of the invention, the multi-dimensional features of the field include textual semantic features, data type features, statistical features, and business process tags. The textual semantic features are obtained by converting the field description into a high-dimensional semantic vector through a BERT pre-trained model. The data type features are the data type identifiers of the field, including numeric, date, and string types. The statistical features include the missing rate, outlier ratio, mean, standard deviation, and number of unique values ​​of the field's sample values. The business process tags are the identifiers of the business processes to which the field belongs, including sales processes, supply chain management processes, and financial audit processes.

[0029] Textual semantic features are core features mined from field descriptions that reflect the business attributes of fields. Their core objective is to achieve accurate matching between the semantic information of field descriptions and rule semantics, avoiding biases caused by relying solely on literal matching of field names. Field descriptions are the data source; these descriptions directly contain the business meaning of the fields and are the core carrier of semantic matching. A BERT pre-trained model is used for processing. The natural language text of the field descriptions is input into the BERT model, which, through understanding the contextual semantics, converts the text into high-dimensional semantic vectors. These vectors not only capture literal meaning but also identify deep business relationships. As the core basis for matching field and rule semantics, subsequent rule attribute modeling extracts the semantic features of the rules. By calculating the similarity between the field semantic vector and the rule semantic vector, it is determined whether their business meanings align.

[0030] Data type characteristics are the fundamental identifiers of fields, directly determining the scope of rule applicability. They serve as a hard threshold for field-rule matching, preventing invalid matches caused by incompatibility between rules and field types. The data type identifier of a field, identified automatically by the system or manually, clearly defines the data type category to which the field belongs, and is the direct basis for the type matching effect of the rule. Data type characteristics include three core data types, including: Numeric type: Suitable for fields that can be quantified, such as quantity, amount, and indicators, like "order amount", "inventory quantity", and "product unit price"; Date type: Suitable for fields representing time and date, such as "order creation date", "contract expiration date", and "financial closing date"; String type: Suitable for fields representing text, codes, and identifiers that are not quantitative, such as "customer number", "product model", and "invoice number".

[0031] Rule attribute modeling defines the type of action of a rule. Only when the data type characteristics of a field are consistent with the type of action of the rule can the rule have the basic conditions to adapt to that field.

[0032] Statistical features are features extracted from field sample values ​​that reflect the data distribution patterns. Their core objective is to dynamically adjust the recommendation priority of rules based on the current data quality of the field, so that the matching results better meet the actual governance needs of the field.

[0033] The field's sample values ​​are used to generate features through statistical analysis of these values, encompassing five specific statistical indicators: Missing rate: The proportion of null (or invalid) values ​​in a field's sample values ​​out of the total number of samples, reflecting the completeness of the field's data (e.g., a missing rate of 30% for a "Contact Information" field indicates that there is a significant amount of missing data in that field). Outlier Ratio: The proportion of field sample values ​​that are outside the normal range (e.g., determined by box plots or standard deviation method) to the total number of samples. It reflects the accuracy of the field data (e.g., if the outlier ratio of a "product unit price" field is 15%, it indicates that there are many abnormal price data in this field). Average: For numerical fields, calculate the arithmetic mean of the sample values ​​to reflect the overall level of the field data; Standard deviation: For numerical fields, it calculates the dispersion of sample values ​​and reflects the data fluctuation. Number of unique values: The number of unique values ​​in the field's sample values, reflecting the diversity of the field data.

[0034] The priority of rules is dynamically adjusted. The subsequent multi-dimensional scoring model will combine statistical features to adjust the priority scores of rules, ensuring that the recommended rules can accurately solve the data quality problems of the fields.

[0035] Business process tags are features that identify the business scenario to which a field belongs. Their core goal is to make field rule matching more closely aligned with specific business needs, avoiding the problem of insufficient adaptability of general rules in vertical business scenarios. Identifying the business process to which a field belongs—that is, assigning a clear business process tag to a field by associating it with its application scenario within the enterprise's business system—is the core basis for scenario-based rule matching. It includes three core business processes, specifically: Sales Process: Applicable to fields related to the enterprise's sales process, such as "Order Number", "Sales Amount", "Customer Order Time", and "Product Sales Volume"; Supply chain management process: Applicable to fields related to enterprise supply chain links, such as "inventory quantity", "purchase order number", "logistics delivery time" and "supplier code"; Financial audit process: Applicable to fields related to corporate finance and auditing processes, such as "invoice amount", "tax amount", "financial closing date", and "cost accounting value".

[0036] In this embodiment of the invention, the application logic of statistical features includes: when the missing rate of a field is greater than a preset threshold, the field is automatically marked as needing to be preferentially matched with non-empty check rules after the field features are extracted; when the proportion of outliers in a field is greater than a preset threshold, the field is automatically marked as needing to be adapted to outlier detection rules.

[0037] After extracting statistical features from the fields, the system assesses the missing rate and outlier ratio based on preset thresholds, and assigns priority to corresponding rules accordingly. If the missing rate exceeds the preset threshold (e.g., 20%), it indicates a high number of null values ​​and insufficient data integrity. The system automatically marks this field as requiring priority matching with non-null check rules to address missing data issues. If the outlier ratio exceeds the preset threshold (e.g., 15%), it indicates a high number of abnormal data and questionable data accuracy. The system automatically marks this field as requiring matching with outlier detection rules (e.g., "box plot outlier identification" or "standard deviation outlier filtering") to specifically address data anomalies. The core of this logic is to strongly correlate rule matching with the actual data quality of the fields, avoiding indiscriminate recommendations and improving matching accuracy.

[0038] S130. By performing attribute modeling on the data quality rules in the preset rule base, the multi-dimensional attributes of the rules are obtained.

[0039] For data quality rules in the preset rule base, the applicable scenarios, importance, and implementation difficulty of each rule are clarified by attribute decomposition, forming an attribute system on the rule side, which provides a rule-side basis for matching with fields.

[0040] In this embodiment of the invention, the multi-dimensional attributes of the rule include semantic features, action type, priority, specialization degree, and complexity; wherein, the semantic features are obtained by extracting the core business semantics from the rule name; the action type is the data type identifier to which the rule applies, corresponding to the data type feature of the field; the priority is the importance level of the rule in data quality governance; the specialization degree is the specificity level of the rule's applicable scenario; and the complexity is the level of technical difficulty and computational resource consumption in implementing the rule.

[0041] Semantic features are the core business semantics extracted from rule names and are the key basis for determining whether a rule aligns with the business logic of a field. The extraction logic is as follows: non-core information is directly extracted from the rule name, retaining the core business meaning to form the rule's semantic identifier. The core function of this attribute is to match it with the textual semantic features of the field. By calculating the semantic similarity between the two, rules consistent with the field's business meaning are selected, avoiding the problem of rule functionality being disconnected from the field's business attributes.

[0042] The `applicabilityType` attribute identifies the data type to which a rule applies, directly corresponding to the data type characteristics of the field. It serves as a hard filter condition for rule matching. Its definition logic is as follows: based on the rule's function, determine the data types of the fields it can apply to, forming a type-matching identifier for the rule. The core function of this attribute is to quickly filter out incompatible rules. Only when the rule's `applicabilityType` matches the field's data type characteristics does the rule have a basis for matching, avoiding invalid matches.

[0043] Priority is the level of importance of a rule in data quality governance, used to determine the recommendation order when adapting multiple rules. Its logic is based on setting the level according to the degree of influence of each rule on the core data quality objectives. The core function of this attribute is to dynamically adjust the recommendation weights based on field statistical features. High-priority rules that have a greater impact on data quality will receive higher scores during matching, ensuring that key data quality issues are addressed primarily through rules.

[0044] Specialization level refers to the specificity of a rule's applicable scenarios, used to adapt to field matching needs in different business scenarios. Its classification logic is as follows: the level is set according to the breadth of the rule's applicable scope; the narrower the applicable scenario, the higher the specialization level; the wider the applicable scenario, the lower the specialization level. The core function of this attribute is to match with the business process tags of the field. High-specialization rules will prioritize matching fields corresponding to the business process, improving the matching accuracy of vertical business scenarios.

[0045] Complexity refers to the level of technical difficulty and computational resource consumption in implementing a rule, used to balance the rule's effectiveness and feasibility. The classification logic is as follows: based on the technical means and computational resources required for rule implementation, rules with low implementation difficulty and low resource consumption have low complexity; rules requiring complex algorithms have high complexity. The core function of this attribute is to adjust recommendation preferences according to business objectives. In resource-constrained scenarios, low-complexity rules will receive higher scores, ensuring efficient rule implementation and avoiding implementation difficulties due to excessive complexity.

[0046] S140. Calculate the matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule using a multi-dimensional scoring model.

[0047] By comprehensively evaluating various dimensions, the degree of compatibility between the two is quantified and a matching score is output.

[0048] In this embodiment of the invention, the evaluation dimensions of the multi-dimensional scoring model include semantic similarity, generality, priority, specialization, complexity, and alignment with business objectives. Semantic similarity is obtained by calculating the cosine similarity between the semantic vector of the field text and the semantic feature vector of the rule, with a value range of [0,1]. Generality is obtained by normalizing the predefined rule action type and field data type matching matrix, with a value range of [0,1]. Priority is obtained by normalizing the rule's preset priority level and field statistical features after dynamic adjustment, with a value range of [0,1]. Specialization is obtained by reverse normalization, converting the specialization to a value range of [0.2,1.0]. Complexity is obtained by converting it to a cost coefficient and normalizing it to a value range of [0.2,1.0]. Business goal alignment is obtained by constructing a business process and rule mapping matrix and calculating it using TF-IDF text similarity, with a value range of [0,1]. The matching score is calculated using a weighted linear combination model. The formula is: Sere=Σ(Wi×Di), where Wi is the weight of each evaluation dimension, the sum of the weights of all dimensions is 1, and Di is the normalized score of each evaluation dimension.

[0049] Semantic similarity matching is used to match the business meaning of fields and rules, based on the textual semantic features of the fields (high-dimensional semantic vectors converted by the BERT model) and the semantic features of the rules (vectors converted after extracting the core semantics from the rule names). The cosine similarity algorithm is used to calculate the similarity between the two vectors; the closer the value is to 1, the better the business meaning of the field and rule matches. The value range is [0,1], where 0 indicates a complete mismatch and 1 indicates a perfect match.

[0050] The generality metric is used to determine the type compatibility between a field and a rule, based on a predefined matching matrix of rule application type and field data type. First, the matching matrix is ​​used to determine if the two types are compatible; compatibility earns a base score of 1, incompatibility earns 0, and then normalization is applied. The value range is [0,1], where 0 indicates complete incompatibility and 1 indicates complete compatibility.

[0051] Priority represents the dynamic adjustment of the importance weight of rules, combining two factors: the preset priority level of the rule and the statistical characteristics of the field. First, the preset priority level of the rule is dynamically adjusted based on the statistical characteristics of the field. For example, a "NOTING 5" rule preset to level 5 might have its level adjusted to 5.5 due to a high field missing rate. Then, the adjusted level is normalized to the [0,1] range, mapping levels 1-5 and the adjusted level to scores of 0-1. The value range is [0,1], where 0 indicates extremely low rule importance and 1 indicates extremely high importance.

[0052] Specialization level indicates the rule's adaptability to specific vertical scenarios. Based on the pre-defined specialization level of the rule, it is defined as level 1-5, where level 5 indicates the most specific application scenario, such as a rule that only applies to financial audit scenarios; level 1 indicates the broadest application scenario, such as a general non-empty check rule. Reverse normalization is used to convert the specialization levels 1-5 according to the mapping relationship of "level 1 → 0.2, level 2 → 0.4, level 3 → 0.6, level 4 → 0.8, level 5 → 1.0" to ensure that rules with high specialization level score higher in field matching for the corresponding business scenario. The value range is [0.2, 1.0], where 0.2 indicates an extremely broad application scenario and 1.0 indicates an extremely specific application scenario.

[0053] Complexity represents the balance between the cost and feasibility of implementing a rule. Based on the pre-defined complexity level of the rule, it is defined as level 1-5, where level 1 represents the lowest implementation difficulty and resource consumption, such as non-null checks; level 5 represents the highest difficulty and resource consumption, such as time series trend analysis. First, the complexity level is converted into a cost coefficient using the formula "6 - complexity level", e.g., level 1 → 5, level 5 → 1. Then, the cost coefficient is normalized to the range [0.2, 1.0] to ensure that low-complexity rules score higher. The value range is [0.2, 1.0], where 0.2 indicates extremely high implementation cost and 1.0 indicates extremely low implementation cost.

[0054] Business goal alignment represents the scenario goal of matching fields and rules. It combines two factors: first, the constructed business process-rule mapping matrix; and second, TF-IDF text similarity, which calculates the text similarity between the field's business process label and the rule's applicable scenario description. First, rules matching the field's business process are filtered out using the mapping matrix. Then, the score is optimized using TF-IDF text similarity, and finally normalized to the [0,1] range. The value range is [0,1], where 0 indicates that the rule and the field's business goal are completely inconsistent, and 1 indicates a perfect match.

[0055] The final matching score between fields and rules is achieved by integrating the scores from six dimensions using a weighted linear combination model. The formula strictly follows the document definition. Calculation formula: Sere = Σ(Wi × Di); Wherein, Wi: the weight of each evaluation dimension, the sum of the weights of all dimensions is 1, and the weights will be dynamically adjusted according to the business goals input by the user. For example, in core business scenarios, the weight of semantic similarity will be increased to 30%, and in general scenarios, the weight of generality will be increased to 20%. Di: Normalized scores for each evaluation dimension, i.e., scores calculated from the above six dimensions that fall within the corresponding value range.

[0056] S150: The rule recommendation engine dynamically adjusts the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives, and outputs the Top-N recommendation rules based on the adjusted weights and matching scores.

[0057] By dynamically adjusting weights, the recommendation results are aligned with the user's business goals, ultimately outputting the optimal rule.

[0058] In this embodiment of the invention, the rule recommendation engine uses the hierarchical analysis method to achieve dynamic weight allocation for each evaluation dimension; In core business scenarios, the weight of semantic similarity is adjusted to 30%, and the weight of rule priority is adjusted to 25%; in general governance scenarios, the weight of type compatibility is adjusted to 20%, and the weight of implementation efficiency is adjusted to 10%. The output of the rule recommendation engine includes a list of the top-N recommended rules sorted in descending order of matching score, and a radar chart of the dimensional scores of each recommended rule; The rule recommendation engine supports dynamic model optimization, including training the weight coefficients of each evaluation dimension using historical matching data and updating the parameters of the multi-dimensional scoring model monthly. It also supports manual calibration of multi-dimensional rule attributes, including adjusting the priority, specialization, and complexity level of rules to adapt to changes in business needs.

[0059] Using the analytic hierarchy process (AHP), the weights of the six evaluation dimensions are adjusted according to different scenarios. Core business scenarios emphasize business adaptability, with semantic similarity weighted at 30% and rule priority weighted at 25%; general governance scenarios emphasize compatibility and efficiency, with type compatibility weighted at 20% and implementation efficiency weighted at 10%.

[0060] The output is a list of the top-N recommendation rules sorted from highest to lowest matching score. Based on the radar chart of each rule's score across six evaluation dimensions, the advantages and disadvantages of each rule are visually displayed.

[0061] The scoring model parameters are updated monthly by training dimensional weight coefficients using historical matching data. On the other hand, it supports manual calibration of rule attributes, allowing adjustment of rule priority, specialization, and complexity levels to adapt to changes in business needs.

[0062] In this embodiment of the invention, the method may further include the following steps: Edit the attributes of rules in the rule base, including modifying the semantic features, action type, priority, specialization degree, and complexity of the rules; Add data quality rules to the rule base and automatically complete multi-dimensional attribute modeling for the new rules; Semantic adaptation of multilingual field descriptions and multilingual rule descriptions is achieved by leveraging the multilingual capabilities of the BERT pre-trained model to realize semantic matching of fields and rules in different languages.

[0063] For existing data quality rules in the rule base, manual modification of core attributes is supported, ensuring that rule attributes can be dynamically adjusted according to changes in business needs. The specific editing scope strictly corresponds to the rule's multi-dimensional attribute system.

[0064] Semantic feature editing: The core business semantics extracted from the rule name can be modified. For example, the semantic feature of date format validation (YYYY-MM-DD) can be changed from date format validation to financial scenario date format validation to ensure that the semantic description is more in line with the business scenario. The data type identifier applicable to the rule can be adjusted. For example, the original range validation rule that is adapted to numeric type can be expanded to be adapted to both numeric type and date type.

[0065] Priority editing allows you to reset the importance level of rules in data quality governance. For example, after a business upgrade, the priority of the cross-border order amount verification rule can be raised from level 3 to level 5 to ensure that core business data is given priority in governance. Specialization editing allows you to adjust the level of specialization of the rules applicable to specific scenarios. For example, you can increase the specialization of the original general invoice amount verification rule from level 2 to level 4, limiting it to cross-border e-commerce invoice amount verification, and adapting it to vertical scenarios. The complexity editor can adjust the technical difficulty and resource consumption level of the rules according to the implementation of technology upgrades. For example, due to algorithm optimization, the complexity of the time series anomaly detection rule has been reduced from level 4 to level 2, thus reducing implementation costs.

[0066] When adding new data quality rules to the rule base, the system automatically models the multi-dimensional attributes of the rules without requiring manual configuration. The modeling logic remains consistent with the existing rule attribute system. When adding a new rule, users only need to enter the rule name and applicable data type. The system automatically extracts the semantic features of the rule, automatically matches and generates the action type, and assigns the initial priority, specialization, and complexity based on the rule's functionality by default. After automatic modeling is completed, the new rule is directly included in the rule base and can participate in subsequent field-rule matching. It also supports manual adjustment of the initial attributes based on business needs.

[0067] Leveraging the multilingual capabilities of the BERT pre-trained model, semantic matching of field descriptions and rule descriptions in different languages ​​is achieved, breaking down language barriers. When the field description is not in Chinese and the rule description is in Chinese, or vice versa, the system invokes the multilingual BERT model. The model converts the field and rule descriptions in different languages ​​into high-dimensional semantic vectors of a unified dimension. Subsequently, by calculating the cosine similarity of the semantic vectors in different languages, the system determines the degree of consistency between the business meaning of the field and the rule, ensuring accurate semantic matching even in multilingual scenarios and adapting to the global data governance needs of enterprises.

[0068] like Figure 2 As shown, this embodiment of the invention provides a field rule matching and recommendation device based on multi-dimensional feature fusion, comprising: Acquisition unit 210 is used to acquire user input information, which includes the target data table and the current business objective; Extraction unit 220 is used to extract features from fields in the target data table through the field feature extraction module to obtain multi-dimensional features of the fields; Modeling unit 230 is used to perform attribute modeling on data quality rules in a preset rule base through rule attribute modeling to obtain multi-dimensional rule attributes; The calculation unit 240 is used to calculate the matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule through a multi-dimensional scoring model; Output unit 250 is used to dynamically adjust the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives through the rule recommendation engine, and output the Top-N recommendation rules based on the adjusted weights and matching scores.

[0069] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the field rule matching and recommendation device based on multi-dimensional feature fusion. In other embodiments of the present invention, the field rule matching and recommendation device based on multi-dimensional feature fusion may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0070] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0071] Figure 3 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0072] like Figure 3As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0073] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0074] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as field rule matching recommendation methods based on multi-dimensional feature fusion.

[0075] In some embodiments, the field rule matching recommendation method based on multi-dimensional feature fusion can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the field rule matching recommendation method based on multi-dimensional feature fusion described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the field rule matching recommendation method based on multi-dimensional feature fusion by any other suitable means (e.g., by means of firmware).

[0076] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0080] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0081] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0082] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0083] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A field rule matching recommendation method based on multi-dimensional feature fusion, characterized in that, include: Obtain user input information, which includes the target data table and the current business objective; The field feature extraction module extracts features from the fields in the target data table to obtain multi-dimensional features of the fields; By performing attribute modeling on the data quality rules in the preset rule base, multi-dimensional attributes of the rules are obtained. The matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule is calculated using a multi-dimensional scoring model. The rule-based recommendation engine dynamically adjusts the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives, and outputs the Top-N recommendation rules based on the adjusted weights and matching scores.

2. The method according to claim 1, characterized in that, The multi-dimensional features of the field include textual semantic features, data type features, statistical features, and business process tags. The textual semantic features are obtained by converting the field description into a high-dimensional semantic vector using a BERT pre-trained model. The data type features are the data type identifiers of the field, including numeric, date, and string types. The statistical features include the missing rate, outlier ratio, mean, standard deviation, and number of unique values ​​for the field's sample values. The business process tags are the identifiers of the business processes to which the field belongs, including sales processes, supply chain management processes, and financial audit processes.

3. The method according to claim 2, characterized in that, The application logic of the statistical features includes: when the missing rate of a field is greater than a preset threshold, the field is automatically marked as needing to be preferentially matched with non-empty check rules after the field features are extracted; when the proportion of outliers in a field is greater than a preset threshold, the field is automatically marked as needing to be matched with outlier detection rules.

4. The method according to claim 1, characterized in that, The rule's multi-dimensional attributes include semantic features, action type, priority, specialization, and complexity. The semantic features are derived from the core business semantics extracted from the rule name. The action type is the data type identifier to which the rule applies, corresponding to the data type features of the field. The priority is the importance level of the rule in data quality governance. The specialization level is the specificity level of the rule's applicable scenario. The complexity level is the technical difficulty and computational resource consumption level of the rule's implementation.

5. The method according to claim 1, characterized in that, The evaluation dimensions of the multi-dimensional scoring model include semantic similarity, generality, priority, specialization, complexity, and alignment with business objectives. The semantic similarity is obtained by calculating the cosine similarity between the field text semantic vector and the rule semantic feature vector, with a value range of [0,1]. The generality is obtained by normalizing the predefined rule action type and field data type matching matrix, with a value range of [0,1]. The priority is obtained by normalizing the rule's preset priority level and field statistical features after dynamic adjustment, with a value range of [0,1]. The specialization degree is obtained by reverse normalization, converting the specialization degree to a value range of [0.2,1.0]. The complexity is obtained by converting it into a cost coefficient and normalizing it to a value range of [0.2,1.0]. The business goal alignment degree is obtained by constructing a business process and rule mapping matrix and calculating it using TF-IDF text similarity, with a value range of [0,1]. The matching score is calculated using a weighted linear combination model. The calculation formula is: Sere=Σ(Wi×Di), where Wi is the weight of each evaluation dimension, the sum of the weights of all dimensions is 1, and Di is the normalized score of each evaluation dimension.

6. The method according to claim 1, characterized in that, The rule recommendation engine uses the hierarchical analysis method to achieve dynamic weight allocation for each evaluation dimension; In core business scenarios, the weight of semantic similarity is adjusted to 30%, and the weight of rule priority is adjusted to 25%; in general governance scenarios, the weight of type compatibility is adjusted to 20%, and the weight of implementation efficiency is adjusted to 10%. The output of the rule recommendation engine includes a list of Top-N recommendation rules sorted in descending order of matching score, and a radar chart of the dimensional scores of each recommendation rule; The rule recommendation engine supports dynamic model optimization, including training the weight coefficients of each evaluation dimension using historical matching data and updating the parameters of the multi-dimensional scoring model monthly. It also supports manual calibration of rule multi-dimensional attributes, including adjusting the priority, specialization, and complexity level of rules to adapt to changes in business needs.

7. The method according to claim 1, characterized in that, The method further includes: Edit the attributes of rules in the rule base, including modifying the semantic features, action type, priority, specialization degree, and complexity of the rules; Add data quality rules to the rule base and automatically complete multi-dimensional attribute modeling for the new rules; Semantic adaptation of multilingual field descriptions and multilingual rule descriptions is achieved by leveraging the multilingual capabilities of the BERT pre-trained model to realize semantic matching of fields and rules in different languages.

8. A field rule matching and recommendation device based on multi-dimensional feature fusion, characterized in that, include: The acquisition unit is used to acquire user input information, which includes a target data table and the current business objective. The extraction unit is used to extract features from the fields in the target data table through the field feature extraction module to obtain multi-dimensional features of the fields; The modeling unit is used to perform attribute modeling on the data quality rules in the preset rule base through rule attribute modeling, so as to obtain multi-dimensional attributes of the rules. The calculation unit is used to calculate the matching score between the multi-dimensional features of the field and the multi-dimensional attributes of the rule through a multi-dimensional scoring model; The output unit is used to dynamically adjust the weights of each dimension in the multi-dimensional scoring model according to the priority of business objectives through the rule recommendation engine, and output the Top-N recommendation rules based on the adjusted weights and matching scores.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the field rule matching recommendation method based on multi-dimensional feature fusion as described in any one of claims 1-7.

10. A computer-readable medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the field rule matching recommendation method based on multi-dimensional feature fusion as described in any one of claims 1-7.

Images