Automatic data quality rule matching method, system and device for public security service data

By building a quality rule knowledge graph and a hierarchical matching engine, combined with the semantic understanding and online optimization mechanism of the BERT model, the problems of insufficient rule coverage, lack of semantic understanding, and poor dynamic adaptability in the matching of public security business data quality rules are solved, and efficient and flexible data quality rule automatic matching is achieved.

CN120745786APending Publication Date: 2025-10-03SHANDONG FUTURE NETWORK RES INST (PURPLE MOUNTAIN LAB IND INTERNET INNOVATION APPL BASE)
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202510804755.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-17
Publication Date
2025-10-03

AI Technical Summary

Technical Problem

Existing technologies have problems with insufficient rule coverage, lack of semantic understanding, and poor dynamic adaptability in matching public security business data quality rules. It is difficult to effectively identify non-standard named fields and understand the business meaning of fields, and manual intervention is required when responding to industry standard updates.

Method used

By constructing a quality rule knowledge graph, using the BERT model to extract semantic vectors for fields, and combining it with a hierarchical matching engine and online optimization mechanisms, we achieve automatic matching of public security business data. This approach involves structured parsing of standard public security data meta-documents, constructing meta-nodes, rule-nodes, and synonym-name nodes, establishing dependency and reference relationships, and employing a three-level matching process with precise, semantic, and statistical matching layers. Furthermore, the knowledge graph is dynamically updated through a feedback learning mechanism.

Benefits of technology

It improves the rule coverage, enhances the semantic understanding ability, supports dynamic rule updates and conflict detection, improves matching efficiency and system adaptability, and ensures the flexible adaptation and continuous optimization of the system in complex business scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120745786A_ABST
    Figure CN120745786A_ABST
Patent Text Reader

Abstract

The invention provides a data quality rule automatic matching method, system and device for public security service data, and belongs to the technical field of computer information processing. The method comprises the following steps: structurally analyzing a public security data element standard document; constructing a basic rule and an association rule according to an analysis result; constructing a quality rule knowledge graph, creating four types of entity nodes including data element nodes, rule nodes, standard nodes and synonymous name nodes, and establishing three types of relation edges including a dependency relation, a synonymous name relation and a reference relation; extracting multi-modal features of the to-be-retrieved field, wherein the multi-modal features comprise semantic features and structured features; a rule is generated through a hierarchical matching engine, and matching processes of an accurate matching layer, a semantic matching layer and a statistical matching layer are executed in sequence; and executing online optimization. According to the method, the bottleneck problems of insufficient rule coverage, semantic understanding deficiency, poor dynamic adaptability and the like of a traditional method are solved, and the method has remarkable technical progress and practical value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method, system and device for automatically matching data quality rules for public security business data, and belongs to the technical field of computer information processing. Background Art

[0002] As data governance requirements in the public security industry become increasingly complex, the types and number of data quality rules across police departments and business systems (such as population databases, entry-exit systems, and traffic management platforms) are growing exponentially. Traditional methods face significant challenges in addressing the unique data characteristics of public security operations. To address the challenges of automatically matching data quality rules, existing technologies typically adopt the following measures: first, standardization of rule templates, which simplifies the configuration process through predefined rule libraries and fixed matching logic (such as regular expressions and exact field name matching); second, statistical feature analysis, which automatically generates threshold rules based on data distribution characteristics (such as null value rate and value range); and third, metadata-driven, which matches rule templates based on field descriptions and business attributes. These methods still have significant shortcomings when dealing with non-standard named fields, cross-system rule conflicts, and iterative industry standards.

[0003] Existing data quality rule-matching technology suffers from the following major flaws: First, insufficient rule coverage: Relying on manually predefined rule templates and fixed matching logic, it cannot effectively identify non-standard named fields (e.g., the synonymous relationship between "document number" and "ID number"). Second, a lack of semantic understanding: Existing statistical analysis and metadata-driven methods fail to deeply understand the business meaning of fields, resulting in the omission of key business rules. Third, poor dynamic adaptability: When industry standards are updated, manual reconfiguration of the rule base is required, resulting in a long response time. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system and device for automatically matching data quality rules for public security business data, which solves the bottleneck problems of traditional methods in insufficient rule coverage, lack of semantic understanding, poor dynamic adaptability, etc., and has significant technological progress and practical value.

[0005] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions: In a first aspect, a method for automatically matching data quality rules for public security business data is provided, comprising: Structural analysis of public security data element standard documents, extracting field identifiers, standard names, data types and formats, constraints, associated standard references, and synonymous names; constructing basic rules and association rules based on the analysis results; Construct a quality rule knowledge graph, create four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establish three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships. Synonymous name relationships include relationship source, confidence, and validity time attributes. Extract the multimodal features of the fields to be inspected, generate semantic vectors for field descriptions using the BERT model, extract business meaning labels as semantic features, and simultaneously obtain structural features, including field name, data type, length, and value distribution. Generate rules through the hierarchical matching engine and execute the three-level matching process of exact matching layer, semantic matching layer and statistical matching layer in sequence; Perform online optimization, dynamically update knowledge graph nodes and implement feedback learning mechanism.

[0006] Preferably, the basic rules include format rules and value range rules, the format rules are used to verify whether the format of the data element complies with the standard, and the value range rules are used to verify whether the value in the data element complies with the allowable value range in the standard; The association rules include cross-field consistency rules and business logic rules. The cross-field consistency rules are used to establish logical association constraints between different fields; the business logic rules derive conditional constraints between fields according to business requirements.

[0007] Preferably, the data element node contains all attributes of the standard, the rule node stores rule logic and execution parameters, the standard node associates and references standard documents, and the synonymous name node stores various synonymous expressions of the data element standard name.

[0008] Preferably, the dependency relationship is: the rule node points to the dependent data element node; The synonymous name relationship is: by parsing the synonymous name column in the standard document and combining the machine learning method to generate synonymous names of the data element standard name, a synonymous name relationship with a confidence attribute is established between the data element node and the synonymous name node; The reference relationship is: the association between the marking rule and the standard document, and a directed relationship from the rule node to the standard node.

[0009] Preferably, the specific value of the confidence is as follows: The confidence level of synonymous names explicitly defined in standard documents is set to 1.0. The initial confidence level of synonymous names generated by machine learning is set to 0.6-0.9. The confidence decay model is applied to synonymous name relationships from non-standard sources: in: is the initial confidence, is the time that has passed since the relationship was created, is the decay rate.

[0010] Preferably, the hierarchical matching engine comprises: Exact matching layer: matches the input field to be tested with the standard name or synonymous name of the data element in the knowledge graph; Semantic matching layer: If the exact matching layer fails to match successfully, the semantic similarity between the field to be tested and the standard data element is calculated, and the comprehensive similarity is obtained by weighted aggregation combined with the synonym name relationship. If the comprehensive similarity is greater than the set threshold, the match is successful; Statistical matching layer: For unmatched fields to be inspected, similarity rules are recommended based on value distribution characteristics.

[0011] Preferably, the semantic matching layer specifically includes: The language model is used to convert the to-be-checked field, standard name, and synonymous name into a to-be-checked field vector, a standard data element vector, and a synonymous name vector, respectively; Calculate the cosine similarity between the field vector to be tested and the standard data element vector; If there are synonyms in the standard data element vector, a weighted aggregation strategy is used to calculate the comprehensive similarity. The first K synonyms are selected in descending order of confidence. The weighted aggregation formula is as follows: in, is the aggregate similarity, is the confidence of the i-th synonym name, is the cosine similarity calculation, is the field vector to be checked, is a synonym name vector; The final similarity formula is: in, is the comprehensive similarity, is the basic similarity, is the basic similarity weight, like , the match is successful.

[0012] Preferably, the online optimization includes: Dynamic rule updates: monitor standard changes and automatically update the knowledge graph; Feedback learning mechanism: The confidence level of synonymous names is adjusted based on user confirmation results. The adjustment formula is as follows: in: is the adjusted confidence level, R is the user feedback score, For the forgetting factor; Performance optimization: establish rule cache index and dynamically adjust matching priority; Incremental expansion: supports user-defined rules to be injected into the knowledge graph.

[0013] Secondly, a data quality rule automatic matching system for public security business data is provided, including: Rule generation module: Structural analysis of public security data element standard documents, extracting field identifiers, standard names, data types and formats, constraints, associated standard references and synonyms; constructing basic rules and association rules based on the analysis results; Knowledge graph construction module: Builds a quality rule knowledge graph, creates four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establishes three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships. Synonymous name relationships include relationship source, confidence level, and validity period attributes. Feature extraction module: This module extracts the multimodal features of the fields to be inspected, generates semantic vectors for field descriptions using the BERT model, extracts business meaning labels as semantic features, and simultaneously obtains structural features, including field name, data type, length, and value distribution. Hierarchical matching engine module: Generates rules through the hierarchical matching engine and sequentially executes the three-level matching process of the exact matching layer, semantic matching layer, and statistical matching layer; Online optimization module: performs online optimization, dynamically updates knowledge graph nodes and implements feedback learning mechanism.

[0014] On the third aspect, a device for automatically matching data quality rules for public security business data is provided, comprising a processor and a memory storing program instructions, wherein the processor is configured to execute the method for automatically matching data quality rules for public security business data when running the program instructions.

[0015] The advantages of the present invention are: Improve rule coverage By generating rules based on public security data element standards, non-standard named fields can be identified, effectively solving the problem of insufficient rule coverage and improving the adaptability of rules to different data situations.

[0016] Semantic understanding ability is significantly enhanced This application proposal uses the BERT model to extract semantic vectors for field descriptions, combined with the synonymous name relationship network in the knowledge graph, to achieve a deep understanding of the business meaning of fields. In the semantic matching layer, the system determines the semantic consistency between fields and standard data elements based on cosine similarity, enabling precise matching across naming methods. This mechanism effectively addresses the semantic understanding deficiencies of traditional methods and improves the accuracy and coverage of rule matching.

[0017] Dynamic knowledge graph supports rule reasoning and conflict detection Existing technologies lack explicit semantic relationship modeling between rules, making rule conflicts difficult to detect and resolve. For example, a field might match both the "not null check" and "allow null values" rules, causing logical confusion in execution.

[0018] This proposal constructs a quality rule knowledge graph consisting of three types of entities: data element nodes, rule nodes, and standard nodes, as well as three types of relationships: dependency, conflict, and synonymous names. This not only supports semantic reasoning between rules but also efficiently detects and resolves rule conflicts through a graph traversal algorithm. Furthermore, the knowledge graph features version management and dynamic updates, enabling real-time response to changes in industry standards, significantly enhancing the system's adaptability and scalability.

[0019] Hierarchical matching engine improves matching efficiency and flexibility Existing technologies generally adopt a single matching strategy (such as exact matching of field names or statistical distribution matching), which makes it difficult to strike a balance between efficiency and accuracy, especially when facing complex business scenarios.

[0020] This application proposal designs a hierarchical matching engine architecture. This hierarchical mechanism not only ensures high matching efficiency but also takes into account flexible adaptability in complex scenarios, which is significantly better than the traditional single-level matching strategy.

[0021] Online optimization mechanism ensures long-term stable operation of the system Traditional methods lack effective feedback mechanisms and continuous optimization capabilities. Once the rule base is configured, it becomes static and difficult to adapt to changing business needs and industry standards.

[0022] This proposal introduces multiple online optimization mechanisms, including dynamic rule updates, feedback learning, performance tuning, and incremental expansion. These mechanisms together form a self-evolving intelligent rule management system, significantly improving the system's sustainable operation and scalability.

[0023] In summary, this application proposal has achieved breakthrough improvements over existing technologies in semantic understanding, multimodal feature fusion, knowledge graph modeling, hierarchical matching mechanism and online optimization, solving the bottleneck problems of traditional methods in insufficient rule coverage, lack of semantic understanding, and poor dynamic adaptability, and has significant technological progress and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0025] Figure 1 Schematic diagram of the process of the present invention.

[0026] Figure 2 Schematic diagram of the data structure of the method of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] Example 1 like Figure 1-Figure 2 As shown, a method for automatically matching data quality rules for public security business data includes the following steps: S1: Structural analysis of public security data element standard documents, extracting field identifiers, standard names, data types and formats, constraints, associated standard references, and synonyms; constructing basic rules and association rules based on the analysis results; S2: Build a quality rule knowledge graph, create four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establish three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships. The synonymous name relationship includes the relationship source, confidence level, and validity period attributes. S3: Extract the multimodal features of the fields to be inspected. Generate semantic vectors for field descriptions using the BERT model and extract business meaning labels as semantic features. Also, obtain structural features, including field name, data type, length, and value distribution. S4: Generate rules through the hierarchical matching engine and execute the three-level matching process of exact matching layer, semantic matching layer and statistical matching layer in sequence; S5: Perform online optimization, dynamically update knowledge graph nodes and implement feedback learning mechanism.

[0029] As a refinement of the above embodiment, step S1 specifically includes: S101: Structural analysis of metadata elements in GA / T 543 documents We perform structured parsing of the GA / T 543 document to extract key metadata elements, including field identifiers (e.g., DE00001), standard names (e.g., "citizen identification number"), data types and formats (e.g., C18 for 18-bit characters), constraints (e.g., GB11643 checksum requirements), references to related standards (e.g., GB / T 2260 administrative division codes), and synonyms (e.g., synonyms for citizen identification number are identity card number and resident identification card number). These elements provide the foundation for the subsequent automatic construction of rules and knowledge graphs.

[0030] S102: Basic rule construction Generate formatting rules based on the parsed data type definitions. For example, for DE00001 (citizen ID number), you can generate the regular expression "^\d{17}[\dXx]$" to verify that the ID number format complies with the standard. Additionally, generate range rules based on the allowed value ranges in the parsed standard. For example, for DE00007 (gender code), you can generate the range set {"1","2","9"}, representing male, female, and unknown gender, respectively.

[0031] S103: Association rule derivation Cross-field consistency rules: Establish relationships between different fields to ensure data consistency. For example, digits 7-14 of DE00008 (date of birth) and DE00001 (ID number) must be consistent because digits 7-14 of the ID number represent the date of birth.

[0032] Business logic rules: Rules are derived based on business requirements. For example, when DE00024 (Military Service Status) = "1", DE00012 (Age) must be ≥ 18, which conforms to actual business logic.

[0033] As a refinement of the above embodiment, step S2 specifically includes: S201: Node creation Data element node: contains all the attributes of the standard, such as field identifier, standard name, data type and format, constraints, etc., providing basic data element information for the knowledge graph.

[0034] Rule node: stores rule logic and execution parameters, and clarifies the specific content and execution method of the rules.

[0035] Standard node: Associates and references standard documents to facilitate querying and tracing the source of the rules.

[0036] Synonymous name node: stores various synonymous expressions of the standard name of the data element. The source types are standard document definition, machine learning generation, and manual maintenance.

[0037] S202: Relationship Building "Dependency" relationship: The rule node points to the dependent data element, indicating that the rule is based on a specific data element.

[0038] "Synonymous Name" Relationships: By parsing the synonym name columns in standard documents and combining them with machine learning methods to generate synonyms for the standard names of data elements, synonymous name relationships with confidence attributes are established between data element nodes and synonymous name nodes. Relationship attributes include the relationship source (defined in standard documents or generated by machine learning), confidence (synonymous names explicitly defined in standard documents have a confidence of 1.0, while synonymous names generated by machine learning have an initial confidence range of 0.6 to 0.9), and validity period. For synonymous name relationships from non-standard sources, the following confidence decay model is introduced: in: is the initial confidence (e.g. 0.8), t is the time elapsed after the relationship is created (unit: day), is the decay rate (can be set to 0.01). This way, over time, unverified relationships discovered by machine learning will gradually become invalid, preventing the accumulation of misjudgments.

[0039] "Reference" relationship: This relationship marks the association between a rule and a standard document. It is a directed relationship from a rule node to a standard node. It indicates that a rule is based on a specific standard document, providing important support for the legality, accuracy, and traceability of the rule.

[0040] As a refinement of the above embodiment, the semantic features in step S3 are as follows: The BERT model generates vector representations of field descriptions and extracts business-meaning labels (such as "time" and "region"). The BERT model understands the semantic information of text, allowing it to more accurately grasp the business meaning of fields.

[0041] Structural features: These include field name, data type, length, and value distribution (null value ratio, unique value ratio, etc.). These features can be directly obtained from the data and provide basic information for rule matching.

[0042] As a refinement of the above embodiment, step S4 specifically includes: S401: Exact Matching Layer First, we try to match the input field with the standard name of the data element in the knowledge graph. If the input field is exactly the same as the standard name of a data element, we consider that a matching data element node has been found.

[0043] If a match with the standard name fails, the synonym name relationship in the knowledge graph is used for matching. This is done by querying the synonym name nodes associated with the input field and checking whether these synonym name nodes have a "synonymous name" relationship with the data meta node in the knowledge graph. If such a relationship exists and the confidence level of the relationship meets a certain threshold, then a matching data meta node is considered to have been found.

[0044] S402: Semantic Matching Layer If an exact match fails, semantic analysis is initiated. The semantic matching layer is responsible for calculating the semantic similarity between the input field and the standard data element, and optimizing the matching results by combining synonymous name relationships to solve the matching problem of non-standard named fields. The semantic matching layer mainly includes the following key steps: S4021: Semantic Vectorization Convert text to semantic vectors using a pre-trained language model (BERT): Field vectors to be inspected =BERT(input field name), standard data element vector =BERT (standard name), synonym name vector =BERT( ).

[0045] S4022: Basic similarity calculation Calculate the field vector to be checked Compared with the standard data element vector in the knowledge graph The cosine similarity S is calculated as follows: S4023: Synonymous Name Enhancement If the standard data element has synonymous names, a weighted aggregation strategy is used to calculate the comprehensive similarity. The first K synonymous names (default K=3) are selected in descending order of confidence. The similarity aggregation formula is: in, is the aggregate similarity, is the confidence of the i-th synonym name, is the cosine similarity calculation, is the field vector to be checked, is a vector of synonym names.

[0046] S4024: The final similarity formula is: Among them, among them, is the comprehensive similarity, is the basic similarity, is the base similarity weight (default 0.7).

[0047] like , the match is successful.

[0048] S403: Statistics matching layer: For unmatched fields, similarity rules are recommended based on value distribution characteristics. For numeric fields, assume the field value is x and generate range rules based on the value distribution characteristics. For example, if 90% of the field values ​​fall within the interval [a, b], the rule is: a ≤ x ≤ b.

[0049] S404: Rule determination and traceability information Once a matching data element node is found, the "dependency" relationship between the rule node and the data element node in the knowledge graph can be used to find the rule node that depends on the data element. By querying the standard nodes associated with the matching rule node, the standard document information underlying the rule can be obtained and corresponding explanatory information generated. This explanatory information may include the matching data element node, the rule's dependencies, the rule's conditions, and why the rule applies to the input field. Rules identified through synonymous name matching and whose relationships are derived from machine learning require user confirmation.

[0050] As a refinement of the above embodiment, the online optimization in step S5 includes: (1) Dynamic rule update: Monitor standard changes (such as GA / T 543 version upgrades), automatically parse new versions, and update knowledge graph nodes. This enables the system to adapt to changes in industry standards in a timely manner without manual intervention.

[0051] (2) Feedback learning mechanism: Each time a user confirms that a set of fields are synonymous, the confidence level of the relationship increases. Use the incremental update formula: in: is the adjusted confidence level, R is the user feedback score (such as 1 means completely correct), is the forgetting factor (value is 0.6).

[0052] This allows the system to continuously optimize the synonym name network and improve the accuracy of semantic matching.

[0053] (3) Performance optimization: Create cache indexes for high-frequency query rules, calculate the execution time of rules in real time, and dynamically adjust the matching priority. Improve the system's response speed and performance, and ensure stable operation even in high-concurrency situations. (4) Incremental expansion: The system supports the injection of user-defined rules into the knowledge graph, automatically associating existing nodes through relational reasoning (e.g., establishing a dependency between the newly added "Health Code Verification Rule" and the "ID Number" node). This makes the system highly scalable and able to meet the personalized needs of different users.

[0054] Example 2 A data quality rule automatic matching system for public security business data, comprising: Rule generation module: Structural analysis of public security data element standard documents, extracting field identifiers, standard names, data types and formats, constraints, associated standard references and synonyms; constructing basic rules and association rules based on the analysis results; Knowledge graph construction module: Builds a quality rule knowledge graph, creates four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establishes three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships. Synonymous name relationships include relationship source, confidence level, and validity period attributes. Feature extraction module: This module extracts the multimodal features of the fields to be inspected, generates semantic vectors for field descriptions using the BERT model, extracts business meaning labels as semantic features, and simultaneously obtains structural features, including field name, data type, length, and value distribution. Hierarchical matching engine module: Generates rules through the hierarchical matching engine and sequentially executes the three-level matching process of the exact matching layer, semantic matching layer, and statistical matching layer; Online optimization module: performs online optimization, dynamically updates knowledge graph nodes and implements feedback learning mechanism.

[0055] The disclosed embodiments also provide a device for automatically matching data quality rules for public security business data, comprising a processor and memory. Optionally, the device may also include a communication interface and a bus. The processor, communication interface, and memory may communicate with each other via the bus. The communication interface may be used for information transmission. The processor may invoke logic instructions in the memory to execute the method for automatically matching data quality rules for public security business data described in the aforementioned embodiment.

[0056] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0057] Memory, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor executes the program instructions / modules stored in the memory to perform functional applications and data processing, thereby implementing the automatic data quality rule matching method for public security business data in the above-mentioned embodiments.

[0058] The memory may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory may include high-speed random access memory and non-volatile memory.

[0059] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A data quality rule automatic matching method for public security business data, characterized in that: include: Structural analysis of public security data element standard documents to extract field identifiers, standard names, data types and formats, constraints, associated standard references and synonyms; Construct basic rules and association rules based on the analysis results; Construct a quality rule knowledge graph, create four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establish three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships; The synonym name relationship includes the relationship source, confidence and validity time attributes; Extract the multimodal features of the fields to be inspected, generate semantic vectors for field descriptions using the BERT model, extract business meaning labels as semantic features, and simultaneously obtain structural features, including field name, data type, length, and value distribution. Generate rules through the hierarchical matching engine and execute the three-level matching process of exact matching layer, semantic matching layer and statistical matching layer in sequence; Perform online optimization, dynamically update knowledge graph nodes and implement feedback learning mechanism.

2. The method for automatically matching data quality rules for public security business data according to claim 1 is characterized in that: The basic rules include format rules and value range rules. The format rules are used to verify whether the format of the data element complies with the standard, and the value range rules are used to verify whether the value in the data element complies with the allowable value range in the standard. The association rules include cross-field consistency rules and business logic rules. The cross-field consistency rules are used to establish logical association constraints between different fields; the business logic rules derive conditional constraints between fields according to business requirements.

3. The method for automatically matching data quality rules for public security business data according to claim 1 is characterized in that: The data element node contains all attributes of the standard, the rule node stores rule logic and execution parameters, the standard node associates and references standard documents, and the synonym name node stores various synonymous expressions of the data element standard name.

4. The method for automatically matching data quality rules for public security business data according to claim 1 is characterized in that: The dependency relationship is: the rule node points to the dependent data element node; The synonymous name relationship is: by parsing the synonymous name column in the standard document and combining the machine learning method to generate synonymous names of the data element standard name, a synonymous name relationship with a confidence attribute is established between the data element node and the synonymous name node; The reference relationship is: the association between the marking rule and the standard document, and a directed relationship from the rule node to the standard node.

5. The method for automatically matching data quality rules for public security business data according to claim 4 is characterized in that: The specific value of the confidence is as follows: The confidence level of synonymous names explicitly defined in standard documents is set to 1.

0. The initial confidence level of synonymous names generated by machine learning is set to 0.6-0.

9. The confidence decay model is applied to synonymous name relationships from non-standard sources: in: is the initial confidence, is the time that has passed since the relationship was created, is the decay rate.

6. The method for automatically matching data quality rules for public security business data according to claim 1 is characterized in that: The hierarchical matching engine includes: Exact matching layer: matches the input field to be tested with the standard name or synonymous name of the data element in the knowledge graph; Semantic matching layer: If the exact matching layer fails to match successfully, the semantic similarity between the field to be tested and the standard data element is calculated, and the comprehensive similarity is obtained by weighted aggregation combined with the synonym name relationship. If the comprehensive similarity is greater than the set threshold, the match is successful; Statistical matching layer: For unmatched fields to be checked, similarity rules are recommended based on value distribution characteristics, and for numeric fields, range rules are generated based on value distribution characteristics.

7. The method for automatically matching data quality rules for public security business data according to claim 6 is characterized in that: The semantic matching layer specifically includes: The language model is used to convert the to-be-checked field, standard name, and synonymous name into a to-be-checked field vector, a standard data element vector, and a synonymous name vector, respectively; Calculate the cosine similarity between the field vector to be tested and the standard data element vector; If there are synonyms in the standard data element vector, a weighted aggregation strategy is used to calculate the comprehensive similarity. The first K synonyms are selected in descending order of confidence. The weighted aggregation formula is as follows: in, is the aggregate similarity, is the confidence of the i-th synonym name, is the cosine similarity calculation, is the field vector to be checked, is a synonym name vector; The final similarity formula is: in, is the comprehensive similarity, is the basic similarity, is the basic similarity weight, like , the match is successful.

8. The method for automatically matching data quality rules for public security business data according to claim 1 is characterized in that: The online optimization includes: Dynamic rule updates: monitor standard changes and automatically update the knowledge graph; Feedback learning mechanism: The confidence level of synonymous names is adjusted based on user confirmation results. The adjustment formula is as follows: in: is the adjusted confidence level, R is the user feedback score, For the forgetting factor; Performance optimization: establish rule cache index and dynamically adjust matching priority; Incremental expansion: supports injecting user-defined rules into the knowledge graph.

9. A data quality rule automatic matching system for public security business data using the method according to any one of claims 1 to 8, characterized in that: include: Rule generation module: Structural analysis of public security data element standard documents, extracting field identifiers, standard names, data types and formats, constraints, associated standard references and synonyms; constructing basic rules and association rules based on the analysis results; Knowledge graph construction module: Builds a quality rule knowledge graph, creates four types of entity nodes: data element nodes, rule nodes, standard nodes, and synonymous name nodes, and establishes three types of relationship edges: dependency relationships, synonymous name relationships, and reference relationships; The synonym name relationship includes the relationship source, confidence and validity time attributes; Feature extraction module: This module extracts the multimodal features of the fields to be inspected, generates semantic vectors for field descriptions using the BERT model, extracts business meaning labels as semantic features, and simultaneously obtains structural features, including field name, data type, length, and value distribution. Hierarchical matching engine module: Generates rules through the hierarchical matching engine and sequentially executes the three-level matching process of the exact matching layer, semantic matching layer, and statistical matching layer; Online optimization module: performs online optimization, dynamically updates knowledge graph nodes and implements feedback learning mechanism.

10. A data quality rule automatic matching device for public security business data, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the data quality rule automatic matching method for public security business data as described in any one of claims 1 to 8 when running the program instructions.

Citation Information

Cited By

  • Header field intelligent benchmarking method, system and device based on semantic index segmentation

    CN120910054A

  • Intelligent matching method, system and device for table header field based on semantic index segmentation

    CN120910054B

  • Power business data security classification and grading method and system

    CN121682432A