Enterprise multidimensional data intelligent mining and correlation analysis system

Abstract

The application relates to the technical field of data processing, and discloses an enterprise multidimensional data intelligent mining and correlation analysis system, which comprises the following modules: a traceability unified module, which is used for obtaining original data records from multiple systems, generating traceability information, and generating a unique caliber identifier based on a business index; an entity alignment module, which is used for extracting and normalizing hard anchors and soft anchors, determining whether the same entity is obtained, and outputting an entity mapping table; a relationship modeling module, which is used for constructing a typed relationship graph based on the entity mapping table, and calculating relationship weights; a label management module, which is used for defining an effective time interval and a caliber version for each label, and outputting a native label state; a diffusion inhibition module, which is used for determining whether a derived label is given in a weighted relationship graph; and a state determination module, which is used for determining the native label and the derived label state of the same entity, and outputting a final label state. The application realizes intelligent mining, correlation analysis, and label management of enterprise multidimensional data.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to an enterprise multidimensional data intelligent mining and correlation analysis system. Background Technology

[0002] With the continuous development of information technology, enterprises accumulate a large amount of data from different business systems, data platforms, and external data sources in their daily operations. This data covers business information from all areas of the enterprise. However, a major challenge facing enterprises is how to effectively manage and utilize this multi-source, multi-dimensional data, and ensure the consistency and accuracy of the data across multiple systems and departments.

[0003] Traditional data management and analysis methods typically rely on the functionality of a single system, making it difficult to handle large-scale data across departments and systems. Especially when dealing with multidimensional data, inconsistencies, label conflicts, and insufficient data source independence between different data sources and business systems often lead to incorrect decisions and data misuse. Therefore, achieving data uniformity, accuracy, and real-time performance in complex multidimensional data environments, and extracting valuable business insights from them, is a key issue in current enterprise data management. Summary of the Invention

[0004] This invention provides an intelligent mining and correlation analysis system for enterprise multidimensional data, which solves the technical problems of multi-source data integration, label consistency management, cross-system entity alignment, and relationship modeling in related technologies.

[0005] This invention provides an intelligent mining and correlation analysis system for enterprise multidimensional data, comprising:

[0006] The unified traceability module is used to acquire raw data records from multiple systems, generate traceability information including data source, provider, access time and data version, unify time zone and time granularity, and generate unique identifiers based on business indicators, tags or market relationships.

[0007] The entity alignment module is used to extract and standardize hard anchors and soft anchors from the source information, determine whether they are the same entity based on preset rules, and output an entity mapping table.

[0008] The relation modeling module is used to construct a typed relation graph based on the entity mapping table, and calculate the relation weights by combining relation type base weights, data quality, freshness, and source independence to obtain a weighted relation graph;

[0009] The tag governance module is used to define the effective time interval and caliber version for each tag based on the tags in the original data records, and to determine and output the original tag status of the tag in the entity and the current time.

[0010] The diffusion suppression module is used to determine whether to assign a derived label to an entity in a weighted relation graph based on constraints such as path reliability, path source independence, minimum number of independent paths, support threshold, maximum number of hops, and blocking evidence.

[0011] The state determination module is used to determine the original label state and derived label state of the same entity at the same time, and outputs the entity mapping table, weighted relationship diagram and the final label state based on the determination result.

[0012] Furthermore, the traceability unification module also includes: adding an integrity verification value to the traceability information, and calibrating based on a reference time source and attaching a calibration mark when the access time field is missing;

[0013] During the generation of unique caliber identifiers, when a conflict is detected between the unique caliber identifier and an identifier already existing in the historical record, a conflict detection process is triggered to compare the elements constituting the identifier. After confirming the conflict, the unique caliber identifier is regenerated by appending a version revision number. When the effective start time is missing, a confirmation mark is attached to the caliber identifier.

[0014] Furthermore, the hard anchor points include: tax identification number, license number, and device serial number; the soft anchor points include: name, address, telephone number, and contact person.

[0015] The standardization includes: removing non-numeric characters, word segmentation, and removing special symbols;

[0016] The preset rules include: when hard anchor points are consistent and there is no conflict, they are determined to be the same entity; the matching degree of soft anchor points is calculated using cosine similarity, and when the matching degree is greater than the preset matching threshold, they are determined to be the same entity.

[0017] The entity mapping table includes: a record for each entity and an identifier for the corresponding entity, as well as a conflict identifier.

[0018] Furthermore, the entity alignment module also includes a process for determining and marking anchor point conflicts, including:

[0019] Step 11: When determining the same entity, if the hard anchor point values ​​are inconsistent, the hard anchor point is missing and the matching degree of the soft anchor point is between the preset non-merging threshold and the preset merging threshold, or the same hard anchor point corresponds to multiple entities, it is determined to be a conflict, and a conflict unit is generated and placed into the judgment pool.

[0020] Step 12: Determine the conflicting units in the judgment pool according to the preset judgment rules. The preset judgment rules include: hard anchor point consistency takes precedence over soft anchor point matching; when hard anchor points have multiple values, the main value is determined according to the priority of the data provider; and the party with more independent evidence providers and a higher number of independent sources takes precedence.

[0021] Step 13: Write the determination result into the entity mapping table and attach a conflict identifier. The conflict identifier includes: no conflict, determined, and pending determination.

[0022] Furthermore, the typed relational graph includes nodes and edges, where nodes represent entities in the entity mapping table, and edges are established in the following ways:

[0023] Calculate the spatial distance between two entities. If the spatial distance is less than a preset distance threshold, an edge is established between the two entities.

[0024] When two entities overlap in time, an edge is established between the two entities;

[0025] When there is a physical connection between two entities, an edge is established between the two entities.

[0026] Furthermore, the steps for calculating relation weights include:

[0027] Step 21: Set the base weights for relation types based on different edge establishment methods;

[0028] Step 22: Obtain the number of verifiable fields and the number of required fields for each edge, and calculate the ratio of the two to obtain data quality;

[0029] Step 23: Calculate the difference between the data access time and the current time, and combine the difference with a preset decay constant using an exponential decay function to obtain the freshness.

[0030] Step 24: Obtain the source independence by subtracting the sum of the proportions of each data provider's evidence to the total number of evidences.

[0031] Step 25: Obtain the relation weight by multiplying the relation type base weight, data quality, freshness, and source independence.

[0032] Furthermore, the relationship modeling module also includes: after obtaining the relationship weights of multiple edges of a pair of entities, dividing the relationship weight of each edge by the sum of the relationship weights of all edges of the pair of entities to obtain a normalized weight; when the normalized weight is lower than a preset weight threshold, the edge is marked as an invalid edge.

[0033] Furthermore, based on the tags in the original data records, a valid time interval and caliber version are defined for each tag, and the original tag status of the tag in the entity and at the current time is determined and output, including:

[0034] Step 31: Extract the label field from the original data record and mark the start time, end time, and caliber version; where the start time and end time are the timestamps of the label taking effect and becoming invalid, respectively;

[0035] Step 32: Determine the validity status of the tag based on the valid time interval of the tag field; if the current time is within the valid time interval, the tag is valid; otherwise, the tag is invalid.

[0036] Step 33: Output the original tag status, including: tag unique identifier, validity status, validity time range, and caliber version.

[0037] Furthermore, in the weighted relationship graph, based on constraints such as path reliability, path source independence, minimum number of independent paths, path support, maximum number of hops, and blocking evidence, it is determined whether to assign a derived label to an entity, including:

[0038] Step 41: For paths between entities, take the minimum value of the relation weights of all edges in the path as the path reliability.

[0039] Step 42: Calculate the geometric mean of the source independence of each edge on the path as the path source independence;

[0040] Step 43: Multiply the path reliability and the path source independence to obtain the path support.

[0041] Step 44: Assign a derived tag to the entity and set the validity flag of the derived tag to 1 when the following conditions are met; otherwise, set it to 0:

[0042] The number of paths with no shared edges and different evidence providers is not less than the preset number of independent paths;

[0043] The support level of each path is no less than the preset support threshold;

[0044] The entity's native tag state is valid only at a specific time.

[0045] The number of hops on the path is not greater than the preset maximum number of hops;

[0046] There is no valid evidence of blocking it.

[0047] Furthermore, for the original and derived label states of the same entity at the same time, the label state is determined, and based on the determination result, an entity mapping table, a weighted relationship graph, and the final label state are output, including:

[0048] Step 51: Obtain the original tag status and derived tag status, and compare them one by one according to the entity identifier and the specific timestamp to form a tag status pair;

[0049] Step 52: Perform priority arbitration on the tag status. If the original tag status is valid and consistent with the derived tag status, the tag status is valid. If the original tag status is valid but the derived tag status is invalid, the original tag status shall prevail. If the original tag status is missing but the derived tag status is valid, the derived tag status shall prevail. If both are valid but conflict, the original tag status shall prevail, and the derived tag status shall cover the original tag status when the original tag status is incomplete.

[0050] Step 53: Based on the determination result of step 52, output the entity mapping table, weighted relationship diagram and the final label status of the label. The final label status includes the label unique identifier, validity identifier, valid time interval and caliber version.

[0051] The beneficial effects of this invention are as follows: The invention efficiently integrates multidimensional data from multiple systems and data sources through a unified traceability module, resolving issues such as time inconsistencies and format differences in cross-system data integration, ensuring data consistency and high quality. The entity alignment module accurately identifies and aligns entity data across systems through a combination of hard and soft anchors, ensuring accurate data mapping. The label governance module defines the effective time interval and caliber version of labels, resolving multi-version label conflicts and ensuring label validity. The relationship modeling module displays multidimensional relationships between entities through a weighted relationship graph, accurately calculating relationship weights based on factors such as path reliability and source independence, providing reliable support for decision-making. The diffusion suppression module and state determination module dynamically determine the derived labels of entities, ensuring that label generation is based on highly reliable path data and avoiding erroneous labels caused by conflicts. Overall, this invention improves the efficiency of data integration and label management, enhances enterprises' intelligent decision-making and analysis capabilities in multi-source data environments, and effectively supports operational optimization and business decisions. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the modules of the enterprise multidimensional data intelligent mining and correlation analysis system of the present invention. Detailed Implementation

[0053] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0054] like Figure 1 As shown, the enterprise multidimensional data intelligent mining and correlation analysis system includes:

[0055] The unified traceability module is used to acquire raw data records from multiple systems, generate traceability information including data source, provider, access time and data version, unify time zone and time granularity, and generate unique identifiers based on business indicators, tags or market relationships.

[0056] The entity alignment module is used to extract and standardize hard anchors and soft anchors from the source information, determine whether they are the same entity based on preset rules, and output an entity mapping table.

[0057] The relation modeling module is used to construct a typed relation graph based on the entity mapping table, and calculate the relation weights by combining relation type base weights, data quality, freshness, and source independence to obtain a weighted relation graph;

[0058] The tag governance module is used to define the effective time interval and caliber version for each tag based on the tags in the original data records, and to determine and output the original tag status of the tag in the entity and the current time.

[0059] The diffusion suppression module is used to determine whether to assign a derived label to an entity in a weighted relation graph based on constraints such as path reliability, path source independence, minimum number of independent paths, support threshold, maximum number of hops, and blocking evidence.

[0060] The state determination module is used to determine the original label state and derived label state of the same entity at the same time, and outputs the entity mapping table, weighted relationship diagram and the final label state based on the determination result.

[0061] In one embodiment of the present invention, the traceability unification module is used to organize and identify raw data records from multiple business systems, data platforms, and external data sources. First, for the raw data accessed by each system, the module generates traceability information including the data source, provider, access time, and data version. It also unifies the access time field to the same time zone and time granularity to eliminate discrepancies caused by inconsistent time representations across systems. During this process, the module further adds an integrity check value to the traceability information. By calculating the check result for the data field and appending it to the traceability information, subsequent data retrieval can quickly verify whether the data is missing or tampered with, thereby improving the credibility of the data in a multi-source environment and ensuring that enterprises can rely on reliable data sources when conducting big data analysis, real-time monitoring, and decision support. When the access time field is missing in the raw data record, the module calls a reference time source for calibration, fills in the missing value, and adds a calibration mark to ensure the accuracy of subsequent time series analysis.

[0062] During the generation of unique identifiers, the module combines key elements such as business metrics, tags, or market relationships from the original records to form the identifier. When a conflict is detected between the generated unique identifier and an existing identifier in the historical record, the system triggers a conflict detection process to compare the various elements constituting the identifier to confirm whether a conflict actually exists. If a conflict is found, the unique identifier is regenerated by appending a version revision number, thereby ensuring the uniqueness and distinguishability of the identifier globally. Furthermore, when the generated identifier lacks an effective start time, the module adds a confirmation flag to the identifier to indicate that further confirmation is needed in the time dimension during subsequent processing.

[0063] Through the above processing flow, the traceability unified module not only ensures the integrity and consistency of data during the access process, but also ensures the uniqueness and traceability of the unique identifier in a dynamic enterprise environment through calibration, conflict detection and version management, thus providing a reliable foundation for subsequent data alignment, relationship modeling and label governance.

[0064] In one embodiment of the present invention, the entity alignment module achieves unified identification of entities across systems by combining hard anchors and soft anchors. Specifically, hard anchors include tax identification numbers, license numbers, and device serial numbers, while soft anchors include names, addresses, telephone numbers, and contact persons. For the accessed entity data, normalization processing is first performed to ensure semantic and format consistency between data from different sources. This normalization processing includes removing non-numeric characters from telephone numbers, performing word segmentation on addresses, and removing special symbols from names, thereby avoiding matching deviations caused by format differences.

[0065] Based on the standardized data, the system determines entities according to preset rules. When hard anchor points have consistent values ​​and no conflicts, they are directly identified as the same entity, ensuring accuracy and efficiency. When hard anchor points are missing or insufficient for determination, cosine similarity is used to calculate the matching degree between soft anchor points. When the matching degree is greater than a preset threshold, they are identified as the same entity. Entities may include: equipment, products, employees, locations, customers, orders, contracts, tags, etc.

[0066] In one embodiment of the present invention, the entity alignment module further includes a process for determining and marking anchor point conflicts, comprising:

[0067] Step 11: When determining the same entity, if the hard anchor point values ​​are inconsistent, the hard anchor point is missing and the matching degree of the soft anchor point is between the preset non-merging threshold and the preset merging threshold, or the same hard anchor point corresponds to multiple entities, it is determined to be a conflict, and a conflict unit is generated and placed into the judgment pool.

[0068] Step 12: Determine the conflicting units in the judgment pool according to the preset judgment rules. The preset judgment rules include: when the hard anchors are consistent and there is no conflict, the system directly determines them as the same entity; when there is a conflict in the hard anchors, the system will give priority to the matching result of the hard anchors, and the matching degree of the soft anchors will be used as an auxiliary judgment basis; when there are multiple values ​​for the hard anchors, the system will judge according to the priority of the data provider and select the hard anchor with higher priority as the final matching basis; if they come from different evidence providers and these sources are independent of each other, the system will use the provider with the higher number of independent sources as the final judgment basis.

[0069] Step 13: Write the judgment result into the entity mapping table and attach a conflict identifier. The conflict identifier includes: no conflict, judged, and pending judgment. When marked as pending judgment, the relevant record will continue to be manually reviewed or re-judged in subsequent processes until the conflict is effectively resolved.

[0070] In this embodiment, the entity alignment module can effectively identify and handle conflicts in data records from different sources, thereby improving the accuracy and consistency of entity mapping. In multi-source data scenarios, this module can dynamically address inconsistencies between hard and soft anchors. Through precise judgment rules, it ensures that the system can efficiently align data across multiple data sources and different departments, resolving data inconsistencies caused by differences in business definitions, data versions, and time intervals.

[0071] In one embodiment of the present invention, the typed relationship graph is used to represent the relationship between entities, wherein nodes represent entities in an entity mapping table, and edges represent the association between entities.

[0072] The methods for establishing edges include:

[0073] Calculate the spatial distance between two entities. When the spatial distance is less than a preset distance threshold, establish an edge between the two entities. This method is suitable for representing the physical proximity or direct association between entities. It can help the system quickly identify entities that are closely connected in geographical location and represent their relationship in the form of edges in a typified relationship graph.

[0074] When two entities overlap in time, an edge is established between them. This approach is suitable for entities that share common behaviors or events within a time period and helps to identify connections between entities that are closely related in time.

[0075] When there is a physical connection between two entities, an edge is established between them. Physical connections include network connections, device connections, and logistics routes. The establishment of such edges can reflect the actual, contactable connections between entities, such as entities connected through physical media such as sensors and communication lines, thereby further enhancing the system's ability to model physical relationships.

[0076] The typified relationship diagram established in the above manner can effectively display the multi-dimensional relationships between entities, reflecting not only spatial proximity but also temporal intersection and physical connection.

[0077] In one embodiment of the present invention, the calculation steps of relation weights include:

[0078] Step 21: Set the base weight of the relation type based on the different ways of establishing the edges; for example, entity pairs that are spatially close will be given a higher base weight of the relation type, while entity pairs that have less temporal overlap will be given a lower base weight of the relation type.

[0079] Step 22: Obtain the number of verifiable fields and the number of required fields for each edge, and calculate the ratio of the two to obtain data quality; where the number of verifiable fields represents the number of fields in the data record that can actually be verified or checked, and the number of required fields represents the number of fields that are expected to exist in the data record.

[0080] Step 23: Calculate the difference between the data access time and the current time. Using an exponential decay function, combine this difference with a preset decay constant to obtain the freshness. The formula for calculating freshness is: , Indicates freshness, reflecting the timeliness of data; exp represents an exponential function. This represents the preset attenuation constant, which is preferably set to 0.01. Indicates the data access time. Indicates the current time.

[0081] Step 24: The source independence is obtained by subtracting the sum of the proportions of evidence from each data provider to the total number of evidences. The higher the source independence, the stronger the credibility of the relationship between the entities; conversely, the lower the source independence, the weaker the credibility. Evidence refers to supporting information submitted by the source or provider associated with the entity data, including the data itself, as well as provider identification, timestamps, versions, etc.

[0082] Step 25: Obtain the relation weight by multiplying the relation type base weight, data quality, freshness, and source independence.

[0083] Through the above steps, the calculation of relation weights not only considers the type of relation, but also integrates multiple factors such as data quality, timeliness, and evidence source, ensuring that the relation weight of each edge in the typified relation graph can accurately reflect the actual strength of the association between entities.

[0084] In one embodiment of the present invention, the relationship modeling module further includes: after obtaining the relationship weights of multiple edges of a pair of entities, dividing the relationship weight of each edge by the sum of the relationship weights of all edges of the pair of entities to obtain a normalized weight; when the normalized weight is lower than a preset weight threshold, marking the edge as an invalid edge; and excluding it from subsequent path calculation or relationship analysis. Through this operation, the system can automatically remove edges that contribute little to the quality of the typed relationship graph, thereby improving the proportion of valid edges in the typed relationship graph and the overall computational efficiency.

[0085] In one embodiment of the present invention, based on the tags in the original data records, a valid time interval and caliber version are defined for each tag, and the original tag status of the tag in the entity and the current time is determined and output, including:

[0086] Step 31: Extract the tag field from the original data record and mark the start time, end time, and caliber version; where the start time and end time are the timestamps of the tag taking effect and becoming invalid, respectively; the caliber version indicates the definition version of the tag at a certain point in time; these parameters ensure that the tag is valid within a specific time range and can be distinguished and managed according to different versions.

[0087] Step 32: Determine the validity status of the tag based on the valid time interval of the tag field; if the current time is within the valid time interval, the tag is valid, otherwise the tag is invalid; this judgment process can accurately reflect the validity of the tag at a specific point in time, avoiding data inconsistency problems caused by time differences or tag expiration.

[0088] Step 33: Output the original tag status, including: tag unique identifier, validity status, validity time range, and caliber version.

[0089] In this embodiment, the system can not only accurately manage and use tags in a multi-version, multi-time-range data environment, but also ensure the time validity and version consistency of tags, further improving the efficiency of data governance and tag management. This mechanism provides a high-quality data foundation for subsequent cross-system data integration, business process optimization, and intelligent decision support, enhancing the accuracy and flexibility of enterprise data management.

[0090] In one embodiment of the present invention, in a weighted relationship graph, determining whether to assign a derived label to an entity based on constraints such as path reliability, path source independence, minimum number of independent paths, path support, maximum number of hops, and blocking evidence includes:

[0091] Step 41: For paths between entities, the minimum value of the relation weights of all edges in the path is taken as the path reliability. This step ensures that the strength of the path is determined by the weakest edge, reflecting the impact of the most unstable part of the path on the overall path reliability. The lower the path reliability, the higher the uncertainty of the path.

[0092] Step 42: Calculate the geometric mean of the source independence of each edge on the path as the path source independence; by using the geometric mean, the system can comprehensively consider the independence of the sources of each edge, ensuring that the evidence sources of each edge in the path are sufficiently diverse and independent, thereby improving the credibility of the path.

[0093] Step 43: Multiply the path reliability and path source independence to obtain the path support. The path support comprehensively considers the reliability and diversity of the path, reflecting the overall credibility of the path. In this process, paths with higher path support indicate more reliable data, and are therefore more likely to support assigning derived labels to entities.

[0094] Step 44: Assign a derived tag to the entity and set the validity flag of the derived tag to 1 when the following conditions are met; otherwise, set it to 0:

[0095] The number of paths with no shared edges and different evidence providers is not less than the preset number of independent paths, meaning that the paths are sufficiently independent.

[0096] The support level of each path is no less than the preset support threshold to ensure that the reliability of the path meets the minimum standard.

[0097] The original tag state of an entity is valid at a specific time, ensuring that the original tag of the entity is consistent with the derived tag.

[0098] The number of hops in the path should not exceed the preset maximum number of hops to avoid excessive information decay due to excessively long paths.

[0099] There is no valid blocking evidence, meaning there is no evidence to suggest that there are problems or conflicts in the data along the path. Blocking evidence refers to evidence that can negate or terminate the data relationship during this process.

[0100] Through the above processing flow, the system can assign derived labels to entities, ensuring that label generation is based on highly reliable, independent, and diverse paths, while avoiding erroneous labeling due to unreliable paths or insufficient evidence. This effectively enhances the accuracy of labels and the credibility of data analysis, ensuring that enterprises can rely on accurate and high-quality label information when conducting data mining, prediction, and decision support based on typified relationship graphs.

[0101] In one embodiment of the present invention, for the original tag state and derived tag state of the same entity at the same time, a tag state determination is performed, and an entity mapping table, a weighted relationship graph, and the final tag state are output based on the determination result, including:

[0102] Step 51: Obtain the original tag status and derived tag status, and compare them one by one with the entity identifier and the specific timestamp to form tag status pairs; each tag status pair corresponds to a specific entity and timestamp. In this way, the system can accurately identify the changes in tag status in the time and entity dimensions, ensuring that the tag status judgment can be closely combined with the actual entity and time information.

[0103] Step 52: Perform priority arbitration on the tag status. If the original tag status is valid and consistent with the derived tag status, the tag status is valid. If the original tag status is valid but the derived tag status is invalid, the original tag status takes precedence. If the original tag status is missing but the derived tag status is valid, the derived tag status takes precedence. If both are valid but conflict, the rule of prioritizing the original tag status and covering the derived tag status when the original tag status is incomplete is applied to ensure that the logic of tag assignment is reasonable and consistent.

[0104] Step 53: Based on the determination result of step 52, output the entity mapping table, weighted relationship diagram and the final label status of the label. The final label status includes the label unique identifier, validity identifier, valid time interval and caliber version.

[0105] Through the above steps, the system can handle the differences between native and derived label states, ensuring high accuracy and consistency in label validity determination even in complex data environments. This mechanism enhances the accuracy and efficiency of data governance, label management, and multi-source data integration, providing reliable data support for enterprises in intelligent decision-making, data analysis, and cross-system data collaboration.

[0106] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0107] The embodiments of the present invention have been described above, but the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of the present embodiments, all of which are within the protection scope of the present embodiments.

Claims

1. A system for intelligent mining and correlation analysis of multi-dimensional data of an enterprise, characterized in that, The method comprises the following steps: a traceability unification module is used to obtain original data records from multiple systems, generate traceability information containing data sources, providers, access times and data versions, unify time zones and time granularities, and generate unique caliber identifiers based on business indicators, labels or market relationships; an entity alignment module is used to extract and normalize hard anchors and soft anchors from the traceability information, determine whether they are the same entity based on preset rules, and output an entity mapping table; wherein the hard anchors include tax numbers, license numbers and device serial numbers, and the soft anchors include names, addresses, phone numbers and contacts; the preset rules include: when the values of the hard anchors are consistent and there is no conflict, it is directly determined that they are the same entity; when the hard anchors are missing or insufficient to complete the determination, the matching degrees between the soft anchors are calculated using cosine similarity, and when the matching degrees are greater than a preset threshold, it is determined that they are the same entity; wherein, when the values of the hard anchors are inconsistent, the hard anchors are missing, and the matching degrees of the soft anchors are between a preset non-merging threshold and a preset merging threshold, or multiple entities correspond to the same hard anchor, it is determined that there is a conflict; a relationship modeling module is used to construct a typed relationship graph based on the entity mapping table, calculate relationship weights by combining relationship type base weights, data quality, freshness and source independence, and obtain a weighted relationship graph; a label management module is used to define valid time intervals and caliber versions for each label based on labels in the original data records, determine and output the original label state of the label under the entity and the current time; a diffusion suppression module is used to determine whether to assign a derived label to an entity based on path reliability, path source independence, a minimum preset number of independent paths, path support, a preset support threshold, a preset maximum number of hops and blocking evidence constraints in the weighted relationship graph; wherein the path support is obtained by multiplying the path reliability and the path source independence; a state determination module is used to determine the label state of the same entity at the same time, and output the entity mapping table, the weighted relationship graph and the final label state of the label according to the determination result.

2. The system for enterprise multidimensional data intelligence mining and correlation analysis of claim 1, wherein, The traceability unification module further comprises adding an integrity check value in the traceability information, and calibrating based on a reference time source when the access time field is missing and adding a calibration mark; in the process of generating the unique caliber identifier, when it is detected that the unique caliber identifier conflicts with the existing unique caliber identifier in the historical record, a conflict detection process is triggered, the elements constituting the unique caliber identifier are compared, and the unique caliber identifier is regenerated by adding a version revision number after the conflict is confirmed; when the effective start time is missing, the unique caliber identifier is attached with a to-be-confirmed mark.

3. The system of claim 1, wherein, The normalization includes removing non-numeric characters from the phone number, word segmentation processing and removing special symbols; the entity mapping table includes the records of each entity and the identification of the corresponding entity, and further includes a conflict identifier.

4. The system of claim 3, wherein, The entity alignment module further comprises a judgment and marking process for anchor conflicts, which comprises: Step 11, when the same entity determination is performed, if the hard anchor point values are inconsistent, the hard anchor point is missing, the matching degree of the soft anchor point is between the preset non-merging threshold and the preset merging threshold, and the same hard anchor point corresponds to multiple entities, it is determined as a conflict, and a conflict unit is generated and placed in a to-be-determined pool; Step 12, according to a preset determination rule, the conflict units in the to-be-determined pool are determined, and the preset determination rule includes: the hard anchor point consistency is prior to the soft anchor point matching, when the hard anchor point has multiple values, the main value is determined according to the priority of the evidence provider, and the party with higher independent source number is prior to the party with independent source; wherein, the evidence includes data itself and provider, timestamp, and data version; Step 13, the determination result is written into an entity mapping table and a conflict identifier is attached, and the conflict identifier includes: no conflict, determined and to-be-determined.

5. The system of claim 1, wherein, The typed relationship graph includes nodes and edges, the node represents an entity in the entity mapping table, and the establishment mode of the edge includes: The spatial distance between two entities is calculated, and when the spatial distance is less than a preset distance threshold, an edge is established between the two entities; When two entities overlap in time, an edge is established between the two entities; When there is a physical connection between two entities, an edge is established between the two entities.

6. The system of claim 5, wherein, The calculation steps of the relationship weight include: Step 21, the relationship type base weight is set based on different establishment modes of the edge; Step 22, the number of verifiable fields and the number of required fields of each edge are obtained, and the ratio of the two is calculated to obtain data quality; Step 23, the difference between the data access time and the current time is calculated, and the difference is combined with a preset attenuation constant through an exponential decay function to obtain freshness; Step 24, the source independence is obtained by subtracting the sum of the proportions of the evidence numbers of each evidence provider in the total evidence number from one; wherein, the evidence includes data itself and provider, timestamp, and data version; Step 25, the relationship weight is obtained by multiplying the relationship type base weight, the data quality, the freshness and the source independence.

7. The system of claim 6, wherein, The relationship modeling module further includes: after obtaining the relationship weights of multiple edges of a pair of entities, the relationship weight of each edge is divided by the total of the relationship weights of all edges of the pair of entities to obtain a normalized weight, and when the normalized weight is lower than a preset weight threshold, the edge is marked as an invalid edge.

8. The system of claim 1, wherein, Based on the label in the original data record, the effective time interval and the caliber version of each label are defined, the original label state of the label under the entity and the current time is determined and output, including: Step 31, the label field is extracted from the original data record, and the start time, end time and caliber version are marked; wherein, the start time and the end time are the timestamps of the label taking effect and invalid respectively; Step 32, according to the effective time interval of the label field, the validity state of the label is determined; if the current time is within the effective time interval, the label is valid, otherwise the label is invalid; Step 33, the original label state of the label is output, including: label unique identifier, validity state, effective time interval and caliber version.

9. The system of claim 1, wherein, In the weighted relation graph, based on path reliability, path source independence, minimum preset independent path number, path support, preset support threshold, preset maximum hop number and blocking evidence constraint, it is judged whether to give a derived label to an entity, including: Step 41, for the path between entities, taking the minimum value of the relationship weight of all edges in the path as the path reliability; Step 42, calculating the geometric mean of the source independence of each edge in the path as the path source independence; Step 43, multiplying the path reliability and the path source independence to get the path support; Step 44, when the following conditions are met, give the entity a derived label, and set the validity identifier of the derived label to 1, otherwise to 0: There are no common edges and the number of paths with different evidence providers is not less than the minimum preset independent path number; The support of each path is not less than the preset support threshold; The original label state of the entity is valid at a certain time; The path hop number is not greater than the preset maximum hop number; There is no effective blocking evidence.

10. The system of claim 1, wherein, For the original label state and the derived label state of the same entity at the same time, the label state is judged, and the final label state of the entity mapping table, the weighted relation graph and the label is output according to the judgment result, including: Step 51, get the original label state and the derived label state, and compare them one by one according to the entity identifier and the specific time stamp to form a label state pair; Step 52, execute priority arbitration on the label state pair, if the original label state is valid and consistent with the derived label state, the label state is valid; if the original label state is valid and the derived label state is invalid, the original label state is used as reference; if the original label state is missing and the derived label state is valid, the derived label state is used as reference; if both are valid but there is conflict, the original label state is given priority, and the rule of the derived label state covering the incomplete original label state is determined; Step 53, according to the judgment result of step 52, output the entity mapping table, the weighted relation graph and the final label state of the label, the final label state contains label unique identifier, validity identifier, valid time interval and caliber version.

Images