Data resource management system based on logic management and management method thereof

Abstract

The invention discloses a data resource management system based on logic management and a management method thereof, relates to the technical field of data resource management, and aims to identify various data resources in an organization, including structured, semi-structured and unstructured data, define attributes, relationships and business meanings of each data resource, record metadata information of the data resources and store the metadata information of the data resources. According to the method, data resources are classified, different data layers are determined according to service purposes and values of the data resources, the data are divided into core service data and supportive data, multiple quality parameters of the data resources are comprehensively analyzed, and the data resources with quality defects are screened out based on a data quality monitoring mechanism. According to the data management method, the data resources with the quality reaching the standard are managed and supervised based on the data management method, the data resources are managed in the life cycle according to the data life cycle management strategy, and the management method not only manages and maintains the data resources in a more targeted mode, but also effectively avoids waste of the management resources.

Description

Technical Field

[0001] The present invention relates to the technical field of data resource management, and particularly relates to a data resource management system based on logical management and its management method. Background Art

[0002] With the rapid development of information technology, the amount of data generated within an organization has increased sharply. Data no longer exists only in paper documents but exists in digital form in various systems and applications, making it crucial to manage data effectively. A management system is a system specifically designed to collect, store, organize, and manage data resources within an organization. The emergence of these systems is to meet the challenges and requirements faced by today's enterprises and organizations in the big data era;

[0003] The existing technologies have the following deficiencies:

[0004] The existing management methods usually manage all data resources in an integrated manner. This management method not only fails to effectively manage different data resources but may also manage data resources with low quality or outside the life cycle, thus wasting management resources. Summary of the Invention

[0005] The purpose of the present invention is to provide a data resource management system based on logical management and its management method to solve the deficiencies in the background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A data resource management method based on logical management, the management method comprising the following steps:

[0007] Identify various data resources within the organization, including structured, semi-structured, and unstructured data, define the attributes, relationships, and business meanings of each data resource, and record the metadata information of the data resources, including data sources, formats, quality standards, and business rules;

[0008] Classify the data resources, determine different data levels according to their business uses and values, divide the data into core business data and supporting data, perform data access control based on data access policies, and protect sensitive data based on security measures;

[0009] Comprehensively analyze multiple quality parameters of the data resources, and perform screening processing on the data resources with quality defects based on a data quality monitoring mechanism;

[0010] The data resources with qualified quality are managed and supervised based on data governance methods, and according to the data life cycle management strategy, including different stages of data creation, storage, use, archiving, and destruction, the data resources are managed within the life cycle;

[0011] Set up a data integration interface to integrate data with other systems and complete the transmission management of data.

[0012] Preferably, obtain the consistency pass rate, data latency, duplicate record ratio, and missing value ratio of the data resources;

[0013] Comprehensively calculate the resource assignment by combining the consistency pass rate, data latency, duplicate record ratio, and missing value ratio. The calculation expression is:

[0014]

[0015] In the formula, ZYF is the resource assignment, yzt, syc, cfj, and qsb are the consistency pass rate, data latency, duplicate record ratio, and missing value ratio respectively, and k1, k2, k3, and k4 are the weights of the consistency pass rate, data latency, duplicate record ratio, and missing value ratio respectively, and k1, k2, k3, and k4 are all greater than 0;

[0016] After obtaining the resource assignment, compare the resource assignment with the quality threshold. If the resource assignment is greater than or equal to the quality threshold, analyze that the quality of the data resources is good. If the resource assignment is less than the quality threshold, analyze that the quality of the data resources is poor.

[0017] Preferably, define the attributes, relationships, and business meanings of each data resource, including the following steps:

[0018] Classify the identified data resources, including customer data, sales data, and product data. Assign a naming standard to each data resource, determine the attributes of each data resource, the specific fields or attributes included in the data, including the name, address, and contact information of customer data. Define the data type, length, and format attributes for each attribute. Use a relational model tool to describe the relationships between data resources, including primary keys and foreign keys. Record the detailed information of each data resource in the metadata management system, including attribute definitions, relational models, and business meanings. Based on the defined attributes and relationships, create a data dictionary, provide documentation for each data resource, and the data dictionary includes terms, definitions, examples, and relationship descriptions with other data resources.

[0019] Preferably, record the metadata information of the data resources, including data sources, formats, quality standards, and business rules, including the following steps:

[0020] Establish a centralized metadata repository for centralized storage and management of metadata information of all data resources, define the fields included in the metadata model, including data sources, formats, quality standards, business rules, record the source of each data resource, including which system, application, or data provider generated the data, record the format of each data resource, including descriptions of structured, semi-structured, and unstructured data. For structured data, define the table structure, field names, and data types. For unstructured data, describe the file format and tags.

[0021] Preferably, classify the data resources, determine different data levels according to their business uses and values, and divide the data into core business data and supporting data, including the following steps:

[0022] Evaluate each data resource to determine its contribution to business decision-making, operations, and strategic goals, rank them according to value, divide the data resources into different categories according to business uses and values, identify data resources directly related to core business processes, and identify data resources that are not directly related to core business processes but are relevant to business support and operations. Divide the data resources into different levels and formulate corresponding management strategies for each data level, including data quality standards, security requirements, and update frequencies.

[0023] Preferably, perform data access control based on data access policies, including the following steps:

[0024] According to the organizational structure and job responsibilities, define different user roles, such as sales representatives, sales managers, and senior management. Analyze the data levels and scopes that each role needs to access, and clearly define data access permissions for each user role, including read, modify, and delete permissions. Divide the sales data into different levels, use rules to distinguish different access scenarios, implement an authentication mechanism, and after authentication, perform authorization according to the user's role and access rules.

[0025] Preferably, protect sensitive data based on security measures to avoid leakage of sensitive data, including the following steps:

[0026] Sensitive data includes customer personal identity information, financial data, etc. Use data classification markers to add tags or metadata to sensitive data, protect sensitive data through encryption algorithms, establish a key management strategy to ensure the management of key generation, storage, distribution, and rotation, encrypt sensitive data in storage, and use a secure transmission protocol during data transmission.

[0027] Preferably, according to the data life cycle management strategy, including different stages of data creation, storage, use, archiving, and destruction, manage data resources throughout their life cycle, including the following steps:

[0028] During the data creation phase, clarify the purpose of data generation and business requirements, record the metadata of the data, including data source, format, structure, and creation time, archive the data that is no longer frequently used to general storage media, update the metadata, mark the data as archived, record the archive time and location, formulate data archiving and retrieval strategies to ensure that the archived data can be accessed on demand, for data that is no longer needed, perform thorough deletion, record the process and results of data destruction, including destruction time, method, and person in charge, and regularly review the data life cycle management strategy to ensure compliance with business requirements and regulations.

[0029] A data resource management system based on logical management, including an identification module, a definition and recording module, a classification module, a security module, a quality analysis module, a management module, and an integrated transmission module;

[0030] Identification module: Identify various data resources within the organization, including structured, semi-structured, and unstructured data;

[0031] Definition and recording module: Define the attributes, relationships, and business significance of each data resource, and record the metadata information of the data resource, including data source, format, quality standards, and business rules;

[0032] Classification module: Classify data resources, determine different data levels according to their business uses and values, and divide data into core business data and supporting data;

[0033] Security module: Perform data access control based on data access policies and protect sensitive data based on security measures;

[0034] Quality analysis module: Comprehensively analyze multiple quality parameters of data resources, and screen out data resources with quality defects based on the data quality monitoring mechanism;

[0035] Management module: Manage and supervise data resources that meet the quality standards based on data governance methods, and manage data resources throughout their life cycle according to the data life cycle management strategy, including different stages of data creation, storage, use, archiving, and destruction;

[0036] Integrated transmission module: Set up data integration interfaces to integrate data with other systems and complete data transmission management.

[0037] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0038] 1. The present invention identifies various data resources within an organization, including structured, semi-structured, and unstructured data, defines the attributes, relationships, and business significance of each data resource, records the metadata information of the data resources, classifies the data resources, determines different data levels according to their business uses and values, divides the data into core business data and supporting data, comprehensively analyzes multiple quality parameters of the data resources, performs screening processing on the data resources with quality defects based on a data quality monitoring mechanism, manages and supervises the data resources that meet the quality standards based on data governance methods, and manages the data resources within their life cycles according to the data life cycle management strategy. This management method not only manages and maintains the data resources more specifically but also effectively avoids waste of management resources;

[0039] 2. The present invention obtains the consistency pass rate, data latency, duplicate record ratio, and missing value ratio of the data resources, comprehensively calculates the resource assignment by combining the consistency pass rate, data latency, duplicate record ratio, and missing value ratio, and after obtaining the resource assignment, compares the resource assignment with the quality threshold, and analyzes the quality of the data resources based on the comparison result. The multi-dimensional analysis method is more comprehensive and effectively improves the accuracy of data resource analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other accompanying drawings based on these drawings.

[0041] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0043] Embodiment 1: Please refer to Figure 1 As shown, a data resource management method based on logical management in this embodiment includes the following steps:

[0044] Identify various data resources within the organization, including structured, semi-structured, and unstructured data. Define the attributes, relationships, and business meanings of each data resource, and record the metadata information of the data resources, including data sources, formats, quality standards, business rules, etc. Ensure the consistency and update of the metadata to maintain an accurate description of the data resources. Classify the data resources, determine different data levels based on their business uses and values, and divide the data into core business data, supporting data, etc., in order to manage and maintain them more specifically. Implement data access control based on data access policies, determining who has the right to access, modify, and delete different levels of data. Protect sensitive data based on security measures to avoid the leakage of sensitive data. Comprehensively analyze multiple quality parameters of the data resources, and screen out data resources with quality defects based on the data quality monitoring mechanism. Manage and supervise data resources that meet the quality standards based on data governance methods. According to the data life cycle management strategy, including different stages such as the creation, storage, use, archiving, and destruction of data, manage the data resources throughout their life cycles to avoid waste of management resources. Set up data integration interfaces to enable effective integration of data with other systems (such as marketing, sales, and customer service), and complete the transmission management of the data.

[0045] This application identifies various data resources within the organization, including structured, semi-structured, and unstructured data, defines the attributes, relationships, and business meanings of each data resource, records the metadata information of the data resources, classifies the data resources, determines different data levels based on their business uses and values, divides the data into core business data and supporting data, comprehensively analyzes multiple quality parameters of the data resources, screens out data resources with quality defects based on the data quality monitoring mechanism, manages and supervises data resources that meet the quality standards based on data governance methods, and manages the data resources throughout their life cycles according to the data life cycle management strategy. This management method not only manages and maintains the data resources more specifically but also effectively avoids waste of management resources.

[0046] Example 2: Identify various data resources within the organization, including structured, semi-structured, and unstructured data, including the following steps:

[0047] Data inventory and investigation: Conduct a data inventory within the organization to determine the locations where the data is stored, such as databases, file systems, applications, etc.

[0048] Conduct an investigation to understand the data that different departments and teams may be using.

[0049] Dialogue with relevant departments: Have conversations with the heads, data administrators, and system administrators of each department and business team.

[0050] Understand the data requirements of each department and the structured, semi-structured, and unstructured data they may have.

[0051] Metadata collection: Collect metadata, including database table structures, field descriptions, data dictionaries, etc.

[0052] Obtain the metadata of semi-structured data, such as the structure and field definitions of XML files.

[0053] For unstructured data, understand the file types, formats, content descriptions, etc.

[0054] Investigate data usage tools and applications: Understand the data analysis tools, business applications, and reporting systems used within the organization.

[0055] Determine the data sources used by these tools and applications and the structured and semi-structured data they generate.

[0056] Data mapping: Create a data map to visualize the relationships of different data resources within the organization.

[0057] Mark the structured data sources, the sources of semi-structured data, and the storage locations of unstructured data.

[0058] Collaborate with the IT team: Collaborate with the IT team to understand the usage of technical infrastructures such as database management systems (DBMS), data warehouses, and data lakes.

[0059] Understand the tools for data integration and data transmission to determine the path of the data flow.

[0060] Business process analysis: Analyze the organization's business processes and determine the types of data generated at each stage.

[0061] Understand the structured and semi-structured data involved in these business processes.

[0062] Investigate external data sources: Investigate the data that the organization may share with external partners, suppliers, or customers.

[0063] Understand the unstructured data from outside, such as contracts, emails, etc.

[0064] Social data discovery: Utilize the social platforms within the organization to communicate with employees and understand the tools, data resources, and other data storage methods they may use.

[0065] Discover unstructured data that may be overlooked, such as meeting records, discussion forums, etc.

[0066] Document and knowledge base analysis: Analyze the documents and knowledge bases within the organization and understand the structured and semi-structured data contained therein.

[0067] Discover unstructured data such as document comments, notes, and reports.

[0068] Define the attributes, relationships, and business significance of each data resource, including the following steps:

[0069] Data resource classification and naming: Classify the identified data resources, such as customer data, sales data, product data, etc.

[0070] Assign a clear naming standard to each data resource to ensure consistency and readability.

[0071] Attribute definition: Determine the attributes of each data resource, i.e., the specific fields or properties contained in the data, such as name, address, contact information, etc. for customer data.

[0072] Define attributes such as data type, length, format, etc. for each attribute.

[0073] Relationship modeling: Analyze the relationships between different data resources, such as the possible relationships between customer data and sales data.

[0074] Use relationship model tools (such as ER diagrams) to describe the relationships between data resources, including primary keys, foreign keys, etc.

[0075] Business significance and value: Understand the actual significance and value of each data resource in business operations.

[0076] Collaborate with the business team to clarify the contribution of each data resource to business decision-making, operations, and strategic goals.

[0077] Metadata recording: Record the detailed information of each data resource in the metadata management system, including attribute definitions, relationship models, business significance, etc.

[0078] Update the metadata to reflect the changes and evolution of the data resources.

[0079] Data dictionary creation: Create a data dictionary based on the defined attributes and relationships to provide an easy-to-understand document for each data resource.

[0080] The data dictionary should include terms, definitions, examples, and explanations of relationships with other data resources.

[0081] Business rule definition: Determine the business rules related to each data resource, including data validation rules, constraints, etc.

[0082] Integrate the business rules into the definition of the data resources to ensure data consistency and validity.

[0083] Version control: Implement a version control mechanism to track the evolution and change history of the data resources.

[0084] Record the changes in the attributes, relationships, and business meanings of each version to maintain the accuracy of the metadata.

[0085] Data quality specifications: Establish data quality specifications to clarify the quality standards that each data resource should meet.

[0086] Define data quality metrics and monitor the actual quality performance of data resources.

[0087] Socialization and communication: Share the definitions, attributes, and business meanings of data resources with various teams within the organization through means such as socialization platforms, training, and communication meetings.

[0088] Ensure broad understanding and consensus to promote the correct use and management of data resources.

[0089] Record the metadata information of data resources, including data sources, formats, quality standards, business rules, etc., and ensure the consistency and update of the metadata to maintain an accurate description of the data resources, including the following steps:

[0090] Establishment of a metadata warehouse: Set up a centralized metadata warehouse for centralized storage and management of the metadata information of all data resources.

[0091] Ensure that the metadata warehouse can support the storage of information for different types of data resources.

[0092] Standardize the metadata model: Develop and use a standard metadata model to ensure consistent descriptions for each data resource.

[0093] Define the fields included in the metadata model, such as data sources, formats, quality standards, business rules, etc.

[0094] Record the source of each data resource, including which system, application, or data provider generated the data.

[0095] Provide a detailed description of the data source, including version information of the data source.

[0096] Description of data format and structure: Record the format of each data resource, including descriptions of structured, semi-structured, and unstructured data.

[0097] For structured data, define the table structure, field names, and data types; for unstructured data, describe the file format, tags, etc.

[0098] Definition of quality standards: Establish data quality standards, including accuracy, integrity, consistency, timeliness, etc.

[0099] Record data quality metrics and thresholds for monitoring and evaluating data quality.

[0100] Business rule and metadata association: Associate the business rules of each data resource with the metadata to ensure that the implementation of business rules is consistent with the description of the data resource.

[0101] Record information such as conditions and trigger events related to business rules.

[0102] Update strategy and frequency: Develop an update strategy for metadata to clarify when to update the metadata.

[0103] Regularly check and update the metadata, especially when the data source or data structure changes.

[0104] Version control of metadata: Implement a version control mechanism for metadata to record the evolution history of the metadata.

[0105] Record the changes for each version, including newly added, modified, and deleted metadata items.

[0106] Permission control: Set permission control for metadata to ensure that only authorized personnel can access and modify the metadata.

[0107] Assign different levels of permissions to prevent misoperations and protect the security of the metadata.

[0108] Monitoring and reporting: Set up a monitoring mechanism to real-time monitor the changes of metadata and the metadata update situation of data resources.

[0109] Provide regular metadata reports to ensure the accuracy and consistency of data resource descriptions.

[0110] Classify data resources, determine different data levels according to their business uses and values, and divide data into core business data, supporting data, etc., for more targeted management and maintenance, including the following steps:

[0111] Business requirements analysis: Collaborate closely with the business team to understand the requirements and priorities of different business areas.

[0112] Determine the importance of core business data and supporting data for business operations.

[0113] Data value assessment: Evaluate each data resource to determine its contribution degree to business decision-making, operations, and strategic goals.

[0114] Rank according to value to better understand the importance of data.

[0115] Develop classification criteria: Develop clear classification criteria to divide data resources into different categories according to business uses and values.

[0116] Consider the criteria in aspects such as business processes, decision support, and customer relationships.

[0117] Core business data identification: Identify the data resources directly related to the core business processes, which are crucial for the organization's main objectives.

[0118] Determine the data that has a direct impact on business decisions and core processes.

[0119] Supporting data identification: Identify the data resources that, although not directly related to the core business processes, are still crucial for business support and operations.

[0120] Determine the auxiliary role of these data resources in business operations.

[0121] Data level division: Divide the data resources into different levels, such as core business data, supporting data, temporary data, etc.

[0122] Clarify the definition and objectives of each level.

[0123] Formulate level management strategies: Develop corresponding management strategies for each data level, including data quality standards, security requirements, update frequencies, etc.

[0124] Ensure that core business data is subject to more stringent management and monitoring.

[0125] Establish a data level directory: Create a data level directory that clearly lists the data resources at each level and their related attributes.

[0126] Provide detailed documentation, including data dictionaries, business rules, and metadata.

[0127] Assign responsibilities and permissions: Identify the teams or individuals responsible for managing each data level and assign corresponding permissions.

[0128] Ensure that core business data is subject to high-level management and protection.

[0129] Monitor and adjust: Set up monitoring mechanisms to regularly evaluate the management effectiveness of each data level.

[0130] Make adjustments based on business needs and data changes to ensure that the data level division still conforms to the actual situation of the organization.

[0131] Perform data access control based on data access policies to determine who has the right to access, modify, and delete data at different levels, including the following steps:

[0132] Determine data access requirements: Communicate with different teams and roles to understand their access requirements for sales data.

[0133] Differentiate different levels of requirements. For example, the sales team needs real-time access to detailed sales data, while management only needs access to high-level summary data.

[0134] Define user roles: Based on the organizational structure and job responsibilities, define different user roles such as sales representatives, sales managers, senior management, etc.

[0135] Analyze the data levels and scopes that each role needs to access.

[0136] Establish access permissions: Clearly define data access permissions for each user role, including read, modify, and delete permissions.

[0137] Ensure that the permission allocation complies with business requirements and regulatory compliance.

[0138] Data level division: Divide sales data into different levels such as detailed transaction data, customer dimension data, summary reports, etc.

[0139] Clarify the sensitivity and importance of each level.

[0140] Establish access rules: Develop access rules to clarify who has the right to access which levels of data.

[0141] Use rules to distinguish different access scenarios. For example, certain users can only view the customer data they are responsible for.

[0142] Authentication and authorization: Implement an effective authentication mechanism to ensure that only authenticated users can access the system.

[0143] After authentication, perform authorization based on the user's role and access rules to ensure that they can only access the authorized data.

[0144] Auditing and monitoring: Set up auditing and monitoring mechanisms to record users' data access activities.

[0145] Regularly review access logs to ensure that there are no unauthorized access behaviors.

[0146] Use of access control lists (ACLs): Use technical means such as access control lists (ACLs) to perform fine-grained access control on different levels of data resources.

[0147] For each data resource, configure the ACL to specify the users and permissions allowed or denied.

[0148] Regular review and update: Regularly review the data access policy to ensure that it is consistent with the organization's business requirements and regulatory requirements.

[0149] According to the changes in the organization and new business requirements, update the access policy in a timely manner.

[0150] Protect sensitive data based on security measures to avoid sensitive data leakage, including the following steps:

[0151] Sensitive data identification: Determine which data is considered sensitive data, such as customer personal identity information, financial data, etc.

[0152] Classify to clarify which data requires additional protection.

[0153] Data classification tagging: Use data classification tags to add labels or metadata to sensitive data so that the system can identify and process this data.

[0154] For example, mark a certain field as containing personal identity information and thus as sensitive data.

[0155] Encryption algorithm selection: Select a suitable encryption algorithm to ensure that the protection of sensitive data is secure and reliable.

[0156] Consider using symmetric encryption or asymmetric encryption, depending on the specific usage scenario and requirements.

[0157] Key management: Establish a robust key management strategy to ensure that the generation, storage, distribution, and rotation of keys are effectively managed.

[0158] Consider using tools such as Hardware Security Modules (HSM) to enhance the security of keys.

[0159] Encrypted data storage: Encrypt sensitive data in storage to ensure that the data remains secure even if the data storage medium is stolen or leaked.

[0160] Use appropriate encryption libraries or tools to ensure the correct implementation of encryption operations.

[0161] Encrypted data transmission: Use a secure transmission protocol such as TLS / SSL during data transmission to ensure that data is not easily intercepted during transmission.

[0162] Use an encrypted channel to prevent man-in-the-middle attacks and data eavesdropping.

[0163] Access control: Establish an access control mechanism to ensure that only authorized users or systems can decrypt and access sensitive data.

[0164] Use authentication, authorization, and auditing mechanisms to manage access to keys and decryption operations.

[0165] Auditing and monitoring: Set up an auditing and monitoring system to record access to and decryption operations on sensitive data.

[0166] Regularly review audit logs to detect potential security threats and abnormal activities.

[0167] Emergency response plan: Develop an emergency response plan to handle sensitive data breaches or attack incidents.

[0168] Ensure that the team understands how to take prompt measures to minimize potential risks and damages.

[0169] Comprehensively analyze multiple quality parameters of data resources, and screen out data resources with quality defects based on the data quality monitoring mechanism, including the following steps:

[0170] Obtain the consistency pass rate, data latency, duplicate record ratio, and missing value ratio of the data resources;

[0171] Comprehensively calculate the resource assignment by combining the consistency pass rate, data latency, duplicate record ratio, and missing value ratio. The calculation formula is:

[0172]

[0173] In the formula, ZYF is the resource assignment, yzt, syc, cfj, and qsb are the consistency pass rate, data latency, duplicate record ratio, and missing value ratio respectively, and k1, k2, k3, and k4 are the weights of the consistency pass rate, data latency, duplicate record ratio, and missing value ratio respectively, and k1, k2, k3, and k4 are all greater than 0;

[0174] This application obtains the consistency pass rate, data latency, duplicate record ratio, and missing value ratio of data resources, comprehensively calculates the resource assignment by combining the consistency pass rate, data latency, duplicate record ratio, and missing value ratio. After obtaining the resource assignment, compare the resource assignment with the quality threshold, and analyze the quality of the data resources based on the comparison result. The multi-dimensional analysis method is more comprehensive and effectively improves the accuracy of data resource analysis.

[0175] After obtaining the resource assignment, compare the resource assignment with the quality threshold. If the resource assignment is greater than or equal to the quality threshold, analyze that the quality of the data resource is good; if the resource assignment is less than the quality threshold, analyze that the quality of the data resource is poor.

[0176] Consistency pass rate: First, define rules related to consistency, such as the value range of fields, business rules, etc. Use data quality tools or custom scripts to perform consistency checks to check whether the data conforms to the predefined consistency rules. The pass rate is equal to the number of records passing the consistency check divided by the total number of records, multiplied by 100%.

[0177] Data Latency: Add a timestamp to each record in the data source or target system to indicate the time of data generation or update. For each record, calculate the data transfer time (timestamp in the target system minus timestamp in the source system). Calculate the average latency of all records.

[0178] Duplicate Record Ratio: Use data quality tools or custom scripts to detect and identify duplicate records in the data. Divide the number of duplicate records by the total number of records and multiply by 100%.

[0179] Missing Value Ratio: Use data quality tools or custom scripts to detect and identify missing values in the data. Divide the number of missing values by the total number of records and multiply by 100%.

[0180] Manage and supervise data resources that meet quality standards based on data governance methods, including the following steps:

[0181] Formulate Data Governance Policies: Formulate clear data governance policies, specifying the standards and rules for data collection, storage, processing, sharing, and reporting.

[0182] Include data quality standards, privacy protection regulations, data security policies, etc.

[0183] Establish Metadata Management: Set up a metadata management system to record detailed information about data resources, including sources, quality standards, business rules, etc.

[0184] Ensure the timely update of metadata to reflect changes in data resources.

[0185] Data Quality Management: Implement data quality management measures, including data quality monitoring, anomaly detection, and corrective actions.

[0186] Determine data quality metrics and monitor the quality performance of data resources.

[0187] Access and Security Management: Formulate data access and security policies to ensure that only authorized personnel can access, modify, and share sensitive data.

[0188] Set up access control lists (ACLs) and authentication mechanisms.

[0189] Establish a Data Glossary and Data Dictionary: Establish a data glossary to unify the definitions of terms and eliminate data ambiguity.

[0190] Create a data dictionary to record the detailed attributes and definitions of each data resource.

[0191] Specify Data Lifecycle Management: Formulate a data lifecycle management strategy to clarify the retention period, storage location, and destruction mechanism of data.

[0192] Ensure compliance, especially with regard to regulations on data retention and privacy.

[0193] Establish data ownership and responsibility: Clearly define the owner of each data resource and specify their responsibilities for data quality and governance.

[0194] Establish a transparent framework for data responsibilities and obligations.

[0195] Implement data privacy protection: Comply with relevant regulations and laws, implement data privacy protection measures, and ensure that sensitive data is properly processed and protected.

[0196] Set data de - identification and anonymization strategies to reduce privacy risks.

[0197] Monitor and report: Establish a monitoring mechanism to regularly evaluate the effectiveness of data governance strategies.

[0198] Provide regular data governance reports, including data quality, compliance, and governance progress.

[0199] Manage data resources throughout their life cycle in accordance with data life cycle management strategies, including different stages such as data creation, storage, use, archiving, and destruction, to avoid waste of management resources, including the following steps:

[0200] Data creation stage: Requirement definition: In the data creation stage, clarify the purpose of data generation and business requirements.

[0201] Metadata recording: Record the metadata of the data, including information such as data source, format, structure, and creation time.

[0202] Data quality standards: Develop data quality standards to ensure that newly created data meets quality requirements.

[0203] Data storage stage: Storage selection: Select appropriate data storage media and technologies, considering data access frequency, performance requirements, and cost.

[0204] Classification and organization: Classify and organize the data, establish a clear directory structure for easy subsequent management and retrieval.

[0205] Backup strategy: Develop a backup strategy to ensure data security and recoverability.

[0206] Data use stage: Access control: Set access controls to ensure that only authorized users can access and use the data.

[0207] Data sharing: Facilitate appropriate data sharing when needed to ensure that data can provide value for business decision - making and operations.

[0208] Monitor usage: Establish a monitoring mechanism to track data usage, understand which data is frequently accessed and which is less used.

[0209] Data archiving phase: Separation of hot and cold data: Archive data that is no longer frequently used to a more economical storage medium to free up high-performance storage resources.

[0210] Metadata update: Update metadata to mark the data as archived, record the archiving time and location.

[0211] Data archiving strategy: Develop data archiving and retrieval strategies to ensure that the archived data can still be accessed on demand.

[0212] Data destruction phase: Compliance considerations: Before data destruction, ensure that compliance requirements are considered and relevant regulations and policies are followed.

[0213] Complete deletion: For data that is no longer needed, perform a complete deletion to avoid potential security risks.

[0214] Destruction record: Record the process and results of data destruction, including the destruction time, method and responsible person.

[0215] Regular review and adjustment: Regular review of strategies: Regularly review the data life cycle management strategy to ensure that it still meets business needs and regulations.

[0216] Adjustment of strategies: According to business changes, technological progress and compliance requirements, timely adjust the data life cycle management strategy.

[0217] Set up data integration interfaces to enable effective integration of data with other systems (such as marketing, sales and customer service), and complete data transmission management, including the following steps:

[0218] Requirement analysis and planning: Define business requirements: Collaborate with relevant departments and stakeholders to understand the specific business requirements and goals of data integration.

[0219] System and data source identification: Identify the systems and data sources to be integrated, including marketing, sales, customer service, etc.

[0220] Select an appropriate integration mode: Point-to-point integration: Directly connect two systems, suitable for simple integration requirements.

[0221] Central integration: Achieve data integration through a central platform to improve flexibility and maintainability between systems.

[0222] Asynchronous or synchronous integration: Select data synchronization or asynchronous transmission methods according to business requirements.

[0223] Formulate data mapping and transformation rules: Field mapping: Clearly define the field mapping rules for the same data in different systems to ensure the correct transmission and interpretation of data.

[0224] Data transformation: According to the requirements of the target system, formulate transformation rules for data formats, types, and structures.

[0225] Interface design and development: API design: If using an API (Application Programming Interface), design a clear interface and define input and output parameters.

[0226] Development and testing: Develop a data integration interface and conduct sufficient testing to ensure that the interface can work stably and efficiently.

[0227] Security and permission control: Authentication and authorization: Implement effective authentication and authorization mechanisms in the interface to ensure that only legitimate users can access the interface.

[0228] Data encryption: For sensitive data, use encryption technology to ensure security during transmission.

[0229] Monitoring and logging: Performance monitoring: Set up a monitoring system to monitor the interface performance in real-time to ensure the efficiency and timeliness of data transmission.

[0230] Logging: Record the activity logs of the interface for problem tracking, auditing, and review.

[0231] Exception handling and fault tolerance mechanism: Exception handling: Design a reasonable exception handling mechanism to capture and handle exceptions during the data integration process.

[0232] Fault tolerance mechanism: Consider possible failure situations of the interface and formulate a fault tolerance strategy to ensure the stability of the system.

[0233] Version control: Interface version: Implement version control for the interface to ensure that changes to the system and data model do not disrupt the integration.

[0234] Upgrade strategy: Formulate an upgrade strategy to ensure seamless integration of changes to the system and data model.

[0235] Documentation and training: Documentation writing: Record detailed information such as interface configuration, parameters, and mapping rules for future maintenance and use.

[0236] Training the team: Provide training to relevant teams so that they can effectively use and maintain the data integration interface.

[0237] Implementation and continuous improvement: Gradual implementation: Gradually implement the data integration interface to ensure stability during actual operation.

[0238] Continuous improvement: Regularly review and optimize the data integration process, and make continuous improvements according to actual requirements.

[0239] Embodiment 3: A data resource management system based on logical management described in this embodiment includes an identification module, a definition and recording module, a classification module, a security module, a quality analysis module, a management module, and an integrated transmission module;

[0240] Identification module: Identify various data resources within the organization, including structured, semi-structured, and unstructured data;

[0241] Definition and recording module: Define the attributes, relationships, and business meanings of each data resource, record the metadata information of the data resources, including data sources, formats, quality standards, business rules, etc., and ensure the consistency and update of the metadata to maintain an accurate description of the data resources;

[0242] Classification module: Classify the data resources, determine different data levels according to their business uses and values, and divide the data into core business data, supporting data, etc., so as to manage and maintain them more pertinently;

[0243] Security module: Perform data access control based on data access policies, determine who has the right to access, modify, and delete data at different levels, and protect sensitive data based on security measures to avoid leakage of sensitive data;

[0244] Quality analysis module: Comprehensively analyze multiple quality parameters of the data resources, and screen out the data resources with quality defects based on the data quality monitoring mechanism;

[0245] Management module: Manage and supervise the data resources that meet the quality standards based on data governance methods, and according to the data life cycle management strategy, including different stages such as the creation, storage, use, archiving, and destruction of data, manage the data resources within the life cycle to avoid waste of management resources;

[0246] Integrated transmission module: Set up a data integration interface to enable effective integration of data with other systems (such as marketing, sales, and customer service), and complete the transmission management of the data.

[0247] The above formulas are all dimensionless and take their numerical values for calculation. The formula is obtained by collecting a large amount of data for software simulation to get a formula that is closest to the actual situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation.

[0248] In the description of this specification, the descriptions referring to the terms "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0249] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to only the specific implementation manners. Obviously, according to the content of this specification, many modifications and changes can be made. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the relevant technical fields can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A data resource management method based on logic management, characterized in that: The management method comprises the following steps: Identify various data resources within the organization, including structured, semi-structured, and unstructured data, define the attributes, relationships, and business significance of each data resource, and record metadata information of data resources, including data sources, formats, quality standards, and business rules; Classify data resources, determine different data levels according to their business purpose and value, divide data into core business data and supporting data, perform data access control based on data access policies, and protect sensitive data based on security measures; Comprehensively analyze multiple quality parameters of data resources and screen out data resources with quality defects based on the data quality monitoring mechanism; Data resources that meet quality standards are managed and supervised based on data governance methods, and data resources are managed throughout their life cycle according to data life cycle management strategies, including different stages of data creation, storage, use, archiving and destruction; Establish a data integration interface to integrate data with other systems and complete data transmission management.

2. A data resource management method based on logic management according to claim 1, characterized in that: Obtain the consistency pass rate, data delay, duplicate record ratio, and missing value ratio of data resources; The consistency pass rate, data delay, duplicate record ratio and missing value ratio are calculated comprehensively to obtain the resource assignment. The calculation expression is: Where ZYF is the resource value, yzt, syc, cfj, qsb are the consistency pass rate, data delay, duplicate record ratio and missing value ratio, respectively, k1, k2, k3, k4 are the weights of the consistency pass rate, data delay, duplicate record ratio and missing value ratio, respectively, and k1, k2, k3, k4 are all greater than 0; After obtaining the resource assignment, the resource assignment is compared with the quality threshold. If the resource assignment is greater than or equal to the quality threshold, the quality of the analysis data resource is good. If the resource assignment is less than the quality threshold, the quality of the analysis data resource is poor.

3. A data resource management method based on logic management according to claim 2, characterized in that: Define the attributes, relationships, and business significance of each data resource, including the following steps: Classify the identified data resources, including customer data, sales data, and product data, assign naming standards to each data resource, determine the attributes of each data resource, the specific fields or attributes contained in the data, including the name, address, and contact information of the customer data, define the data type, length, and format attributes for each attribute, use relational model tools to describe the relationship between data resources, including primary keys and foreign keys, record detailed information for each data resource in the metadata management system, including attribute definitions, relational models, and business significance, create a data dictionary based on the defined attributes and relationships, and provide documentation for each data resource. The data dictionary includes terms, definitions, examples, and descriptions of relationships with other data resources.

4. The data resource management method based on logic management according to claim 3, characterized in that: Recording metadata information of data resources, including data sources, formats, quality standards, and business rules, includes the following steps: Establish a centralized metadata warehouse to centrally store and manage metadata information of all data resources, define the fields included in the metadata model, including data source, format, quality standards, and business rules, record the source of each data resource, including which system, application, or data provider generated the data, and record the format of each data resource, including descriptions of structured, semi-structured, and unstructured data. For structured data, define the table structure, field name, and data type; for unstructured data, describe the file format and tags.

5. A data resource management method based on logic management according to claim 4, characterized in that: Classify data resources, determine different data levels according to their business purpose and value, and divide data into core business data and supporting data, including the following steps: Evaluate each data resource, determine the extent to which the data resource contributes to business decisions, operations and strategic goals, rank them according to value, divide data resources into different categories according to business purpose and value, identify data resources directly related to core business business processes, and identify data resources that are not directly related to core business processes but are related to business support and operations, divide data resources into different levels, and develop corresponding management strategies for each data level, including data quality standards, security requirements, and update frequency.

6. A data resource management method based on logic management according to claim 5, characterized in that: Data access control based on data access policy includes the following steps: Define different user roles, such as sales representative, sales manager, and senior management, based on organizational structure and job responsibilities. Analyze the data level and scope that each role needs to access. Clearly define data access permissions for each user role, including read, modify, and delete permissions. Divide sales data into different levels, use rules to distinguish different access scenarios, implement an identity authentication mechanism, and after identity authentication, authorize based on the user's role and access rules.

7. A data resource management method based on logic management according to claim 6, characterized in that: Protect sensitive data based on security measures to avoid sensitive data leakage, including the following steps: Sensitive data includes customer personal identity information, financial data, etc. Use data classification tags to add labels or metadata to sensitive data, protect sensitive data through encryption algorithms, establish key management strategies to ensure that key generation, storage, distribution and rotation are managed, encrypt sensitive data in storage, and use secure transmission protocols during data transmission.

8. The data resource management method based on logic management according to claim 7, characterized in that: According to the data lifecycle management strategy, including the different stages of data creation, storage, use, archiving and destruction, data resources are managed during their lifecycle, including the following steps: During the data creation phase, clarify the purpose of data generation and business needs, record the data metadata, including data source, format, structure and creation time, archive data that is no longer frequently used to general storage media, update metadata, mark data as archived, record archiving time and location, formulate data archiving and retrieval strategies, ensure archived data can be accessed on demand, completely delete data that is no longer needed, record the process and results of data destruction, including the time, method and person responsible for destruction, and regularly review data lifecycle management strategies to ensure compliance with business needs and regulations.

9. A data resource management system based on logic management, used to implement the management method according to any one of claims 1 to 8, characterized in that: It includes identification module, definition and recording module, classification module, security module, quality analysis module, management module and integrated transmission module; Identification module: Identify various data resources within the organization, including structured, semi-structured and unstructured data; Definition and recording module: defines the attributes, relationships and business significance of each data resource, and records the metadata information of the data resource, including data source, format, quality standard and business rules; Classification module: classifies data resources, determines different data levels according to their business purpose and value, and divides data into core business data and supporting data; Security module: performs data access control based on data access policies and protects sensitive data based on security measures; Quality analysis module: Comprehensively analyzes multiple quality parameters of data resources and screens out data resources with quality defects based on the data quality monitoring mechanism; Management module: manages and supervises data resources that meet quality standards based on data governance methods, and manages data resources throughout their life cycle according to data life cycle management strategies, including different stages of data creation, storage, use, archiving and destruction; Integrated transmission module: establish a data integration interface to integrate data with other systems and complete data transmission management.

Images