Complex scene-oriented special field data resource directory construction method

Abstract

The invention relates to a special field data resource directory construction method for a complex scene, and belongs to the field of data fusion treatment. According to the method, all data assets in an organization are systematically identified, classified, described, marked and organized to form a data asset directory; a data resource system framework template is designed, integration of different fields and different levels of data resource frameworks is achieved through cross-subject data resource summarization and aggregation, data resource merging, duplicate removal and overall integration and through category dimensions and the data resource system framework template, and the category dimension support is expanded and created according to multiple business fields to be integrated. The data resource system framework template supports integration of data resource frameworks of different fields and different hierarchies based on a unified framework standard of the system, and finally the data resource frameworks of different fields, different hierarchies and different systems are integrated into a globally unified and standard data resource framework system in the system.

Description

Technical Field

[0001] The present invention belongs to the field of data fusion management, and specifically relates to a method for constructing a proprietary domain data resource directory for complex scenarios. Background Art

[0002] The data resource catalog is designed to help data analysts discover and understand data, and is the core of data management, data accountability, and data governance. The data resource catalog promotes the transformation from centralized, IT-centric data management to data management with extensive business collaboration.

[0003] There are usually two ways to build a data resource directory for a specific domain: one is the data resource classification method for information system resource management, which takes the system as the sorting object and establishes a data resource classification. The other is the data resource classification method for business matters, which builds a classification directory according to basic data resources, departmental data resources, and subject data resources.

[0004] In general, the above data resource directory construction method is mainly based on the existing system or business architecture, and the top-level design of the data resource system and its directory is carried out from top to bottom. The classification method is very intuitive, easy for data resource management and business understanding, and conducive to data responsibility and sharing and exchange. However, for complex scenario data applications across departments, levels, and fields, the business application matching degree of such data resource directories is low, and the requirements for secondary data organization and development are high.

[0005] In response to the above problems, the present invention proposes a method for constructing a proprietary domain data resource directory for complex scenarios. Driven by typical complex scenario data applications, various application-oriented data resource directories are integrated into a framework from the bottom up, and combined with the top-down data resource directory classification design based on the business architecture to form a comprehensive and global data resource directory, which promotes the data resource directory to have a higher application matching degree when serving complex business scenarios across domains, departments, and levels. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] The technical problem to be solved by the present invention is how to provide a method for constructing a proprietary domain data resource directory for complex scenarios, so as to solve the problem that the method for constructing a data resource directory is limited by the existing business architecture or system architecture, and the matching degree of business applications for complex scenarios is low; to solve the problem of effective identification, extraction, grading and classification, and access control of data resources in complex business scenarios; to solve the problem of integrating cross-domain, cross-department, and cross-level data resource frameworks in complex business scenarios.

[0008] (II) Technical solution

[0009] In order to solve the above technical problems, the present invention proposes a method for constructing a proprietary domain data resource directory for complex scenarios, which comprises the following steps:

[0010] S1. Cataloging of activity data assets

[0011] Systematically identify, classify, describe, label and organize all data assets within the organization to form a structured, easy-to-query and understand data asset catalog; thematic activity data resource cataloging includes three steps: data resource identification and extraction, data resource analysis, and data resource compilation;

[0012] S2. Integration of cross-theme data resource framework

[0013] Design a data resource system framework template, integrate data resource frameworks in different fields and at different levels through cross-theme data resource aggregation, data resource merging, deduplication and overall integration, and realize the integration of data resource frameworks in different fields and at different levels through category dimensions and data resource system framework templates. Category dimensions are expanded and created based on the multiple business fields to be integrated. The data resource system framework template integrates data resource frameworks in different fields and at different levels based on the system's unified framework standards, and ultimately realizes the integration of data resource frameworks in different fields, at different levels and in different systems into a globally unified and standard data resource framework system; cross-theme data resource framework integration includes: building a data resource system framework template, cross-theme data resource aggregation, and data resource merging, deduplication and overall integration steps.

[0014] (III) Beneficial effects

[0015] The present invention proposes a method for constructing a proprietary domain data resource directory for complex scenarios. The present invention discloses a method for constructing a proprietary domain data resource directory for complex scenarios, and its main advantages are embodied in the following aspects:

[0016] (1) A bottom-up data resource framework integrated directory construction method based on service object application was designed. Driven by the actual application of typical complex scenario data, several thematic activity data applications with complex business structures, wide department coverage, and multiple levels were selected for standardized cataloging. On this basis, the data resource directories of each subject were aggregated, merged, deduplicated, and integrated to form a unified data resource directory for a specific field, providing unified data resource services for all levels, systems, and departments.

[0017] (2) A processing model for the integrated construction of data resource catalogs was designed to form a complete set of processes from data resource cataloging to data resource framework integration. A comprehensive data resource identification and extraction method was used in catalog compilation, and data classification, grading, encryption, annotation and access control were performed based on factors such as the nature, sensitivity, and application value of the data. A data resource system framework template was used in data resource framework integration to integrate cross-departmental and cross-level data resource frameworks based on unified framework standards, to achieve a globally unified, standard and normative data resource catalog system, which is conducive to clarifying proprietary data resources, studying data application requirements, and optimizing data construction methods and processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall framework diagram of the present invention;

[0019] Figure 2 Flow chart for cataloguing data resources for thematic activities;

[0020] Figure 3 Integrate a flow chart for a cross-thematic data resource framework. DETAILED DESCRIPTION

[0021] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0022] 1. Overall framework of this method

[0023] like Figure 1 As shown, the present invention provides a method for constructing a proprietary domain data resource directory for complex scenarios, mainly focusing on large-scale complex activities in the proprietary domain with large-scale organizations, complex business structures, wide coverage of departments, and multiple levels as the research objects and data application needs, to study the method of constructing and integrating the data resource directory. Focus on selecting typical subject application scenarios such as earthquake relief, network attack and defense, and on the basis of data collection, data governance, and data integration of each subject, perform data resource identification and extraction, catalog compilation, and carry out cross-theme data resource framework aggregation, merging and deduplication, and overall integration to form a unified data resource directory for the proprietary domain, and provide unified data services for various major subject activities, daily business systems, data sharing and exchange, etc. in this proprietary domain. Specifically including the following steps:

[0024] S1. Cataloging of activity data assets

[0025] Systematically identify, classify, describe, label and organize all data assets within the organization to form a structured, easy-to-query and understand data asset catalog, improve the visibility, comprehensibility, accessibility and manageability of data assets, and lay the foundation for the effective use, compliance management and value enhancement of data assets; Thematic activity data resource cataloging mainly includes three steps: data resource identification and extraction, data resource analysis, and data resource compilation;

[0026] S2. Integration of cross-theme data resource framework

[0027] Design a data resource system framework template, integrate data resource frameworks in different fields and at different levels through cross-theme data resource aggregation, data resource merging and deduplication, and overall integration, and realize the integration of data resource frameworks in different fields and at different levels through category dimensions and data resource system framework templates. Category dimensions support expansion and creation based on multiple business fields to be integrated. The data resource system framework template supports the integration of data resource frameworks in different fields and at different levels based on the unified framework standard of the system, and finally realizes the integration of data resource frameworks in different fields, at different levels, and in different systems into a globally unified and standard data resource framework system in this system; including: building a data resource system framework template, cross-theme data resource aggregation, data resource merging and deduplication, and overall integration steps;

[0028] 2. Cataloging of theme activity data resources

[0029] Data asset cataloging aims to systematically identify, classify, describe, label and organize all data assets within an organization, forming a structured, easy-to-query and understand data asset catalog, improving the visibility, comprehensibility, accessibility and manageability of data assets, and laying the foundation for the effective use, compliance management and value enhancement of data assets. Thematic activity data resource cataloging mainly includes three links: data resource identification and extraction, data resource analysis, and data resource compilation. Figure 2 shown.

[0030] S1.1 Data resource identification and extraction

[0031] In the stage of cataloguing data resources for thematic activities, we first start from typical thematic application scenarios and analyze in detail the specific needs of various business application scenarios for data resources. This process includes three steps:

[0032] (1) Business requirements analysis

[0033] We went to the scene of typical theme activities such as earthquake relief and network attack and defense, sorted out the actual business processes in detail, recorded data interactions, and combined with in-depth interviews with front-line staff and business experts to fully understand the data types, data sources and data application methods used in the work. At the same time, we collected and analyzed business documents, operation manuals and data reports related to the theme activities to extract key business information.

[0034] (2) Data Requirements Identification

[0035] On the basis of clarifying the business process and actual needs, conduct an in-depth analysis of the data types, formats, and sources required for each business link. Specifically, based on the business process and operation steps, determine the data types that each business link relies on, such as geographic information, personnel information, equipment status, etc. Analyze the storage format of the data, including structured data (such as database tables), semi-structured data (such as XML, JSON), and unstructured data (such as text, images, videos). At the same time, clarify the way to obtain data, which may come from internal systems, external cooperative organizations, or sensor devices. Through comprehensive identification of data needs, ensure that subsequent data collection and processing have clear goals and directions.

[0036] (3) Correlation analysis

[0037] Based on the identification of data requirements, the correlation and dependency between different data are further analyzed.

[0038] First, clarify the position and role of each data item in the business process, and analyze the logical relationship and interaction between the data.

[0039] Secondly, draw a data flow diagram to show the flow path of data between different departments and systems, and identify the key nodes for data sharing and exchange.

[0040] Finally, analyze the dependencies and constraints of the data, clarify the data update mechanism and consistency requirements, and use the association rule algorithm to measure the association rules through support, confidence and lift to discover the association relationship between the data.

[0041] Support: It indicates the probability of item sets X and Y appearing at the same time, and measures the universality of the rule. Rules with low support are usually filtered out. The formula is as follows:

[0042]

[0043] Count(X∪Y): the number of times item sets X and Y appear simultaneously;

[0044] N: total number of transactions.

[0045] Confidence: It indicates the probability that a transaction also contains Y under the condition that the transaction contains X. It measures the reliability of the rule. The higher the confidence, the more credible the rule. The formula is as follows:

[0046]

[0047] Count(X): the number of occurrences of item set X;

[0048] Lift: Measures whether X and Y are truly related, defined as:

[0049]

[0050] Support(Y): The probability of item set Y appearing.

[0051] Its meaning is:

[0052] Lift>1: Positive correlation, X and Y are more likely to occur at the same time;

[0053] Lift = 1: no correlation, X and Y are independent;

[0054] Lift<1: negative correlation, the occurrence of X inhibits the occurrence of Y.

[0055] Through relationship analysis, we lay the foundation for data integration and unification, ensuring the coordination and consistency of data resources on a global scale.

[0056] S1.2 Data resource analysis

[0057] Identify and extract data resources based on in-depth analysis of data application requirements. Specifically, the following steps are included:

[0058] (1) Systematic data collection

[0059] Before identifying data resources, systematically collect relevant data from each theme activity, and collect all data resources related to each theme activity based on the results of the previous demand analysis. Use ETL, DATAX and other tools to extract required data from various business systems and databases to ensure data integrity and accuracy. For data from different sources, perform unified format conversion and encoding processing to solve the problem of data heterogeneity. Through data backup, ensure that no data is lost or damaged during the data extraction process.

[0060] (2) Metadata Extraction

[0061] Use data extraction tools to extract data that meets the standards and generate corresponding metadata description files for each data resource. Metadata descriptions include basic attributes (such as name, type, format, size and source), business descriptions (such as purpose, applicable scenarios and related business processes), technical attributes (such as storage location, data model and interface information) and security attributes (such as confidentiality level, access control policy and encryption method), etc., to provide detailed information support for the subsequent management and use of data resources.

[0062] (3) Standardized data governance

[0063] Standardize data governance for the collected data to improve data quality. First, clean the data to handle errors, missing values, and duplicate values. Second, standardize the data to unify the naming rules, units, and date formats of the data to ensure data consistency. Finally, establish data verification rules to verify the accuracy and completeness of the data to ensure that the data meets business and technical specifications.

[0064] (4) Data resource classification

[0065] Scientifically classify and grade data resources based on the nature, business relevance, sensitivity, and application value of the data. In terms of classification, data is divided into categories such as basic data, business data, and subject data to facilitate application in different business scenarios. In terms of classification, data is divided into public, internal, and confidential levels based on the sensitivity and importance of the data, and the security control requirements of the data are clarified. Through classification and grading, clear guidance is provided for the management and use of data.

[0066] S1.3 Compilation of data resource catalogue

[0067] After completing the data resource identification, we will start the catalog compilation work and conduct permission control to ensure that the data resources are safe and controllable. Specifically, we will:

[0068] (1) Directory structure design

[0069] According to the classification and grading of data resources and the needs of thematic activities, a directory system with clear hierarchy and reasonable structure is designed to ensure that data resources have a clear organizational form both physically and logically. A tree structure is used to organize data resources according to categories, themes, and business dimensions, which is convenient for users to quickly find and locate. A unified naming specification is formulated to ensure the standardization and consistency of directory names. At the same time, data resources are annotated with tags and keywords to improve the efficiency of data retrieval. The directory structure usually includes basic data layer, department data layer, theme data layer, etc., to ensure that data at different levels can be effectively managed.

[0070] (2) Permission Control

[0071] Assign appropriate data access rights according to user functions and responsibilities. Set strict access control policies for sensitive data, including authority authentication, access auditing, and abnormal behavior monitoring. For confidential data, adopt encrypted storage, transmission, and access methods to prevent data leakage and illegal access.

[0072] (3) Catalog publishing and maintenance

[0073] All compiled data resource directories need to complete permission authentication and registration for subsequent auditing and access tracing. By registering the access logs of each data resource, the use of data can be accurately traced to ensure the security and compliance of data resources throughout the life cycle. The compiled data resource directory will be registered and published on a unified platform to facilitate query and use by various departments and users. Establish a version management mechanism for the directory, record the change history of the directory, and ensure the traceability of the directory. Regularly update and maintain the directory to promptly reflect changes in business and data and maintain the timeliness and accuracy of the directory.

[0074] 3. Integration of cross-theme data resource framework

[0075] Design a data resource system framework template, and integrate data resources in different fields and at different levels through cross-theme data resource aggregation, data resource merging, deduplication and overall integration. The data resource frameworks in different fields and at different levels can be integrated through category dimensions and data resource system framework templates. The category dimension supports expansion and creation based on multiple business fields to be integrated. The data resource system framework template supports the integration of data resource frameworks in different fields and at different levels based on the unified framework standards of the system, and ultimately realizes the integration of data resource frameworks in different fields, at different levels and in different systems into a globally unified and standard data resource framework system in this system. Figure 3 shown.

[0076] S2.1 Build a data resource system framework template

[0077] In order to improve the standardization and automation level of data resource framework integration, an integration template based on the data resource system framework standard is designed:

[0078] (1) Building integration standards

[0079] Based on the unified framework standard, a data resource system framework template is designed. This template specifies the organization of data resources, metadata description requirements, permission control strategies, etc., and defines in detail the standards for cross-departmental and cross-level data integration, including data format specifications, data field standards, data transmission protocols, etc., to ensure that data can be smoothly integrated and meet the requirements of the overall architecture.

[0080] (2) Integrated template application

[0081] In the specific data resource integration process, the integration template can provide structured guidance to ensure that the data collection, processing, integration and release are in compliance with the specifications. The application of templates not only improves the integration efficiency, but also greatly reduces the workload of manual participation, making data integration more automated and systematic.

[0082] (3) Directory publishing and sharing

[0083] The integrated data resource directory is published to a unified data sharing platform, providing services such as data query, retrieval, download and subscription. Through perfect permission control and security measures, the safe sharing and compliant use of data resources are ensured to support the business application needs of various departments and systems.

[0084] S2.2 Aggregation of cross-theme data resources

[0085] After the data resources of each theme activity are catalogued, the cross-theme data resource aggregation phase begins. The main purpose is to standardize and organize similar data resources in each theme activity. Specifically, it includes:

[0086] (1) Data attribute consistency check

[0087] Before data aggregation, we first conduct a property consistency check on the data resources in each subject activity to ensure that data with similar attributes are consistent in format and field definition. This includes consistency processing of data field names, field types, data units, etc., for subsequent aggregation and integration.

[0088] (2) Standardization of data resources

[0089] For similar data resources of different subjects, standardization is performed. For data resources of different departments, if they are essentially the same type of data (such as "population information"), their data fields, data formats, etc. need to be unified to form a unified standard data description.

[0090] (3) Cross-topic resource aggregation

[0091] Based on business needs and application scenarios, cross-theme data resources with the same or similar attributes are aggregated and integrated to form a cross-theme comprehensive data directory. Aggregation strategies include data merging, data splicing, and data alignment to ensure that the final data resource directory can efficiently support applications in different business scenarios.

[0092] S2.3 Data resource merging, deduplication and overall integration

[0093] After aggregation, the data resources of each subject need to be merged and deduplicated to ensure the high quality and accuracy of the data directory:

[0094] (1) Duplicate data identification

[0095] Using data analysis and matching algorithms, the aggregated data resources are compared to identify duplicate or similar data items in different subject activities. The comparison process takes into account the name, attributes, content and business meaning of the data, and uses algorithms such as field matching-based duplicate detection and content-based similarity calculation to ensure the accuracy of duplicate identification.

[0096] (2) Data Merger

[0097] Through semantic-based similarity calculation and natural language processing technology, similarity is calculated for field names or descriptions through vector words, and the identified duplicate data items are merged.

[0098]

[0099] in, Vector representation of fields or directory items. Fields with high similarity are considered to be the same item.

[0100] Determine the data selection and merging strategy based on the completeness, accuracy and authority of the data. Formulate resolution rules for data conflicts, such as taking the latest data as the standard or giving priority to data from a specific source, to ensure that the merged data is correct and reliable.

[0101] (3) Overall integration

[0102] After the data is merged, redundant and invalid data is cleaned up, and the data resources of various theme activities are integrated to form a global comprehensive data resource directory. The overall integration follows a unified data resource framework standard to ensure that various types of data can coexist in the global directory and provide unified data resource services for the business applications of various departments and systems.

[0103] The present invention discloses a method for constructing a proprietary domain data resource directory for complex scenarios, the main advantages of which are reflected in the following aspects:

[0104] (1) A bottom-up data resource framework integrated directory construction method based on service object application was designed. Driven by the actual application of typical complex scenario data, several thematic activity data applications with complex business structures, wide department coverage, and multiple levels were selected for standardized cataloging. On this basis, the data resource directories of each subject were aggregated, merged, deduplicated, and integrated to form a unified data resource directory for a specific field, providing unified data resource services for all levels, systems, and departments.

[0105] (2) A processing model for the integrated construction of data resource catalogs was designed to form a complete set of processes from data resource cataloging to data resource framework integration. A comprehensive data resource identification and extraction method was used in catalog compilation, and data classification, grading, encryption, annotation and access control were performed based on factors such as the nature, sensitivity, and application value of the data. A data resource system framework template was used in data resource framework integration to integrate cross-departmental and cross-level data resource frameworks based on unified framework standards, to achieve a globally unified, standard and normative data resource catalog system, which is conducive to clarifying proprietary data resources, studying data application requirements, and optimizing data construction methods and processes.

[0106] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for constructing a proprietary domain data resource directory for complex scenarios, characterized in that: The method comprises the following steps: S1. Cataloging of activity data assets Systematically identify, classify, describe, label and organize all data assets within the organization to form a structured, easy-to-search and understand data asset catalog; The cataloging of data resources for thematic activities includes three steps: data resource identification and extraction, data resource analysis, and data resource compilation; S2. Integration of cross-theme data resource framework Design a data resource system framework template, integrate data resource frameworks in different fields and at different levels through cross-theme data resource aggregation, data resource merging, deduplication and overall integration, and realize the integration of data resource frameworks in different fields and at different levels through category dimensions and data resource system framework templates. Category dimensions are expanded and created based on the multiple business fields to be integrated. The data resource system framework template integrates data resource frameworks in different fields and at different levels based on the system's unified framework standards, and ultimately realizes the integration of data resource frameworks in different fields, at different levels and in different systems into a globally unified and standard data resource framework system; cross-theme data resource framework integration includes: building a data resource system framework template, cross-theme data resource aggregation, and data resource merging, deduplication and overall integration steps.

2. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 1, characterized in that: The data resource identification and extraction includes: Business needs analysis: Go to the theme event site, analyze the actual business process in detail, record data interactions, and conduct in-depth interviews with front-line staff and business experts to fully understand the data types, data sources, and data application methods used in the work; at the same time, collect and analyze business documents, operation manuals, and data reports related to the theme event to extract key business information; Data demand identification: Based on the clarification of business processes and actual needs, in-depth analysis of the data types, formats and sources required by each business link; based on business processes and operating steps, determine the data types that each business link relies on; analyze the storage format of data; and clarify the way to obtain data; Association relationship analysis: Based on the identification of data requirements, further analyze the association and dependency between different data; first, clarify the position and role of each data item in the business process, and analyze the logical relationship and interaction mode between data; second, draw a data flow diagram to show the flow path of data between different departments and systems, and identify the key nodes of data sharing and exchange; finally, analyze the dependency and constraints of data, clarify the data update mechanism and consistency requirements, use the association rule algorithm, measure the association rules through support, confidence and lift, and discover the association relationship between data.

3. The method for constructing a complex scenario-oriented proprietary domain data resource directory as claimed in claim 2, characterized in that: Support: It indicates the probability of item sets X and Y appearing at the same time, and measures the universality of the rule. Rules with low support are usually filtered out. The formula is as follows: Count(X∪Y): the number of times item sets X and Y appear simultaneously; N: total number of transactions; Confidence: It indicates the probability that a transaction also contains Y under the condition that the transaction contains X. It measures the reliability of the rule. The higher the confidence, the more credible the rule. The formula is as follows: Count(X): the number of occurrences of item set X; Lift: Measures whether X and Y are truly related, defined as: Support(Y): the probability of item set Y appearing; Its meaning is: Lift>1: Positive correlation, X and Y are more likely to occur at the same time; Lift = 1: no correlation, X and Y are independent; Lift<1: negative correlation, the occurrence of X inhibits the occurrence of Y.

4. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 2, characterized in that: The data resource analysis includes: Systematic data collection: Before identifying data resources, systematically collect relevant data from various theme activities, and collect all data resources related to each theme activity based on the previous demand analysis results; use ETL and DATAX tools to extract required data from various business systems and databases, and perform unified format conversion and coding processing for data from different sources; Metadata extraction: Use data extraction tools to extract data that meets the standards and generate corresponding metadata description files for each data resource; metadata descriptions include basic attributes, business descriptions, technical attributes, and security attributes; Standardized data governance: Standardized data governance is performed on the collected data. First, the data is cleaned to handle errors, missing values, and duplicate values ​​in the data. Second, data standardization is performed to unify the naming rules, units, and date formats of the data to ensure data consistency. Finally, data verification rules are established to verify the accuracy and completeness of the data to ensure that the data complies with business and technical specifications. Data resource grading and classification: Scientifically classify and grade data resources based on the nature, business relevance, sensitivity and application value of the data; in terms of classification, divide data into basic data, business data and subject data categories; in terms of grading, divide data into public, internal and confidential levels based on their sensitivity and importance, and clarify data security control requirements.

5. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 4, characterized in that: The data resource directory compilation includes: Directory structure design: Based on the classification and grading of data resources and the needs of theme activities, a directory system with clear hierarchy and reasonable structure is designed to ensure that data resources have a clear organizational form both physically and logically; a tree structure is used to organize data resources according to categories, themes and business dimensions to facilitate users to quickly find and locate; a unified naming specification is formulated to ensure the standardization and consistency of directory names; at the same time, tags and keywords are used to annotate data resources; the directory structure includes basic data layer, department data layer, and theme data layer to ensure that data at different levels can be effectively managed; Permission control: According to the functions and responsibilities of users, corresponding data access rights are assigned, and strict access control policies are set for sensitive data, including permission authentication, access auditing and abnormal behavior monitoring measures; for confidential data, encrypted storage, transmission and access are adopted; Catalog publishing and maintenance: All compiled data resource catalogs need to complete permission authentication and registration to facilitate subsequent audits and access tracing; by registering the access logs of each data resource, the use of data can be accurately traced to ensure the security and compliance of data resources throughout their life cycle; the compiled data resource catalog will be registered and published on a unified platform to facilitate query and use by various departments and users; a version management mechanism for the catalog will be established to record the change history of the catalog to ensure the traceability of the catalog; the catalog will be updated and maintained regularly to promptly reflect changes in business and data and maintain the timeliness and accuracy of the catalog.

6. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 5, characterized in that: The data resource system framework template includes: Build integration standards: Based on unified framework standards, design a data resource system framework template; this template specifies the organization of data resources, metadata description requirements, and permission control policy content, and defines in detail the standards for cross-departmental and cross-level data integration, including data format specifications, data field standards, and data transmission protocols, to ensure that data can be smoothly aggregated during integration and meet the requirements of the overall architecture; Integration template application: In the process of integrating specific data resources, the integration template can provide structured guidance to ensure that data collection, processing, integration and release are in compliance with the specifications; Catalog publishing and sharing: The integrated data resource catalog is published to a unified data sharing platform, providing data query, retrieval, download and subscription services. Through comprehensive permission control and security measures, it ensures the safe sharing and compliant use of data resources, and supports the business application needs of various departments and systems.

7. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 6, characterized in that: The cross-theme data resource aggregation includes: Data attribute consistency check: Before data aggregation, first conduct an attribute consistency check on the data resources in each subject activity to ensure that data with similar attributes are consistent in format and field definition; including consistency processing of data field names, field types, and data units; Standardization of data resources: For similar data resources of different subjects, standardization is carried out. For data resources of different departments, if they are essentially the same data, their data fields and data formats need to be unified to form a unified standard data description; Cross-theme resource aggregation: Based on business needs and application scenarios, cross-theme data resources with the same or similar attributes are aggregated and integrated to form a cross-theme comprehensive data directory. The aggregation strategy includes data merging, data splicing and data alignment to ensure that the final data resource directory can efficiently support applications in different business scenarios.

8. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 7, characterized in that: The data resource merging, deduplication and overall integration include: Duplicate data identification: Use data analysis and matching algorithms to compare aggregated data resources and identify duplicate or similar data items in different subject activities. The comparison process takes into account the name, attributes, content and business meaning of the data, and uses field matching-based duplicate detection and content-based similarity calculation algorithms to ensure the accuracy of duplicate identification. Data merging: Through semantic-based similarity calculation, natural language processing technology is used to calculate the similarity of field names or descriptions through vector words, and the identified duplicate data items are merged; Overall integration: After the data is merged, the redundant and invalid data is cleaned up, and the data resources of various thematic activities are comprehensively integrated to form a global comprehensive data resource directory. The overall integration follows a unified data resource framework standard to ensure that various types of data can coexist in the global directory and provide unified data resource services for the business applications of various departments and systems.

9. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 8, characterized in that: When merging data, the similarity is calculated as follows: in, Vector representation of fields or directory items. Fields with high similarity are considered to be the same item.

10. The method for constructing a complex scenario-oriented proprietary domain data resource directory according to claim 8, characterized in that: When merging data, determine the data selection and merging strategy based on the data's completeness, accuracy, and authority. For data conflicts, formulate resolution rules, take the latest data as the basis, or give priority to data from a specific source, to ensure that the merged data is correct and reliable.

Images