Digital asset acquisition device and method
Through digital asset collection devices, a variety of technical means are used to achieve efficient and unified collection and integration of digital assets, solving the problems of scattered data sources, backward collection technology, and security and privacy, improving data quality and transmission security, and supporting scientific decision-making and data value mining for enterprises.
Patent Information
- Application Number
- CN202510967656.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-14
- Publication Date
- 2025-09-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The sources of digital assets are scattered and difficult to integrate, the collection technology is backward and inefficient, the data quality is uneven, and security and privacy issues are prominent.
A digital asset collection device is provided, including data collection, transmission, processing, quality monitoring, storage and application modules, which adopts ETL tools, text mining technology, TCP/IP protocol, SSL/TLS encryption, data caching, asynchronous transmission, data cleaning, conversion, integration and visual analysis technologies to ensure the security, integrity and consistency of data.
It achieves efficient and unified collection and integration of digital assets, improves data quality, ensures transmission security, and supports scientific decision-making and data value mining for enterprises.
Smart Images

Figure CN120676024A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of digital asset technology, and in particular to a digital asset collection device and method. Background Art
[0002] In the digital age, digital assets have become a crucial asset for businesses and organizations. These encompass various forms of digital information, including business data, customer information, intellectual property, and digital content. These assets are crucial for businesses' decision-making, business operations, innovation, and market competitiveness.
[0003] With the continuous development of information technology, various industries are placing increasing emphasis on digital assets. For example, in the financial industry, digital assets include transaction data and customer credit information, which are crucial for risk assessment and investment decisions. In the manufacturing industry, digital assets such as product design data and production process data are the foundation for intelligent manufacturing, improving production efficiency and product quality. In the cultural and creative industries, digital content such as music, film, television, and games are core assets. However, to fully realize the value of digital assets, they must first be collected efficiently and accurately.
[0004] However, the following problems may occur during the current collection process:
[0005] 1. Dispersed data sources make integration difficult: Digital assets come from a wide range of sources, including multiple internal business systems, external partners, IoT devices, and social media. For example, a retail company's sales data comes from online e-commerce platforms and offline store checkout systems, while customer data comes from membership management systems and user interaction data on social media platforms. This data is dispersed across various systems and platforms, with varying data formats and interface specifications, making integration extremely difficult and making it difficult to form a unified view of digital assets.
[0006] 2. Outdated and inefficient data collection technology: Some companies still rely on traditional manual data entry or simple data capture methods to collect digital assets. Some small and medium-sized enterprises may still rely on manual entry of financial data from paper invoices and receipts into financial systems. This method is not only labor-intensive and time-consuming, but also prone to human error, resulting in inaccurate data. Furthermore, traditional data collection methods are unable to capture and process data requiring high real-time performance, such as real-time monitoring data generated by IoT devices, impacting the data's timeliness and application value.
[0007] 3. Inconsistent data quality: Due to the lack of effective quality control mechanisms during the data collection process, the collected data is subject to errors, duplications, and omissions. During the customer information collection process, errors in key information such as customer names and contact information may occur due to user input errors or lax system validation. During data integration, duplicate records may appear due to inconsistent identification of the same entity across different data sources. This low-quality data can affect the accuracy of data analysis and, in turn, the scientific nature of corporate decision-making.
[0008] 4. Security and Privacy Issues: Digital asset collection involves a significant amount of sensitive information, such as personal customer information and corporate trade secrets. Some data collection systems lack adequate security measures during data transmission and storage, making them vulnerable to hacker attacks, data leaks, and other security threats. Some small online service platforms fail to encrypt the data they collect, creating the risk of data theft during network transmission. Furthermore, improper access control during data use can lead to data misuse and infringe on user privacy.
[0009] Based on the above, a digital asset collection device and method are invented. Summary of the Invention
[0010] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:
[0011] A digital asset collection device, comprising:
[0012] The data collection module is used to collect digital assets from various data sources and can use corresponding data collection technologies and tools for different data sources;
[0013] The data transmission module is used to transmit the collected data using a transmission protocol. For the transmission of large amounts of data, data caching and asynchronous transmission technology are used to improve transmission efficiency and reduce network congestion. At the same time, to ensure the security of data transmission, the transmitted data is encrypted.
[0014] The data processing module is used to clean, convert, and integrate the collected data to improve data quality and meet the requirements of subsequent analysis and application;
[0015] Data quality monitoring module, used to monitor the quality of processed data;
[0016] The data storage module is used to select an appropriate storage method to store qualified data; at the same time, a data backup and recovery mechanism is established to back up data regularly to ensure data security and integrity;
[0017] The application module is used to provide users with data query, analysis, and visualization application functions; it first provides users with data query through the data query function, and then uses data visualization tools to display digital assets to users in an intuitive form, so that users can understand the information and trends behind the data; then it provides data analysis functions to support users in data mining and machine learning analysis operations, providing data support for corporate decision-making.
[0018] As a preferred solution of the digital asset collection device described in the present invention, the data processing module includes:
[0019] The data cleaning module is used to clean the data to remove errors, duplications, and omissions in the collected data, thereby improving the accuracy and completeness of the data;
[0020] Data conversion module, used to unify data of different formats, encodings, and structures into a recognizable and processable format to facilitate subsequent data integration and analysis;
[0021] The data integration module is used to associate and merge data from different data sources according to certain rules to build a unified digital asset data set.
[0022] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of the data cleaning module are:
[0023] M1, Error Data Identification and Correction: Data is checked according to pre-set data verification rules. Data that can be corrected through logical deduction or reference to other reliable data sources is automatically corrected. If the data cannot be automatically corrected, it is recorded for further manual verification and processing.
[0024] M2, duplicate data processing: Through a specific algorithm and field combination, duplicate data records are identified, one of which is retained and the remaining duplicate records are deleted to prevent data redundancy from affecting subsequent analysis and storage;
[0025] M3, missing value processing: For data with missing values, choose the appropriate processing method based on the data type and business needs.
[0026] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of the data conversion module are:
[0027] N1, data format conversion: convert the data into the required format;
[0028] N2, data encoding conversion: This handles the different encoding methods used by different data sources to ensure that data can be correctly displayed and processed within the system, avoiding problems such as garbled characters;
[0029] N3, data value mapping: converting code values and abbreviations in the data into actual meanings.
[0030] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of the data integration module are:
[0031] Q1, Data association rule definition: Determine the basis for data association based on business logic and data relationships;
[0032] Q2, Data merging: Merge the associated data according to the association rules. During the merging process, for duplicate fields, determine which data source's value to retain based on factors such as data accuracy and timeliness to ensure the accuracy of the merged data.
[0033] Q3, Data consistency check: After data integration is completed, the integrated data is checked for consistency to ensure data consistency across different data sources. If inconsistencies are found, the data source is promptly traced to identify the cause and make corrections.
[0034] As a preferred solution of the digital asset collection device described in the present invention, the data quality monitoring module includes:
[0035] The real-time monitoring data module is used to first access and initialize data, then execute monitoring rules, and finally mark abnormal data;
[0036] Develop a quality assessment module to first set indicator weights, then calculate data scores, and finally generate a quality report;
[0037] The problem data positioning module is used to first trace the source of abnormal data, then determine the problem location, and then analyze the cause of the problem.
[0038] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of the real-time monitoring data module are:
[0039] S1, data access and initialization: first receive the cleaned, converted and integrated data, then establish the corresponding monitoring data structure according to the data type and storage structure;
[0040] S2, execute monitoring rules: Scan the data in real time according to the preset monitoring rules. At the same time, check whether the data in related fields after integrating data from different data sources is consistent to avoid conflicts.
[0041] S3, abnormal data marking: When data that does not comply with monitoring rules is found, it will be marked to add a specific identification field to the data record, indicating the abnormal type and specific problem description to facilitate subsequent processing and analysis.
[0042] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of carrying out the quality assessment module are:
[0043] P1, indicator weight setting: according to the enterprise's emphasis on data quality and business needs, set corresponding weights for evaluation indicators;
[0044] P2, data scoring calculation: Based on the results of real-time monitoring and the pre-set scoring algorithm, quantitative scoring is performed on each quality indicator;
[0045] P3, generate quality report: organize the scores of each quality indicator, the overall quality score and detailed information of abnormal data to generate a data quality report.
[0046] As a preferred solution of the digital asset collection device described in the present invention, the specific steps of locating the problem data module are:
[0047] E1, Abnormal Data Tracing: For marked abnormal data, trace it back from the data collection source along the data processing process;
[0048] E2, Problem Location Determination: During the traceability process, the problem data is precisely located within the data source, data processing steps, or data storage to determine which record in which data source has the problem, or which step in data processing caused the anomaly. This provides clear guidance for technical personnel to conduct targeted repairs.
[0049] E3, Problem Cause Analysis: Combine data processing flow and operation logs to deeply analyze the cause of the problem.
[0050] Compared with existing technologies:
[0051] 1. To address the challenges of integrating digital assets from diverse and dispersed sources, the data collection module leverages diverse collection technologies by integrating with multiple data sources, including internal enterprise business systems, databases, file systems, IoT devices, and external data platforms. ETL tools are used for structured data, text mining techniques are employed for unstructured data, and IoT device data is collected via specific protocols. This enables unified collection of disparate and diverse data, significantly reducing the complexity of data integration and facilitating the formation of a unified view of digital assets.
[0052] 2. To address issues such as outdated data collection technology, low efficiency, and data transmission security, the data transmission module uses the TCP / IP protocol to ensure stability. It also utilizes data caching and asynchronous transmission technologies, such as Redis for data caching and Apache Kafka for asynchronous transmission. This improves the efficiency of large-scale data transmission, reduces network congestion, and addresses the inefficiency of traditional data collection methods. Furthermore, SSL / TLS encryption protects transmitted data from being stolen during transmission, addressing deficiencies in data transmission security in some systems. For example, user information leaks due to unencrypted transmission on small online service platforms will no longer occur.
[0053] 3. To address the uneven quality of data, the data processing module incorporates cleaning, conversion, and integration. The data cleaning process utilizes validation rules to remove errors and duplicate data and fill in missing data; data conversion unifies data formats and encodings; and data integration combines and associates data according to rules. In customer information collection scenarios, this effectively corrects user input errors and avoids duplicate records due to inconsistent data source identifiers, ensuring the accuracy and completeness of collected data and providing a reliable foundation for subsequent data analysis and business decision-making.
[0054] 4. By setting up data storage modules, appropriate storage methods can be selected based on data types, using relational databases for structured data and distributed file systems for unstructured data. Data backup and recovery mechanisms can be used to ensure secure data storage and address data storage security risks. Furthermore, application modules can be set up to provide data query, analysis, and visualization functions. Data visualization tools can be used to intuitively display data, and data analysis functions can be used to mine data value. Customized modules can also be developed to manage digital assets throughout their lifecycle. This will change the situation in which enterprises have previously struggled to fully realize the value of digital assets, helping them make scientific decisions based on accurate data and enhance their market competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a schematic diagram of the overall framework of the present invention. DETAILED DESCRIPTION
[0056] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0057] The present invention provides a digital asset collection device, please refer to Figure 1 ,include:
[0058] The data collection module is used to collect digital assets from various data sources, including but not limited to internal enterprise business systems (such as ERP, CRM, OA, etc.), databases (relational databases, non-relational databases), file systems (office documents, log files, etc.), IoT devices (sensors, smart devices, etc.), and external data platforms (third-party data suppliers, social media platforms, etc.). It can use corresponding data collection technologies and tools for different data sources. For structured data, such as data in databases, ETL (Extract-Transform-Load) tools can be used for extraction, transformation, and loading; for unstructured data, such as documents and logs, text mining technology is used for data extraction; for IoT device data, real-time collection is carried out through protocols such as MQTT and HTTP;
[0059] The data transmission module is used to transmit the collected data using a transmission protocol (such as TCP / IP). For the transmission of large amounts of data, data caching and asynchronous transmission technologies are used to improve transmission efficiency and reduce network congestion. At the same time, to ensure data transmission security, the transmitted data is encrypted, such as using the SSL / TLS encryption protocol.
[0060] The data processing module is used to clean, convert, and integrate the collected data to improve data quality and meet the requirements of subsequent analysis and application;
[0061] Data quality monitoring module, used to monitor the quality of processed data;
[0062] The data storage module is used to select an appropriate storage method to store qualified data. For structured data, a relational database can be used for storage to facilitate data query and analysis. For unstructured data, such as documents, images, and videos, a distributed file system or object storage is used to store them to meet the needs of massive data storage and high scalability. At the same time, a data backup and recovery mechanism is established to regularly back up data to ensure data security and integrity.
[0063] The application module is used to provide users with data query, analysis, and visualization application functions; it first provides users with data query through the data query function, and then uses data visualization tools to display digital assets to users in an intuitive form, so that users can understand the information and trends behind the data; then it provides data analysis functions to support users in data mining and machine learning analysis operations, providing data support for corporate decision-making.
[0064] The data processing module includes:
[0065] The data cleaning module is used to clean the data to remove errors, duplications, and omissions in the collected data, thereby improving the accuracy and completeness of the data;
[0066] Data conversion module, used to unify data of different formats, encodings, and structures into a recognizable and processable format to facilitate subsequent data integration and analysis;
[0067] The data integration module is used to associate and merge data from different data sources according to certain rules to build a unified digital asset data set.
[0068] The specific steps of the data cleaning module are:
[0069] M1, Error Data Identification and Correction: Data is checked according to pre-set data verification rules. In the customer age field, if a negative number or an obviously unrealistically large value appears, the system will identify it as erroneous data. For data with clear value ranges or format requirements, such as ID card numbers and date formats, non-compliant data will also be marked. Data that can be corrected through logical deduction or reference to other reliable data sources will be automatically corrected. If the data cannot be automatically corrected, it will be recorded for further manual verification and processing.
[0070] M2, duplicate data processing: Duplicate data records are identified through specific algorithms and field combinations. In the customer information table, key information such as customer ID number and contact information is used as the basis for judgment. If multiple records have the same key information, they are considered duplicate data. One of the records will be retained and the remaining duplicate records will be deleted to prevent data redundancy from affecting subsequent analysis and storage.
[0071] M3, missing value processing: For data with missing values, choose the appropriate processing method based on the data type and business needs; for numerical data, statistics such as mean, median, mode, etc. can be used to fill in the missing values; for categorical data, the most frequently occurring value can be used to fill in the missing values.
[0072] The specific steps of the data conversion module are:
[0073] N1, Data format conversion: Convert data to a format that meets the requirements; convert date data from different display formats (such as "YYYY / MM / DD" and "MM-DD-YYYY") to the standard format specified by the system; convert string-type numeric data to numeric data for mathematical operations and statistical analysis;
[0074] N2, Data Encoding Conversion: This handles different encoding methods used by different data sources, such as converting UTF-8 encoded data to GBK encoded data, ensuring that data can be correctly displayed and processed within the system and avoiding problems such as garbled characters.
[0075] N3, Data Value Mapping: Converts code values and abbreviations in the data into their actual meanings. In the order status field, "0" represents "pending payment" and "1" represents "paid." The system converts these code values into corresponding Chinese descriptions, making the data easier to understand and use. At the same time, for inconsistent data standards, such as different representations of gender fields in different regions ("Male / Female", "M / F", etc.), data value mapping is also used for unified conversion.
[0076] The specific steps of the data integration module are:
[0077] Q1, Data Association Rule Definition: Determine the basis for data association based on business logic and data relationships. When integrating customer-related data, use the customer ID as the association field to associate basic customer information from the CRM system, customer transaction information from the sales system, and customer feedback information from the customer service system. When processing order data, use the order number as the association field to integrate order details, logistics information, payment information, etc.
[0078] Q2, Data Merge: Merge the associated data according to the association rules. Combine the customer's basic information, transaction information, feedback information, and other information into a complete customer data record containing all relevant customer information. During the merge process, for duplicate fields, the data source's value is determined based on factors such as data accuracy and timeliness to ensure the accuracy of the merged data.
[0079] Q3, Data consistency check: After data integration is completed, the integrated data is checked for consistency to ensure data consistency across different data sources. Check whether the customer's transaction amount is consistent in the sales system and the financial system, and whether the order delivery time matches in the order system and the logistics system. If any inconsistency is found, trace the data source in a timely manner to find the cause and make corrections.
[0080] The data quality monitoring module includes:
[0081] The real-time monitoring data module is used to first access and initialize data, then execute monitoring rules, and finally mark abnormal data;
[0082] Develop a quality assessment module to first set indicator weights, then calculate data scores, and finally generate a quality report;
[0083] The problem data positioning module is used to first trace the source of abnormal data, then determine the problem location, and then analyze the cause of the problem.
[0084] The specific steps of the real-time monitoring data module are:
[0085] S1, Data Access and Initialization: First, receive the cleaned, converted, and integrated data. Then, establish the corresponding monitoring data structure based on the data type and storage structure. For structured data, define field monitoring rules based on the database table structure. For unstructured data, determine the method for extracting and monitoring key information to prepare for subsequent monitoring work.
[0086] S2, execute monitoring rules: scan the data in real time according to the preset monitoring rules; check whether there are missing values in each field in the data set, and determine whether the field value is empty or meets the specific empty value identifier; when detecting duplicate values, perform hash calculation on the specified key field combination and compare the hash values to find duplicate records; verify whether the data field values comply with the preset rules, such as whether the numeric field is within the specified value range, whether the date field conforms to the correct date format, etc. At the same time, for the data integrated from different data sources, check whether the data in related fields are consistent to avoid contradictions;
[0087] S3, abnormal data marking: When data that does not comply with the monitoring rules is found, it will be marked to add a specific identification field to the data record, indicating the abnormal type (such as missing values, duplicate values, value errors, etc.) and specific problem description to facilitate subsequent processing and analysis.
[0088] The specific steps of carrying out the quality assessment module are:
[0089] P1. Indicator weighting: Based on the company's emphasis on data quality and business needs, appropriate weights should be assigned to evaluation indicators such as data integrity, accuracy, timeliness, and consistency. For financial transaction data, accuracy may be given a higher weight; while for real-time monitoring data, timeliness may be given a higher weight.
[0090] P2, Data Scoring Calculation: Based on the results of real-time monitoring, various quality indicators are quantitatively scored according to a pre-set scoring algorithm. For completeness indicators, the score is calculated based on the proportion of missing values; accuracy indicators are scored based on the number and severity of erroneous data; timeliness indicators are evaluated based on the delay in data collection and update; consistency indicators are scored based on the number and impact of data contradictions. Then, the overall data quality score is calculated by combining the weights of each indicator.
[0091] P3, Generate Quality Report: Organize the scores of various quality indicators, the overall quality score, and detailed information of abnormal data to generate a data quality report; the report is presented in the form of a combination of visual charts (such as bar charts, line charts, dashboards, etc.) and detailed text descriptions, clearly showing the data quality status, so that users can quickly understand the overall level of data quality and existing problems.
[0092] The specific steps of the positioning problem data module are:
[0093] E1, Abnormal Data Tracing: For marked abnormal data, trace back from the data collection source along the data processing process. By reviewing data collection logs, data processing records, and interaction information with the data source, determine whether the abnormal data is caused by problems in the original data source itself or errors in the data cleaning, conversion, and integration process.
[0094] E2, Problem Location Determination: During the traceability process, the problem data is precisely located within the data source, data processing steps, or data storage to determine which record in which data source has the problem, or which data processing step (such as a data cleaning rule, conversion function, or integration operation) caused the anomaly. This provides clear guidance for technical personnel to carry out targeted repairs.
[0095] E3, Problem Cause Analysis: Combined with the data processing process and operation logs, conduct an in-depth analysis of the cause of the problem. If it is a data source problem, determine whether it is a data entry error, system failure, or data transmission anomaly. If it is a data processing problem, analyze whether it is an unreasonable rule setting, algorithm defects, or parameter configuration errors, etc., to provide a basis for formulating effective solutions.
[0096] When used, the specific steps are as follows:
[0097] S1: Collect digital assets from various data sources through the data collection module, and use corresponding data collection technologies and tools for different data sources;
[0098] S2: The data transmission module uses a transmission protocol to transmit the collected data. For the transmission of large amounts of data, data caching and asynchronous transmission technology are used to improve transmission efficiency and reduce network congestion. At the same time, to ensure the security of data transmission, the transmitted data is encrypted.
[0099] S3: The data processing module cleans, converts, and integrates the collected data to improve data quality and meet the requirements of subsequent analysis and application;
[0100] S4: Perform quality monitoring on the processed data through the data quality monitoring module;
[0101] S5: Select an appropriate storage method through the data storage module to store qualified data; at the same time, establish a data backup and recovery mechanism and back up data regularly to ensure data security and integrity;
[0102] S6: Provide users with data query, analysis, and visualization application functions through application modules; first provide users with data query through the data query function, and then display digital assets to users in an intuitive form through data visualization tools, so that users can understand the information and trends behind the data; then provide data analysis functions to support users in data mining and machine learning analysis operations, and provide data support for corporate decision-making.
[0103] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A digital asset collection device, characterized in that: include: The data collection module is used to collect digital assets from various data sources and can use corresponding data collection technologies and tools for different data sources; The data transmission module is used to transmit the collected data using a transmission protocol. For the transmission of large amounts of data, data caching and asynchronous transmission technology are used to improve transmission efficiency and reduce network congestion. At the same time, to ensure the security of data transmission, the transmitted data is encrypted. The data processing module is used to clean, convert, and integrate the collected data to improve data quality and meet the requirements of subsequent analysis and application; Data quality monitoring module, used to monitor the quality of processed data; The data storage module is used to select an appropriate storage method to store qualified data; at the same time, a data backup and recovery mechanism is established to back up data regularly to ensure data security and integrity; The application module is used to provide users with data query, analysis, and visualization application functions; it first provides users with data query through the data query function, and then uses data visualization tools to display digital assets to users in an intuitive form, so that users can understand the information and trends behind the data; then it provides data analysis functions to support users in data mining and machine learning analysis operations, providing data support for corporate decision-making.
2. A digital asset collection device according to claim 1, characterized in that: The data processing module includes: The data cleaning module is used to clean the data to remove errors, duplications, and omissions in the collected data, thereby improving the accuracy and completeness of the data; Data conversion module, used to unify data of different formats, encodings, and structures into a recognizable and processable format to facilitate subsequent data integration and analysis; The data integration module is used to associate and merge data from different data sources according to certain rules to build a unified digital asset data set.
3. The method for collecting digital assets according to claim 2, wherein: The specific steps of the data cleaning module are: M1, Error Data Identification and Correction: Data is checked according to pre-set data verification rules. Data that can be corrected through logical deduction or reference to other reliable data sources is automatically corrected. If the data cannot be automatically corrected, it is recorded for further manual verification and processing. M2, duplicate data processing: Through a specific algorithm and field combination, duplicate data records are identified, one of which is retained and the remaining duplicate records are deleted to prevent data redundancy from affecting subsequent analysis and storage; M3, missing value processing: For data with missing values, choose the appropriate processing method based on the data type and business needs.
4. A digital asset collection method according to claim 3, characterized in that: The specific steps of the data conversion module are: N1, data format conversion: convert the data into the required format; N2, data encoding conversion: This handles the different encoding methods used by different data sources to ensure that data can be correctly displayed and processed within the system, avoiding problems such as garbled characters; N3, data value mapping: converting code values and abbreviations in the data into actual meanings.
5. A digital asset collection method according to claim 3, characterized in that: The specific steps of the data integration module are: Q1, Data association rule definition: Determine the basis for data association based on business logic and data relationships; Q2, Data merging: Merge the associated data according to the association rules. During the merging process, for duplicate fields, determine which data source's value to retain based on factors such as data accuracy and timeliness to ensure the accuracy of the merged data. Q3, Data consistency check: After data integration is completed, the integrated data is checked for consistency to ensure data consistency across different data sources; If any inconsistency is found, trace the data source in time, find the cause and make corrections.
6. A digital asset collection device according to claim 1, characterized in that: The data quality monitoring module includes: The real-time monitoring data module is used to first access and initialize data, then execute monitoring rules, and finally mark abnormal data; Develop a quality assessment module to first set indicator weights, then calculate data scores, and finally generate a quality report; The problem data positioning module is used to first trace the source of abnormal data, then determine the problem location, and then analyze the cause of the problem.
7. A digital asset collection method according to claim 6, characterized in that: The specific steps of the real-time monitoring data module are: S1, data access and initialization: first receive the cleaned, converted and integrated data, then establish the corresponding monitoring data structure according to the data type and storage structure; S2, execute monitoring rules: Scan the data in real time according to the preset monitoring rules. At the same time, check whether the data in related fields after integrating data from different data sources is consistent to avoid conflicts. S3, abnormal data marking: When data that does not comply with monitoring rules is found, it will be marked to add a specific identification field to the data record, indicating the abnormal type and specific problem description to facilitate subsequent processing and analysis.
8. A digital asset collection method according to claim 7, characterized in that: The specific steps of carrying out the quality assessment module are: P1, indicator weight setting: according to the enterprise's emphasis on data quality and business needs, set corresponding weights for evaluation indicators; P2, data scoring calculation: Based on the results of real-time monitoring and the pre-set scoring algorithm, quantitative scoring is performed on each quality indicator; P3, generate quality report: organize the scores of each quality indicator, the overall quality score and detailed information of abnormal data to generate a data quality report.
9. A digital asset collection method according to claim 6, characterized in that: The specific steps of the positioning problem data module are: E1, Abnormal Data Tracing: For marked abnormal data, trace it back from the data collection source along the data processing process; E2, Problem Location Determination: During the traceability process, the problem data is precisely located within the data source, data processing steps, or data storage to determine which record in which data source has the problem, or which step in data processing caused the anomaly. This provides clear guidance for technical personnel to conduct targeted repairs. E3, Problem Cause Analysis: Combine data processing flow and operation logs to deeply analyze the cause of the problem.
Citation Information
Patent Citations
Periodic digital twinning auxiliary management platform for photovoltaic construction
CN118839617A
Distribution network line interconnection switch multi-source data management method and system
CN118889666A
Acquisition method, device and system based on IOT (Internet of Things) data and medium
CN119357260A
Multi-dimensional accounting data integration and analysis system
CN119515578A