Industrial data statistical reporting system

Abstract

The invention provides an industrial data statistical reporting system. The industrial data statistical reporting system comprises an acquisition task module, a data acquisition module, a data processing module, a data aggregation and summarization module, an intelligent reporting module, a process real-time supervision module and a security module. Wherein the data acquisition module can extract original data from various data sources according to data acquisition tasks; the data processing module can process data through the data standard processing component, including data standardization, monitoring and analysis of data indexes and data verification, so as to obtain standard data and store the standard data; the security module can realize the security guarantee of the whole process of data acquisition, processing, reporting and the like, thereby realizing the standardization and normalization of the data to meet the requirements of different data sources, and supporting the data sharing and cooperative work on the premise of ensuring the data security.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an industrial data statistics reporting system. Background Art

[0002] With the rapid development of information technology and the advent of the big data era, the amount of data generated by all walks of life has increased exponentially, and the data types have become increasingly complex and diverse. In order to meet this challenge and make full use of the value of big data, the industrial data statistical reporting system came into being. This system is usually built based on advanced technologies such as cloud computing, big data processing, data mining, and artificial intelligence, aiming to realize the automated collection, processing, analysis, and reporting of industrial data. It has the following characteristics:

[0003] 1. Data collection technology: Through API interfaces, smart terminal devices and other means, industry-related structured, semi-structured and unstructured data can be collected in real time and accurately.

[0004] 2. Data storage technology: Use advanced storage technologies such as databases to ensure efficient and secure storage of massive data.

[0005] 3. Data processing technology: Use data cleaning, data conversion, data integration and other processing technologies to pre-process the collected raw data to improve data quality.

[0006] 4. Data analysis and mining technology: Use data mining, machine learning and other algorithms to conduct in-depth analysis of processed data and explore the patterns and trends behind the data.

[0007] 5. Data visualization technology: Through visualization methods such as charts and dashboards, the data analysis results are intuitively displayed to facilitate user understanding and use.

[0008] At present, although the background technology of the industrial data statistics reporting system has many advantages, there are still some shortcomings in practical applications:

[0009] 1. Data security issues: With the increase in data volume and the complexity of data types, data security issues are becoming increasingly prominent. How to ensure the security of data during collection, storage, processing, analysis and reporting is an issue that needs to be considered in system design and implementation.

[0010] 2. Data standardization and normalization issues: There are differences in data standards and formats between different industries and enterprises. How to achieve data standardization and normalization and improve data comparability and consistency is a major challenge in system design and implementation. Summary of the invention

[0011] The technical problem to be solved by the present invention is to provide an industrial data statistical reporting system to achieve data standardization and normalization to meet the needs of different data sources, and to support data sharing and collaborative work under the premise of ensuring data security.

[0012] In order to solve the above technical problems, the present invention provides an industrial data statistics reporting system, comprising:

[0013] Collection task module: used to establish data collection tasks according to the issued industrial data statistical reporting tasks, configure various data sources and reporting methods, and connect the required data preliminary inspection components, data standard processing components, data approval components and data correction components;

[0014] Data collection module: used to extract raw data from various data sources according to data collection tasks, and perform a preliminary check on the raw data through the connected data preliminary check component to filter out invalid or erroneous data entries;

[0015] A data processing module is used to process data through the data standard processing component, including data standardization, monitoring and analysis of data indicators, and data verification, to obtain standard data and store it;

[0016] The data aggregation and summary module is used to aggregate and summarize the data through the configured aggregation algorithm and summary algorithm to obtain the reported data and store it;

[0017] An intelligent reporting module, after the reported data is manually pre-examined by the data approval component, amends the reported data by the data amendment component, and performs automatic intelligent submission using the configured reporting method;

[0018] The real-time process monitoring module is used to monitor the task issuance, data collection, data processing and data reporting in real time by establishing a large data screen and data analysis reports;

[0019] Safety modules for:

[0020] During the data collection phase, the purpose of data collection should be clarified to ensure the legality and compliance of data collection; the purpose, scope and usage of data collection should be fully disclosed to the data source to obtain explicit authorization from the data source: encryption technology should be used to protect data transmission security, and access control technology should be used to ensure that only authorized personnel can access sensitive data;

[0021] Establish a sound data management system during the data storage stage, clarify the requirements for data storage, backup, recovery and access rights, and ensure data security and integrity; strengthen data security protection, including data encryption, identity authentication, access control, secure backup and disaster recovery;

[0022] The reported data is encrypted during the data reporting stage to ensure that it is not stolen or tampered with during transmission; a strict identity authentication mechanism is implemented to ensure that only authorized users can report data; and a secure communication protocol is used to protect the security of data transmission in the network.

[0023] Furthermore, the collection task module supports multi-source heterogeneous data source configuration, structured unstructured data configuration and data collection RPA script editing; the reporting method includes automated intelligent reporting through RPA scripts or automated intelligent reporting using RPA to configure fixed business processes.

[0024] Furthermore, the data standardization further includes defining a unified data model, formulating a data dictionary and implementing data mapping; the definition of a unified data model is to design a detailed data model for each data asset, including field names, data types, and character restrictions; the formulation of a data dictionary is used to create a data dictionary to clarify the meaning, format, and value range of each data item to ensure data consistency and comprehensibility; the implementation of data mapping is to use data mapping technology to convert data from different sources into a unified format and structure when sharing data between different systems or departments;

[0025] The monitoring and analysis of data indicators further includes establishing a key performance indicator system, real-time monitoring and analysis, and automated report generation; the key performance indicator system is to determine a series of key performance indicators based on business objectives and data analysis requirements; the real-time monitoring and analysis is to use big data technology and tools to monitor the performance of these indicators in real time, promptly detect abnormal situations and make adjustments; the automated report generation is to use automated tools to regularly generate reports containing key indicators to help decision makers quickly understand business conditions;

[0026] The data verification further includes setting verification rules and implementing automated verification processes; the setting of verification rules sets verification rules for data fields to ensure the accuracy and completeness of the data; the implementation of automated verification processes is based on the rule engine to develop automated scripts or tools to perform real-time verification on input data, and automatically trigger alarms or corrective operations when data that does not conform to the rules is found.

[0027] The present invention has at least the following technical effects or advantages: the industrial data statistical reporting system of the present invention extracts original data from various data sources through a data acquisition module, processes the data through a data processing module, including data standardization, monitoring and analysis of data indicators, and data verification, to obtain standard data, thereby realizing standardization and normalization of industrial data and improving the comparability and consistency of data; and ensures the security of data during the collection, storage, processing, analysis and reporting process through a data security module, supporting data sharing and collaborative work.

[0028] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0030] Figure 1 It is a structural block diagram of an embodiment of the system of the present invention;

[0031] Figure 2 The business framework and process diagram of an application embodiment of the system of the present invention. DETAILED DESCRIPTION

[0032] The embodiment of the present application provides an industrial data statistical reporting system to achieve data standardization and normalization to meet the needs of different data sources, and supports data sharing and collaborative work under the premise of ensuring data security.

[0033] The technical solution in the embodiments of the present application has the following overall idea: the industrial data statistical reporting system of the present invention extracts original data from various data sources through a data acquisition module, processes the data through a data processing module, including data standardization, monitoring and analysis of data indicators, and data verification, to obtain standard data, thereby achieving standardization and normalization of industrial data and improving the comparability and consistency of data; and ensures the security of data during the collection, storage, processing, analysis and reporting process through a data security module, supporting data sharing and collaborative work.

[0034] like Figure 1 As shown, an embodiment of the present invention provides an industrial data statistics reporting system, including a collection task module, a data collection module, a data processing module, a data aggregation and summary module, an intelligent reporting module, a process real-time monitoring module and a security module.

[0035] Collection task module: used to establish data collection tasks and design task templates according to the issued industrial data statistical reporting tasks, configure various data sources and reporting methods, and connect the required data preliminary inspection components, data standard processing components, data approval components, and data correction components;

[0036] Among them: Task templates are various report templates designed according to business needs, including table formats, field definitions, etc., and templates can be selected or customized as needed; the task assignment system automatically assigns report tasks to designated personnel, and sends notifications via email, SMS or system messages;

[0037] The preliminary check component is summarized as the basic verification of data when the user fills in the data for the first time or uses the data collection function to collect data for the first time, for example: verification of empty fields, length range of mobile phone numbers, etc.;

[0038] The data standard processing component is generally a component that can standardize field attributes. For example, when collecting data, the name field attribute name is defined as XINGMING, and the client or data user requires the standard attribute name to be DB_NAME. In this scenario, data conversion processing is performed;

[0039] The data approval component is summarized as attaching an OA approval component to the collected data or manually submitted data, so that the data can be approved by the superiors;

[0040] The data modification can be summarized as the operation of editing the approved data in real time and storing it in the database.

[0041] Furthermore, the collection task module supports multi-source heterogeneous data source configuration, structured unstructured data configuration and data collection RPA script editing; the reporting method includes automated intelligent reporting through RPA scripts or automated intelligent reporting using RPA to configure fixed business processes.

[0042] Data collection module: used to extract raw data from various data sources according to data collection tasks, and perform preliminary inspection on the raw data through the connected data preliminary inspection component to filter out invalid or erroneous data entries; report filling performs data filling in a real-time visual interface, and the specific implementation is as follows: the data collection module provides an online reporting interface for data entry, editing and uploading.

[0043] A data processing module is used to process data through the data standard processing component, including data standardization, monitoring and analysis of data indicators, and data verification, to obtain standard data and store it;

[0044] Furthermore, the data standardization further includes defining a unified data model, developing a data dictionary, and implementing data mapping;

[0045] Defining a unified data model is to design a detailed data model for each data asset, including field name, data type, character restrictions, etc.

[0046] The data dictionary is used to create a data dictionary, clarify the meaning, format and value range of each data item, and ensure the consistency and comprehensibility of the data;

[0047] The implementation of data mapping is to use data mapping technology to convert data from different sources into a unified format and structure when sharing data between different systems or departments;

[0048] The monitoring and analysis of data indicators further includes establishing a key performance indicator system, real-time monitoring and analysis, and automated report generation;

[0049] The key performance indicator system is to determine a series of key performance indicators based on business objectives and data analysis requirements;

[0050] The real-time monitoring and analysis mentioned above is to use big data technology and tools to monitor the performance of these indicators in real time, detect abnormal situations in time and make adjustments;

[0051] The automated report generation is to periodically generate reports containing key indicators through automated tools, thereby helping decision makers to quickly understand business conditions;

[0052] The data verification further includes setting verification rules and implementing an automated verification process;

[0053] The setting of verification rules is to set verification rules for data fields to ensure the accuracy and completeness of data;

[0054] The automated verification process is to develop an automated script or tool based on a rule engine to perform real-time verification on input data and automatically trigger an alarm or corrective action when data that does not conform to the rules is found.

[0055] The data aggregation and summary module is used to aggregate and summarize data through the configured aggregation and summary algorithms to obtain reported data and store it; aggregation algorithms such as SQL aggregation functions, MapReduce, machine learning algorithms, and data fusion technology; summary algorithms such as SUM, AVG, COUNT and other QL aggregation functions are used to summarize data in the database.

[0056] The intelligent reporting module, after manually pre-examining the reported data through the data approval component, corrects the reported data through the data correction component, and uses the configured reporting method for automatic intelligent reporting; the reporting method is such as automatic reporting using RPA scripts, or automatic intelligent reporting of fixed business processes configured using RPA.

[0057] The real-time process monitoring module is used to monitor the task issuance, data collection, data processing and data reporting in real time by establishing a large data screen and data analysis reports;

[0058] Safety modules for:

[0059] During the data collection phase, the purpose of data collection should be clarified to ensure the legality and compliance of data collection; the purpose, scope and usage of data collection should be fully disclosed to the data source to obtain explicit authorization from the data source: encryption technology should be used to protect data transmission security, and access control technology should be used to ensure that only authorized personnel can access sensitive data;

[0060] Establish a sound data management system during the data storage stage, clarify the requirements for data storage, backup, recovery and access rights, and ensure data security and integrity; strengthen data security protection, including data encryption, identity authentication, access control, secure backup and disaster recovery;

[0061] The reported data is encrypted during the data reporting stage to ensure that it is not stolen or tampered with during transmission; a strict identity authentication mechanism is implemented to ensure that only authorized users can report data; and a secure communication protocol is used to protect the security of data transmission in the network.

[0062] like Figure 2 As shown, the system of the present invention can be used in specific applications, such as when a township collects and reports industrial data within the township according to the superior data demand department, and the business system is divided into: data demand department client, township handling client and village handling client. The business process is as follows:

[0063] (1) Demand initiation: The data demand department initiates the reporting demand and sends it to the townships.

[0064] (2) Sending down the requirements: The township office receives the application form and sends it to the village office.

[0065] (3) Data reporting: The village office fills in the contents of the form issued. The form design and layout can be used to upload attachments, manually fill in fields, automatically fill in RPA, and automatically echo the API interface after the data is filled in and reported.

[0066] (4) Data review: The township office will review the reported data and return any unqualified data to the village office for revision.

[0067] (5) Data aggregation: After all data are filled in, the township office will aggregate and summarize the data and report it to the department that needs the data.

[0068] (6) Data review: The data demand department reviews the summary data and conducts statistical analysis on the reported data.

[0069] The system provides flexible and versatile data reporting, summary and error correction application tools to realize report task issuance, report filling, data filling task tracking, etc.

[0070] in:

[0071] To issue a report task, you need to design a task template, assign tasks, set permissions, and then issue the task. The specific implementation is as follows:

[0072] Design task templates: Design a variety of report templates based on business needs, including table formats, field definitions, etc. Users can select or customize templates as needed.

[0073] Task assignment and notification: The system automatically assigns report tasks to designated personnel and sends notifications via email, SMS or in-system messages.

[0074] Permission control: Set different permission levels to ensure that only authorized personnel can receive and process report tasks.

[0075] Report filling is to fill in data in the real-time visual interface. The specific implementation is as follows:

[0076] Online reporting interface: provides an intuitive and easy-to-use online reporting interface that supports data entry, editing and uploading.

[0077] Data Verification: During the reporting process, data in the table is verified in real time to ensure the accuracy and completeness of the data.

[0078] Save Draft: Allows users to save unfinished reports as drafts so they can continue filling in or modify them later.

[0079] Data reporting task tracking, including:

[0080] Status monitoring: Real-time tracking of the reporting status of each report, including not started, in progress, completed, etc.

[0081] Historical record query: Provides historical report record query function to facilitate users to review past report information.

[0082] Data reporting can be linked to the OA approval flow for level-by-level approval. The specific implementation is as follows:

[0083] Hierarchical relationship setting: Set up the organizational structure and hierarchical relationship in the system, and clarify the reporting process between each level.

[0084] Level-by-level review: Superiors can review the data reported by subordinates and make suggestions for revisions or approve it directly.

[0085] Summarize and correct the data, that is, correct the errors. The specific implementation is as follows:

[0086] Automatic summary: The system automatically aggregates the data reported at each level and generates a comprehensive report.

[0087] Error detection: Automatically detect errors and inconsistencies in data through preset rules or algorithms.

[0088] Error correction feedback: Feedback detected errors to relevant personnel and provide error correction guidance or suggestions.

[0089] For example, based on the business scenario of "autumn harvest and winter planting", users need to count the progress of autumn harvest in real time. Based on the data reporting system, they first establish and design a form sample for collecting data, and then establish a daily progress task for autumn harvest to issue data reporting tasks. In this process, the data collection situation can be monitored in real time through visual data reports and large data screens, and the situation reported by the task personnel of the day can be viewed through task management. Finally, the data is submitted after manual pre-examination through data summary and no problems are found.

[0090] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. An industrial data statistics reporting system, characterized by: include: Collection task module: used to establish data collection tasks according to the issued industrial data statistical reporting tasks, configure various data sources and reporting methods, and connect the required data preliminary inspection components, data standard processing components, data approval components and data correction components; Data collection module: used to extract raw data from various data sources according to data collection tasks, and perform a preliminary check on the raw data through the connected data preliminary check component to filter out invalid or erroneous data entries; A data processing module is used to process data through the data standard processing component, including data standardization, monitoring and analysis of data indicators, and data verification, to obtain standard data and store it; The data aggregation and summary module is used to aggregate and summarize the data through the configured aggregation and summary algorithms, and store them; An intelligent reporting module, after the reported data is manually pre-examined by the data approval component, amends the reported data by the data amendment component, and performs automatic intelligent submission using the configured reporting method; The real-time process monitoring module is used to monitor the task issuance, data collection, data processing and data reporting in real time by establishing a large data screen and data analysis reports; Safety modules for: Clarify the purpose of data collection during the data collection stage to ensure the legality and compliance of data collection; Fully disclose the purpose, scope and usage of data collection to the data source to obtain explicit authorization from the data source: use encryption technology to protect data transmission security and use access control technology to ensure that only authorized personnel can access sensitive data; Establish a sound data management system during the data storage stage, clarify the requirements for data storage, backup, recovery and access rights, and ensure data security and integrity; Strengthen data security protection, including data encryption, identity authentication, access control, secure backup and disaster recovery; Encrypt the reported data during the data reporting stage to ensure that it is not stolen or tampered with during transmission; implement a strict identity authentication mechanism to ensure that only authorized users can report data; Use secure communication protocols to protect the security of data transmission in the network.

2. The industrial data statistics reporting system according to claim 1 is characterized by: The collection task module supports multi-source heterogeneous data source configuration, structured unstructured data configuration and data collection RPA script editing; the reporting method includes automated intelligent reporting through RPA scripts or automated intelligent reporting using RPA to configure fixed business processes.

3. The industrial data statistics reporting system according to claim 1 is characterized by: The data standardization further includes defining a unified data model, formulating a data dictionary and implementing data mapping; the definition of a unified data model is to design a detailed data model for each data asset, including field names, data types, and character restrictions; the formulation of a data dictionary is used to create a data dictionary to clarify the meaning, format, and value range of each data item to ensure data consistency and comprehensibility; the implementation of data mapping is to use data mapping technology to convert data from different sources into a unified format and structure when sharing data between different systems or departments; The monitoring and analysis of the data indicators further includes establishing a key performance indicator system, real-time monitoring and analysis, and automatic report generation; the key performance indicator system is to determine a series of key performance indicators based on business objectives and data analysis requirements; The real-time monitoring and analysis is to use big data technology and tools to monitor the performance of these indicators in real time, detect abnormal situations in time and make adjustments; the automatic report generation is to use automatic tools to regularly generate reports containing key indicators to help decision makers quickly understand the business status; The data verification further includes setting verification rules and implementing automated verification processes; the setting of verification rules sets verification rules for data fields to ensure the accuracy and completeness of the data; the implementation of automated verification processes is based on the rule engine to develop automated scripts or tools to perform real-time verification on input data, and automatically trigger alarms or corrective operations when data that does not conform to the rules is found.

Images