Production domain data quality quasi-real-time monitoring system and method for based on big data technology

A big data technology and data quality technology, applied in the field of computer software design, can solve problems such as heavy data processing and analysis workload, affecting real-time data problems, and difficulty in obtaining high-quality data, so as to reduce incremental problem data and improve Data quality level, effect of shortening verification time

Inactive Publication Date: 2018-12-07
GUANGZHOU POWER SUPPLY CO LTD
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, due to the huge amount of data in the process of production and operation, each data object has dozens of field data, the workload of data processing and analysis is heavy, and it cannot be realized by manual monitoring, which makes it difficult to obtain high-quality data, especially for augmented A large amount of problem data cannot be quickly verified and responded, which affects the real-time performance of data problem discovery

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The present invention will be further described in detail below in conjunction with specific embodiments.

[0023] A quasi real-time monitoring system for data quality in the production domain based on big data technology, including: data access layer, data storage processing layer and data reference layer. Among them, the data access layer is used to access various types of source data, such as 4A systems, human resources systems, financial systems, material systems, marketing systems, and production systems. The data storage processing layer performs distributed storage of streaming data and automatically runs offline batch processing tasks for workflow in real time. The data storage processing layer is set up with a Kafka message queue that provides real-time streaming data, which can decouple data writing and data reading, real-time high availability and high concurrency; it also includes a stream processing engine that performs real-time calculation of streaming data;...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a production domain data quality quasi-real-time monitoring system and method based on big data technology. The system comprises a production domain data quality quasi-real-time monitoring system based on big data technology. The system comprise: a data access layer for accessing various types of source data; a data storage and processing layer, which is used for distributed storage of the data stream and performing real-time automatic operation of the workflow of offline batch processing tasks; and a data application layer, which is used for real-time analysis of dataquality and obtaining incremental data. The invention can quickly check and react to the incremental problem data, reduce the generation of the incremental problem data from the root cause, and accelerate the improvement of the data quality level of the production management information system.

Description

Technical field [0001] The invention relates to the technical field of computer software design, in particular to a quasi real-time monitoring system and method for data quality in the production domain. Background technique [0002] At present, big data covers all aspects of safety production. With the support of big data, safety production management is becoming smarter. By integrating various data information, the operation status of various fields of safety production is monitored to realize the improvement and optimization of safety production management. Safe production data management based on big data, with high-efficiency capture, discovery and analysis capabilities, can economically mine out valuable information from complex and large amounts of data, and realize real-time collection of data information, data storage, and Data analysis and comprehensive query functions provide data support for comprehensive management, comprehensive scheduling, comprehensive coordinatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 廖嘉炜孙煜华陆宏治吴永欢徐炫东池燕清张雨邹时容
Owner GUANGZHOU POWER SUPPLY CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap