Mass data processing method and system, terminal and storage medium

A mass data processing and mass data technology, applied in the field of big data, can solve the problems of high system resource occupancy rate and high requirements for data processing software and hardware, and achieve the effect of simple structure, reliable design principle and wide application prospect

Pending Publication Date: 2020-07-31
INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Massive data processing software and hardware requirements are high, that is, the system resource occupancy rate is high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mass data processing method and system, terminal and storage medium
  • Mass data processing method and system, terminal and storage medium
  • Mass data processing method and system, terminal and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0051] figure 1 is a schematic flowchart of a method in one embodiment of the present invention. in, figure 1 The execution subject may be a massive data processing system.

[0052] Such as figure 1 As shown, the method 100 includes:

[0053] Step 110, setting basic rules for data quality inspection;

[0054] Step 120, modify the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a mass data processing method and system, a terminal and a storage medium. The mass data processing method comprises the steps: setting basic rules of data quality detection; modifying the basic rule according to service requirements, and generating a specific service data verification template according to the modified basic rule; verifying the mass data by utilizing a basic rule and the specific service verification template; and maintaining the verified problem data and the verification result of the problem data to a problem database. According to the method, the problem data can be quantitatively, quickly and accurately positioned in batches, and the problem data is listed in a data report mode, so that the method has guiding significance for data governance. And the problem of low data processing efficiency caused by insufficient system resources is avoided.

Description

technical field [0001] The present invention relates to the technical field of big data, in particular to a massive data processing method, system, terminal and storage medium. Background technique [0002] With the rapid development of informatization, big data is already an inevitable product of informatization, and in-depth mining and analysis of big data is the top priority. The data quality has become an important indicator of big data analysis, and the most valuable data information can only be obtained by in-depth mining and in-depth analysis in the data with better data quality. [0003] The hardware and software requirements for massive data processing are high, that is, the system resource occupancy rate is high. To process massive data, in addition to good methods and reasonable use of tools, the most important thing is to allocate system resources reasonably. In general, if the data to be processed exceeds TB level, minicomputers must be considered. If there is...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215
CPCG06F16/215
Inventor 王雪静
Owner INSPUR SUZHOU INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap