A data processing system and method incorporating multiple intelligent analysis languages

A data processing system and intelligent analysis technology, applied in database management systems, database models, structured data retrieval, etc., can solve problems such as inconsistent data source standards, inability to meet parallelization, and inability to meet all-domain processing needs. The effect of improving analytical capabilities and improving analytical efficiency

Inactive Publication Date: 2019-03-26
南京中新赛克科技有限责任公司
View PDF3 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Under the general trend of convenience and informatization, the problem of independent intelligent language processing has gradually become prominent, mainly in the following aspects: (1) A single intelligent processing language is mostly for specific fields, and cannot meet the needs of all fields. Processing needs; (2) The standards for data sources of various analysis languages ​​​​are not uniform, which puts forward very high requirements for data cleaning and transformation; (3) With the increase of data volume, intelligent languages ​​that can only be processed by a single machine cannot meet Parallelization, the need for distributed processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data processing system and method incorporating multiple intelligent analysis languages
  • A data processing system and method incorporating multiple intelligent analysis languages
  • A data processing system and method incorporating multiple intelligent analysis languages

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] figure 1 It is a schematic structural diagram of a data processing system that integrates multiple intelligent analysis languages ​​implemented according to the present invention. It includes three basic logic modules: data access module, analysis and mining module and data storage module.

[0027] The data access module is responsible for accessing different types of data sources and formatting them into formats required for subsequent processing, such as relational databases, big data storage systems, and streaming data.

[0028] The analysis and mining module is responsible for using the accessed data, scheduling tasks according to the configured task flow, and submitting tasks to a specific intelligent language system for processing. In particular, if users need to use parallel processing, they can call spark-based processing.

[0029] The data storage module is responsible for receiving the results of the analysis and mining module and storing them in specific de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing system and a method for fusing multiple intelligent analysis languages. The system comprises three basic logic modules: a data access module, an analysis mining module and a data storage module. The method comprises the following steps: a user initiates a process; User configures the multi-intelligent language process by dragging and dropping operators onOceanMind platform. The system generates a task execution tree according to the configured process. Executing the task, based on the data access provided by OceanMind, cleaning the conversion function, uniformly processing the data source, and sending the request for processing according to different intelligent processing languages; Receiving processing tasks, selecting native language-based or spark-based parallel processing methods according to different languages and whether parallelization is needed or not; Consolidating that result from the different language processes and perform persistence operation for subsequent processes to use. The invention can provide a unified data cleaning conversion function, which is used for distributed analysis of all data sources.

Description

technical field [0001] The invention relates to the technical field of intelligent analysis languages, in particular to a data processing system and method integrating multiple intelligent analysis languages. Background technique [0002] In recent years, with the rapid development of computer technology and the Internet, the era of information explosion has begun. The society is flooded with more data than ever before, prompting people to seek more suitable ways of data intelligent analysis. Existing intelligent analysis languages, such as: python, R language and tensorflow, after a certain period of development, have been able to achieve effective analysis of data in specific fields. [0003] Under the general trend of convenience and informatization, the problem of independent intelligent language processing has gradually become prominent, mainly in the following aspects: (1) A single intelligent processing language is mostly for specific fields, and cannot meet the need...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/25G06F16/28G06F16/27
Inventor 何海峰王文志谢东
Owner 南京中新赛克科技有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap