Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method and system

A data processing system and data table technology, applied in the computer field, can solve the problem of high threshold of use

Pending Publication Date: 2022-07-01
CHINA CONSTRUCTION BANK
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The rest of the process also includes model building, but this process also requires modelers to understand the mathematical principles of various modeling algorithms, be proficient in various programming grammars and interface calling specifications of various modeling algorithms, and be able to use the corresponding programming language End-to-end realization of model building, high barriers to use

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system
  • Data processing method and system
  • Data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0055] In this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that any such relationship exists between these entities or operations. an actual relationship or sequence. Also the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion su...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data processing method and system, and the method comprises the steps: submitting a task request in an sqlite database to an Executor tool when a task execution module in the data processing system scans that the task request exists in the sqlite database; the Executor tool loads and prepares data in the task request, splices the feature engineering and the wide table and submits the data to a Spark server; the Spark server executes data loading and prepares to obtain a plurality of data tables; executing feature engineering based on each data table to obtain an independent variable box and a coding mapping table of each data table; executing wide table splicing based on the plurality of independent variable sub-boxes and the coding mapping table, and sending a target splicing table and a wide table splicing completion notification to the Executor tool; and the Executor tool receives a splicing completion notification, and sends the target splicing table and the model construction to a server corresponding to the task type of the Executor tool for model construction to obtain a target model.

Description

technical field [0001] The present invention relates to the field of computer technology, and more particularly, to a data processing method and system. Background technique [0002] With the continuous development of computer technology, the Internet, banking, securities companies, insurance and other industries have widely used computer technology for big data analysis and model construction, such as the CRISP-DM machine learning data mining standard process developed by computer technology; [0003] In the standard process of CRISP-DM machine learning data mining, data understanding and data preparation at least involve the loading and preparation of dozens or even hundreds of original data tables, feature engineering and modeling wide table processing. These processes require the modeler to be very familiar with the business meaning and metadata of the original data, and also require the modeler to be proficient in SQL programming syntax and be able to write SQL code wit...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F16/2455
CPCG06F9/4881G06F16/2455
Inventor 杨俊谢辰昊韩晗
Owner CHINA CONSTRUCTION BANK