Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and system for quickly loading data into database based on multi-process concurrency and plug-in

A data loading and data system technology, applied in the database field, can solve problems such as dependence, low loading speed, and high coupling, and achieve the effects of reducing coupling, enhancing scalability, and reducing intermediate processing processes

Active Publication Date: 2022-06-03
北京人大金仓信息技术股份有限公司
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] Aiming at the above-mentioned technical problems in the related art, the present invention proposes a method and system for quickly loading data into the database based on multi-process concurrency and plug-ins, which can solve the problems of low loading speed, heavy dependence on the framework of the database, and high coupling

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for quickly loading data into database based on multi-process concurrency and plug-in
  • Method and system for quickly loading data into database based on multi-process concurrency and plug-in
  • Method and system for quickly loading data into database based on multi-process concurrency and plug-in

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art fall within the protection scope of the present invention.

[0058] The following takes the loading of the CSV file format to the KingbaseES database server as an example to describe.

[0059] like figure 1 As shown, a method for quickly loading data into a database based on multi-process concurrency and plug-ins according to an embodiment of the present invention includes the following steps:

[0060] The database loading tool is responsible for reading the configuration file, connecting with the writing process of the dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and system for quickly loading data into a database based on multi-process concurrency and plug-ins, including a data loading tool and a data loading plug-in registered to a database server, and the data loading plug-in includes a writing process and multiple parsing processes; the method The method includes the following steps: the data loading tool is responsible for reading the configuration file, and connecting and calling the data loading plug-in to perform the loading work; the process that the data loading plug-in accepts the connection is a writing process, and the writing process creates a plurality of the parsing processes; Each of the parsing processes reads relevant data in the source data file in parallel by tasks and parses and assembles them into tuples, and puts the tuples into their corresponding shared memory queues; the writing process traverses all the shared memory queues , take out the tuple data and write it directly into the target data file. The beneficial effect of the invention is that the expansibility is enhanced, the coupling with the database server is reduced, and the loading efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a method and system for rapidly loading data into a database based on multi-process concurrency and plug-ins, which are suitable for batch data loading. Background technique [0002] With the rapid development of computer technology, the application of databases is becoming more and more extensive, and the efficiency of data loading is a very important indicator of the overall performance of the database. [0003] At present, the main technologies of database data loading are as follows: the traditional data loading process first reads a line of data from a data file, then parses it, then assembles the parsed data into a tuple, and finally writes the tuple into the database , that is, analyze a row of data and write a data record to the database. The main disadvantages of this method: analyzing data and writing data to the database cannot be performed at the same time; each ti...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/445G06F16/25
CPCG06F9/44526G06F16/252
Inventor 袁远松王建华杜春魁
Owner 北京人大金仓信息技术股份有限公司
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com