Internet financial consumption credit batch service distributed flow calculation processing engine

A batch business and processing engine technology, applied in the field of stream computing processing, can solve problems such as large single-machine batch systems, affecting normal transactions and downstream system data processing, and long batch processing time, so as to improve scalability and batch processing efficiency. , the effect of shortening the processing time

Pending Publication Date: 2021-03-23
SICHUAN XW BANK CO LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The current Internet finance consumer credit batch uses a single-machine single-process model. As the amount of data continues to increase, the pressure on the single-machine batch system is great, and the batch processing time is getting longer and longer, which has affected normal transactions and data processing of downstream systems.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Internet financial consumption credit batch service distributed flow calculation processing engine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following briefly introduces the drawings that need to be used in the description of the embodiments. Apparently, the accompanying drawings in the following description are only some examples or embodiments of this specification, and those skilled in the art can also apply this specification to other similar scenarios.

[0027] see figure 1 , Internet finance consumer credit batch business distributed stream computing processing engine, including the following steps,

[0028] Step S1001: read data from the source database, and send the source data stream to the kafka topic partition through the kafka producer;

[0029] Step S1002: read the source data message in Kafka, and create a reader for each partition in the topic of Kafka as a consumer;

[0030] Step S1003: the reader collects the messages from the same topic through stream computing, and forwards t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of flow calculation processing, and discloses an Internet financial consumption credit batch service distributed flow calculation processing engine, which comprisesthe following steps: step S1001, reading data from a source database, and sending a source data flow to kafka topic partition through a kafka producer; step S1002, reading a source data message in theKafka, and creating a reader as a consumer for each part in the top of the Kafka; step S1003, enabling the readers to gather the messages from the same topic through stream operation, and then forwarding the messages to each Segment of the GP through the HTTP Connector; step S1004, scheduling, loading and starting GP-SQL for Java using a stream data processing mode and stream operation through ascheduler Dispatcher, and realizing JOB of stream data processing to complete a specific service function; and S1005, writing the processed result data into a result database. According to the engine,the batch processing efficiency is improved through a distributed parallel technology, the processing time is shortened, and the Kafka high-speed stream transmission capability and the Greenplum strong stream operation execution capability are combined.

Description

technical field [0001] The invention relates to the field of stream computing processing, and specifically refers to a distributed stream computing processing engine for Internet financial consumer credit batch business. Background technique [0002] Internet consumer finance is essentially an online loan, which means that Internet financial companies with relevant qualifications provide consumers with a specific consumer product (except real estate or cars) or service loans through the Internet on the basis of big data credit investigation. financial operating model. Users can enjoy the convenience brought by Internet consumer finance as long as they log in to relevant websites and apply. [0003] Currently, Internet finance consumer credit batches use a single-machine, single-process model. As the amount of data continues to increase, the single-machine batch system is under great pressure, and the batch processing time is getting longer and longer, which has affected nor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q40/02G06F16/21G06F16/23G06F16/28
CPCG06F16/21G06F16/23G06F16/284G06Q40/03
Inventor 冯宇罗喜川
Owner SICHUAN XW BANK CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap