Stable high-throughput asynchronous task processing method based on kafka

An asynchronous task, high-throughput technology, used in electrical digital data processing, multi-programming devices, instruments, etc., can solve problems such as large coupling of business systems, and achieve the effect of reducing coupling, ensuring integrity, and improving functions

Active Publication Date: 2019-10-25
福建南威软件有限公司
View PDF7 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This mode is highly coupled with the business system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stable high-throughput asynchronous task processing method based on kafka

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The technical solution of the present invention will be specifically described below in conjunction with the accompanying drawings.

[0024] The present invention realizes a stable high-throughput asynchronous task processing method based on kafka, such as figure 1 As shown, the specific implementation steps are as follows:

[0025] 1. According to the configuration information of Kafka, initialize consumer-related parameters, such as the maximum number of acquisition tasks, topic information, number of threads, monitoring interval configuration, and timeout.

[0026] 2. Create a thread pool object and consumer object according to the parameter information, and search for the corresponding consumer in the spring container according to the name of "topic + KafkaServiceImpl" according to the configured topic information.

[0027] 3. Add the number of management threads to the monitoring task, and output the relevant monitoring information through the scheduled task.

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an asynchronous task processing method for realizing stable high throughput based on kafka. An asynchronous task mode based on kafka is adopted; in the processing mechanism for asynchronous tasks, a management thread is adopted to monitor various activity conditions of the working thread and is combined with the timing task to output the execution condition of the asynchronous task according to the frequency of the monitoring interval time; a timeout task processing mechanism and an abnormal task processing mechanism are provided, the monitoring task consumption situation, the statistics task execution time and the task completion detail situation are increased, parameters can be optimized and adjusted through the statistics situation, statistical information can be written into a log, and a user can analyze the performance through the log. According to the invention, the coupling of the system to the service system can be greatly reduced.

Description

technical field [0001] The invention relates to a stable high-throughput asynchronous task processing method based on kafka. Background technique [0002] The traditional asynchronous task technology is mainly to classify and insert tasks into the task record table or related queues according to the task type, and let idle threads execute this asynchronous task through the time in the task record table or related queues. An application-level asynchronous scheduling task (patent number: 201010217283.7) The system implementation process is roughly similar to the traditional asynchronous task technology. This technology stores the relevant information and result information of the asynchronous task through the data service device, and detects the work through a management thread. Threads and asynchronous task queues, and use this management thread to terminate overtime tasks and update their status. [0003] The implementation of the existing technology model is relatively cum...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30G06F9/48
CPCG06F11/302G06F11/3051G06F9/4812Y02D10/00
Inventor 胡玉煌吴浩
Owner 福建南威软件有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap