Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A stable and high-throughput asynchronous task processing method based on kafka

A technology with asynchronous tasks and high throughput, applied in the direction of electrical digital data processing, multi-programming devices, instruments, etc., can solve problems such as high coupling of business systems, achieve the effects of reducing coupling, improving functions, and easy expansion and upgrading

Active Publication Date: 2022-07-08
福建南威软件有限公司
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This mode is highly coupled with the business system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A stable and high-throughput asynchronous task processing method based on kafka

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] The present invention is a kafka-based asynchronous task processing method for achieving stable high throughput, such as figure 1 The specific implementation steps are as follows:

[0025] 1. According to the configuration information of kafka, initialize consumer-related parameters, such as the maximum number of tasks, topic information, number of threads, monitoring interval configuration, and timeout.

[0026] 2. Create a thread pool object and a consumer object according to the parameter information, and search for the corresponding consumer in the spring container according to the name of "topic + KafkaServiceImpl" according to the configured topic information (topic).

[0027] 3. Add the number of management threads to the monitoring task, and output monitoring-related information through timed tasks.

[0028] 4. Start the mana...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a kafka-based asynchronous task processing method for realizing stable high throughput. Using the asynchronous task mode based on kafka, for the processing mechanism of asynchronous tasks, the management thread is used to monitor the various activities of the worker threads and combined with the timing task to output the execution of the asynchronous task according to the frequency of the monitoring interval, providing processing for overtime tasks. Mechanism and abnormal task processing mechanism, increase monitoring task consumption, statistics task execution time, task completion details, and optimize and adjust parameters through statistics. This statistical information will be written into the log, and users can analyze performance through the log. The invention can greatly reduce the coupling between the system and the business system.

Description

technical field [0001] The invention relates to a kafka-based asynchronous task processing method for realizing stable high throughput. Background technique [0002] The traditional asynchronous task technology mainly classifies and inserts it into the task record table or related queue according to the task type, and allows idle threads to execute the asynchronous task according to the time in the task record table or related queue. An application-level asynchronous scheduling task (Patent No.: 201010217283.7) The process implemented by the system is generally similar to the traditional asynchronous task technology. This technology stores the relevant information and result information of the asynchronous task through a data service device, and detects the work through a management thread. Thread and asynchronous task queue, and use this management thread to terminate the timeout task and update its state. [0003] The prior art mode is cumbersome to implement. It is neces...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/30G06F9/48
CPCG06F11/302G06F11/3051G06F9/4812Y02D10/00
Inventor 胡玉煌吴浩
Owner 福建南威软件有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com