Load balancing method used in massive data multithread parallel processing

A data processing and parallel processing technology, applied in the field of computer networks, can solve the problems affecting system performance, inconsistent data processing time, idle threads in the data processing thread pool, etc. Effect

Inactive Publication Date: 2013-01-09
HOHAI UNIV
View PDF3 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] After the system receives the data, if it randomly or simply allocates the received data, due to the different allocation of threads to computer time slices and data content, the processing time of each data will be inconsistent, and the random allocation will cause the data processing thread pool to be limited. The thread is idle, and some threads discharge a large amount of data to be processed in the data area, which seriously affects the performance of the system and affects the concurrent processing capability of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Load balancing method used in massive data multithread parallel processing
  • Load balancing method used in massive data multithread parallel processing
  • Load balancing method used in massive data multithread parallel processing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] The technical scheme of the present invention is described in detail below in conjunction with accompanying drawing:

[0017] The idea of ​​the present invention is to allocate data reasonably to each data processing thread according to the relative idle rate of each data processing thread when distributing data processed in parallel by multiple threads, thereby improving the system's parallel processing and intensive calculation of massive data. Ability.

[0018] figure 1 It shows an example of multi-thread parallel data processing on the server side. In this example, when the process starts, the server starts a data receiving thread R, and creates a thread pool containing multiple data processing threads D, and a data distribution thread S, open up a section of dynamic storage area BM at the same time, and open up a dynamic storage area SM and a counter C with an initial value of 0 for each data processing thread D in the thread pool; the data receiving thread R is o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a load balancing method used in massive data multithread parallel processing. According to the method, a server processes received data by using a plurality of parallel data processing threads, and for each received new data packet, the server allocates the data processing thread according to the method which comprises the following steps of: judging whether the data processing threads in which the number C of the current to-be-processed data packets is equal to zero or the number n of processed data packets is equal to zero exist; if the data processing threads exist, the received new data packets are allocated to any one of the data processing threads; and if the data processing threads does not exist, the received new data packets are allocated to a data processing thread which has a highest relative idle rate. According to the method, reasonable data allocation for each data processing thread can be performed according to a relative idle rate, the capability of parallel processing and intensive calculation on massive data of a system can be obviously improved, the algorithm of the method is easy to implement and occupies a small quantity of resources.

Description

technical field [0001] The invention relates to multi-thread parallel processing of massive data, in particular to a load balancing method in multi-thread parallel processing of massive data, and belongs to the technical field of computer networks. Background technique [0002] In a large-scale deployment of terminal equipment access and massive data concurrent processing systems, the system receives and processes a large amount of data per unit time, and the system generates a new thread immediately after receiving the data to process the received data. Performance efficiency will be significantly reduced. This is because creating threads requires more system overhead, and frequent allocation and release of system resources will also take a lot of time. In order to try to reduce unnecessary overhead and calculation time of the system and improve the parallel processing capability of the system, Generally, it is carried out in the way of multi-threading and thread pool. Afte...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
Inventor 彭建华徐立中石爱业周惠郭宇燕高红民
Owner HOHAI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap