Method and apparatus for parallel processing heteromorphism data

Abstract

The present invention provides a parallel processing method and processing device of heteromorphism data. The method includes: receiving any one processor of data of sharing resource exclusivity access, output the data to an appointed buffer queue; inputting the data of the pointed buffer queue to a multiple-processor according to queuing or speed limiting mode again, ensuring not more than one processor process receive and process. The device includes: a buffer unit connecting with the multiple-processor, containing a transmission module, the buffer queue is used for buffering data transmitted by any processor and relating to the sharing resource exclusivity access, the transmission module is used for re-inputting the data of the buffer queue according to the queuing or speed limiting mode, ensuring not more than one processor process receive and process. The invention avoids the competition of sharing resource, ensuring the performance of parallel processing, and being easy to realize.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a parallel processing method and processing device for heteromorphic data. Background technique [0002] In order to improve the performance of data processing, data parallel processing is often used in the prior art, such as using multiple processing engines, or using multiple processors, or using multi-core processors, etc., so as to provide support for heavy calculations. For the convenience of description, we collectively refer to multi-processing engines, multi-processors, and multi-cores of processors as multi-processors, but those of ordinary skill in the art can understand that this parallel processing method may have different implementation mechanisms that are not limited to the above examples . [0003] However, in the process of data parallel processing, the processed data often has heteromorphism. For example, some data processing involves exclusive access t...

AI Technical Summary

Problems solved by technology

[0003] However, in the process of data parallel processing, the processed data often has heteromorphism. For example, some data processing involves exclusive access to shared resources while other data does not. The referred to exclusive access to shared resources refers to the need to Exclusive use of shared resources. In this case, if other processors perform related operations on shared resources at the same time, it will cause processing conflicts and data errors of shared resources.

[0004] In the process of processing heterogeneous data, the purely parallel processing method may face the problem of sharp decline in processing performance: in order to avoid exclusive access conflicts of different processors for shared resources, the command must be used in parallel processing mode Card or lock mechanism - only one processor that has obtained a token or lock can have exclusive access to shared resources, while other processors that need exclusive access to shared resources have to enter a suspended state until they obtain tokens; Obviously, these processors actually perform serial access processing of shared resources, and the suspended processor can no longer process other data at the same time.

In this case, even if the processing speed of the processor for the NAT follow-up packet is much higher than the processing speed for the NAT first packet, only part of the unsuspended processors can execute the processing of the NAT follow-up packet, or even In extreme cases, all processors are occupied by the first NAT packet, and the subsequent NAT packets can only wait for a processor to be released, that is, the waiting time for subsequent NAT data packets is at least dozens or even hundreds of times longer than its own processing time. Therefore, NAT forwarding performance will drop sharply

[0009] In addition, the prior art defect that needs to be pointed out is that if a lock or token mechanism is used to process data in a parallel processing link (pipeline mode or non-pipeline mode), then it is necessary to repeatedly perform token transfer on different processors in this link Or lock / unlock processing, which is extremely complex and expensive to develop

At the same time, the complex locking / unlocking process can easily lead to processing vulnerabilities. If the locked processor is not unlocked in time due to mistakes or program flaws, the entire processing process will be paralyzed

Images