Message transmission method and device

Abstract

The invention provides a message transmission method and device. The method comprises the following steps: receiving an initial message tuple sent by a client through a receiving thread, obtaining a task number of the initial message tuple; determining a target actuator group based on the task number, transmitting the initial message tuple to a target executor group through a receiving thread; sequentially processing the initial message tuples through N actuators contained in the target actuator group, outputting the target message tuples corresponding to the initial message tuples by the Nthactuator in the target actuator group, and sending the target message tuples to a transmission queue of a sending thread; and determining the target message tuple received by the transmission queue asa to-be-transmitted message tuple through the sending thread, judging whether the to-be-transmitted message tuple stored in the transmission queue meets a preset transmission condition or not, and ifso, transmitting the to-be-transmitted message tuple in the transmission queue to the server. Resources of the processor can be saved, so that the working efficiency of the processor is improved.

Description

technical field [0001] The invention relates to the technical field of streaming data processing, in particular to a message transmission method and device. Background technique [0002] With the development of science and technology, streaming computing frameworks have also been more and more widely used. Streaming computing is a computing method different from batch computing. Streaming computing puts more emphasis on computing data flow and low latency. Therefore, for latency-sensitive distributed big data applications, streaming computing frameworks are usually used Process the data flow of such applications, so as to ensure the stable operation of the applications and improve the user experience. [0003] At present, the streaming computing framework generally uses the Java Virtual Machine (JavaVirtualMachine, JVM) as the underlying processing environment of the computing node. Therefore, when the two computing nodes of the streaming computing system interact with each...

AI Technical Summary

Problems solved by technology

[0005] However, in the case of a large number of small message interactions among the various working processes in the streaming computing system, using the existing asynchronous communication architecture based on the TCP / IP protocol for communication will cause the processor to perform frequent context switching, and , when the computing node is processing the message of the application program based on the TCP / IP protocol, the buffer copy is frequently performed between the working thread of the application program and the operating system, as well as the buffer copy of the protocol stack inside the operating system, resulting in a large amount of processing server resources are consumed

Images