Real-time communication method and device in distributed system and distributed system

Abstract

The invention discloses a real-time communication method and device in a distributed system and the distributed system, and belong to the distributed technical field, the distributed system comprisesa main node and a working node, an algorithm task unit and a communication task unit are deployed on the working node, the method comprises the steps that the communication task unit executes initialization operation after being started and enters a monitoring state after the initialization operation is completed, and the initialization operation comprises initialization of a shared memory and network connection; the algorithm task unit is started after the communication task unit enters a monitoring state, and processes a calculation task initiated by the main node after being started; the algorithm task unit writes a calculation result of the calculation task into a shared memory; when the communication task unit monitors that the calculation result is stored in the shared memory, the communication task unit reads the calculation result; and the communication task unit returns a calculation result to the main node through network connection. According to the embodiment of the invention, real-time communication in the whole distributed system can be realized so as to ensure real-time output of the algorithm.

Description

technical field [0001] The invention relates to the field of distributed technology, in particular to a real-time communication method, device and distributed system in a distributed system. Background technique [0002] With the vigorous development of artificial intelligence technology, more and more artificial intelligence technologies have begun to be applied. In the process of implementing artificial intelligence algorithms, in addition to the performance of the algorithm itself, it also relies heavily on hardware computing resources. At present, a single device has multiple cores, and its performance is already very strong. However, when the computing power resources required by the project exceed the limit that a single device can provide, a distributed solution is usually adopted, that is, the resources required for computing power are allocated to different devices. Distributed solutions need to divide the system into nodes, each node undertakes certain computing t...

AI Technical Summary

Problems solved by technology

When real-time communication between these two nodes is required, if the amount of data concurrency is large, a large amount of IO will be generated. Such tasks requiring a large amount of IO are generally called IO-intensive tasks, and IO-intensive tasks generally do not consume Too much CPU, but interrupts the CPU frequently

[0003] In the process of realizing the present invention, the inventor found that: when the algorithm is deployed on the distributed system architecture to respond to the output in real time, the tasks on the multi-core nodes are both computing-intensive and network IO-intensive tasks. It will cause the algorithm calculation task to be frequently interrupted and seriously affect the efficiency of algorithm calculation, resulting in the entire distributed system not being able to respond to the output in real time

Images