Message stream processing method, device and equipment

Abstract

The embodiment of the invention provides a message stream processing method, device and equipment. The method comprises the following steps: determining a transmission relation link of a transmission message between transmission nodes based on identification information of the transmission message; determining a causal relationship link of different transmission messages at the transmission node based on the attribute information of the transmission messages; and constructing a message flow path diagram based on the transmission path link and the causal relationship link, wherein the message flow path diagram is used for indicating path information of each transmission message. After the transmission relation link and the causal relation link of each transmission message are determined, the transmission relation link and the causal relation link can be expanded forwards or back and forth at any transmission node, so that a path diagram of the whole message flow is efficiently constructed, and tracking and analysis of the transmission messages are facilitated.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a message stream processing method, apparatus and device. Background Art

[0002] With the continuous development of robotics technology, modern robotic systems often adopt complex distributed processing architectures, including distributed perception, motion planning, and decision making. ROS2 is a robot operating system that adopts a distributed architecture to achieve efficient and stable information transmission. As a high-level task scheduler that processes ROS2 messages, the ROS2 actuator directly affects the real-time performance of the system. In the prior art, the performance analysis and optimization of the ROS2 actuator mainly focus on the following aspects: communication delay, actuator performance, and information tracking and analysis. The above optimization method cannot effectively analyze the flow and causal relationship of messages in a distributed system, and it is difficult to detect and model the causal relationship related to complex applications. In addition, the above process requires modification of user code and will also result in significant runtime overhead. Summary of the invention

[0003] In view of this, the present application provides a message flow processing method, device and equipment to help solve the problem that messages are difficult to track and analyze in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a message flow processing method, including:

[0005] Determining a transmission relationship link of the transmission message between transmission nodes based on identification information of the transmission message;

[0006] Determining causal relationship links between different transmission messages at transmission nodes based on attribute information of the transmission messages;

[0007] A message flow path diagram is constructed based on the transmission relationship link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message.

[0008] In an optional embodiment, the transmission node includes a publisher or a subscriber, and the transmission relationship link includes a link between the publisher and the subscriber;

[0009] The determining, based on the identification information of the transmission message, the transmission relationship link of the transmission message between the transmission nodes includes:

[0010] A publisher or a subscriber of the transmission message is determined based on the identification information of the transmission message.

[0011] In an optional embodiment, the identification information includes one or more combinations of the following:

[0012] The subject name of the transmission message;

[0013] The publisher's global identifier G ID;

[0014] The timestamp of the transmitted message.

[0015] In an optional embodiment, the publisher's G ID includes: a pointer value, a process identification ID, or a host identification ID.

[0016] In an optional embodiment, determining the causal relationship link of different transmission messages at the transmission node based on the attribute information of the transmission message includes:

[0017] Determining direct causal relationship links between different transmission messages at a transmission node based on timestamps of the transmission messages;

[0018] An indirect causal relationship link between different transmission messages at a transmission node is determined based on the annotation information of the transmission message.

[0019] In an optional embodiment, the determining the indirect causal relationship link of different transmission messages at the transmission node based on the annotation information of the transmission message includes:

[0020] The periodic asynchronous links of different transmission messages at the transmission node are determined based on the annotation information of the transmission message, and the output message of the periodic asynchronous link is triggered by a timer callback function.

[0021] In an optional embodiment, the determining the indirect causal relationship link of different transmission messages at the transmission node based on the annotation information of the transmission message includes:

[0022] Partial synchronization links of different transmission messages at the transmission node are determined based on the annotation information of the transmission message, and the output messages of the partial synchronization links are triggered by a subscription callback function.

[0023] In a second aspect, an embodiment of the present application provides a message flow processing device, including:

[0024] A first determining module, configured to determine a transmission relationship link of the transmission message between transmission nodes based on identification information of the transmission message;

[0025] A second determination module, configured to determine causal relationship links between different transmission messages at a transmission node based on attribute information of the transmission message;

[0026] A construction module is used to construct a message flow path diagram based on the transmission path link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute any one of the methods described in the first aspect above.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any method described in the first aspect.

[0029] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes any method described in the first aspect.

[0030] By adopting the scheme provided in the embodiment of the present application, the transmission relationship link of the transmission message between each transmission node is determined based on the identification information of the transmission message; the causal relationship link of different transmission messages at the transmission node is determined based on the attribute information of the transmission message; and the message flow path diagram is constructed based on the transmission path link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message. After the transmission relationship link and the causal relationship link of each transmission message are determined, they can be extended forward or backward at any transmission node, thereby efficiently constructing the path diagram of the overall message flow, which is convenient for tracking and analyzing the transmission message. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 A flowchart of a message flow processing method provided in an embodiment of the present application;

[0033] Figure 2 An example schematic diagram of a message flow processing method provided in an embodiment of the present application;

[0034] Figure 3 An example schematic diagram of another message flow processing method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the structure of a message flow processing device provided in an embodiment of the present application;

[0036] Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0039] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0040] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0041] As a high-level task scheduler that processes ROS2 messages, the ROS2 executor directly affects the real-time performance of the system. The current performance analysis and optimization of the ROS2 executor mainly focus on the following aspects:

[0042] (1) Communication delay: There are many methods to improve the communication delay of ROS2, including evaluating the overall performance, analyzing the overhead of the Data Distribution Service (DDS) middleware, improving serialization technology, and reducing the overhead of inter-process communication (IPC). However, these methods often require modifying user code or incur significant runtime overhead, making them difficult to apply to actual systems.

[0043] (2) Executor performance: Existing technologies have also studied the performance issues of ROS2 executors, including analyzing their overhead, building scheduling models, and evaluating the performance of different executor designs. However, these methods cannot effectively analyze the flow and causal relationships of messages in distributed systems.

[0044] (3) Tracing and analysis: Low-overhead tracing techniques are widely used to collect execution information and for performance analysis. However, existing tracing and analysis methods have difficulty in effectively detecting and modeling causal relationships associated with complex applications.

[0045] In summary, the existing technologies have at least the following problems: (1) It is impossible to effectively analyze the flow and causal relationship of messages in a distributed system: the existing technologies can only evaluate the delay of a single node or a single message, and it is difficult to analyze the flow path and causal relationship of messages in the entire distributed system. (2) It is necessary to modify user code or cause significant runtime overhead: the existing technologies often require modifying user code or adding custom fields to track messages or detect causal relationships, which limits their scope of application. (3) It is difficult to detect and model causal relationships related to complex applications: the existing technologies have difficulty in detecting and modeling indirect causal relationships between messages in complex applications, such as causal relationships implemented through user-level code caching mechanisms.

[0046] In response to the above problems, an embodiment of the present application provides a message flow processing method, which can efficiently construct a path diagram of the overall message flow, facilitate the tracking and analysis of transmitted messages, and thereby improve the performance of the ROS2 executor.

[0047] Figure 1 The present invention provides a flowchart of a message flow processing method. The method can be applied to a processing device. Optionally, the processing device can refer to an electronic device with a logic processing function, such as a computer, a smart terminal, etc. Figure 1 As shown, the method may include:

[0048] Step 101: determine the transmission relationship link of the transmission message between transmission nodes based on the identification information of the transmission message.

[0049] Step 102: determining causal relationship links between different transmission messages at transmission nodes based on attribute information of the transmission messages.

[0050] Step 103: construct a message flow path diagram based on the transmission relationship link and the causal relationship link. The message flow path diagram is used to indicate the path information of each transmission message.

[0051] A transmission relationship link mainly refers to the relationship link between the publisher and subscriber of a transmission message. The publisher is the transmission node that publishes the transmission message, and the subscriber is the transmission node that receives the transmission message. The processing device can associate the publisher with (one or more) corresponding subscribers. The transmission relationship link is usually a one-to-many relationship, that is, the transmission message is usually generated by a single publisher, but can be received by multiple subscribers.

[0052] When a publisher publishes a transmission message, it will be accompanied by corresponding identification information. The processing device can determine the publisher and subscriber of the transmission message based on the identification information, that is, the transmission relationship link between the publisher and the subscriber. Based on the transmission relationship link, the processing device can clarify the transmission path of the transmission message between different transmission nodes.

[0053] The memory address of the transmission node is unique within a single process, but may conflict across processes or hosts. To avoid such conflicts, the identification information may consist of "the topic name of the transmission message", "the publisher's global identifier (GID)", and "the timestamp of the transmission message". Optionally, the identification information may be implemented by one or more combinations listed above. For example, the identification information may be implemented only through the GID. Alternatively, the identification information may be implemented by a combination of the GID and the timestamp of the transmission message. Alternatively, the identification information may also be implemented by the GID and the topic name of the transmission message, and the specific form is not limited. The GID is generated by the DDS layer, and the specific implementation method depends on the DDS implementation used. For example, the GID may include: a pointer value, a process identification ID, or a host identification ID.

[0054] In the embodiment of the present application, by establishing a transmission relationship link, the accuracy and uniqueness of the transmission message tracking can be ensured, which is helpful for analyzing the performance of ROS2 execution.

[0055] In an optional embodiment, the process of determining the causal relationship links of different transmission messages at the transmission node based on the attribute information of the transmission message by the processing device may include: (1) determining the direct causal relationship links of different transmission messages at the transmission node based on the timestamp of the transmission message; (2) determining the indirect causal relationship links of different transmission messages at the transmission node based on the annotation information of the transmission message.

[0056] Causality links are used to indicate the causal relationship between input messages and output messages at a transmission node, that is, input messages are used to generate output messages. Causality links include direct causal links and indirect causal links. Direct causal links indicate that the processing of input messages directly leads to the publication of output messages, and the causal relationship occurs below the Application Programming Interface (API) layer of ROS2. Indirect causal links occur at the user code level, not the API layer of ROS2. For example, a transmission node may cache multiple input messages and then generate output messages based on these messages in a timer callback. Based on causal links, the processing device can clearly identify the correspondence between input messages and output messages at each transmission node, ensuring the tracking and accuracy of transmission messages.

[0057] Direct causal relationship links can be determined based on the timestamps of transmitted messages. Specifically, the processing device can obtain the timestamps of transmitted messages, including: the start timestamp and end timestamp of the subscriber callback function, the timestamp when the publisher publishes a message, etc. Then, the causal relationship between different transmitted messages can be established based on the sequence of timestamps. For example, the input message I is received by the subscriber S and triggers the start timestamp X of the callback function, the output message O is published by the publisher P at the timestamp Y, and the end timestamp of the callback function of the input message I is Z. If X < Y < Z is satisfied, it can be determined that the input message I is the direct cause of the output message O. Because the output message O is published in the callback function for processing the input message I and both are executed on the same thread.

[0058] Indirect causal relationship links can be determined based on annotation information. During the system initialization phase, after the subscriber and publisher are created, the annotation information is added to the code through a one-time trace point. The annotation information can include the following: (1) Message link type: indicating the type of annotation, such as periodic asynchronous link or partial synchronous link. (2) Input subscriber list: listing all input subscribers that have an indirect causal relationship link with the output message. (3) Output publisher list: listing all output publishers triggered by the input subscribers. (4) List length: indicating the number of elements in the input subscriber and output publisher lists.

[0059] In a periodic asynchronous link, the transmission node can receive messages from multiple upstream transmission nodes and cache them. The timer periodically triggers the callback function, calculates the result using the last message in each cache, and publishes the message to multiple downstream transmission nodes. The output frequency and latency of the transmitted message depend on the period value of the timer. For example, multiple sensor nodes send data to the central processing node, and the central processing node periodically integrates this data and publishes the result.

[0060] In a partial synchronous link, the transmission node can receive messages from N upstream transmission nodes and cache them. When all N caches contain messages, the result is calculated in the subscription callback function and published to multiple downstream transmission nodes. The output frequency of the transmitted message depends on the rate of the input message. For example, multiple sensor nodes send data to the target tracking node, and the target tracking node needs all sensor data to calculate the target position and publish the result. Both of the above link types allow multiple input messages to trigger multiple output messages, thus supporting complex message flows and data processing logics.

[0061] Based on the determined transmission relationship links and causal relationship links, the processing device can construct a message flow path graph. The message flow path graph can be used to indicate the path information of each transmitted message, referring to Figure 2,Each circle may represent a transmission node. After each transmission message enters a transmission node, the transmission node may publish another transmission message. The relationship between different transmission messages may include: direct causal relationship link, periodic asynchronous link or partial synchronous relationship link.

[0062] The following example illustrates the first message (top): the transmission node publishes a message through the timer callback function, which is received by two subscribing nodes through message transmission. The second message (middle): the transmission node receives the first message through the subscription callback function and puts it into the cache. The third message (bottom): the node receives the second message through the subscription callback function and generates and publishes the fourth message in combination with the first message in the cache. There is a periodic asynchronous link between the first message and the fourth message because the first message is cached and used in the timer callback to generate the fourth message. There is a partial synchronous link between the second message and the fourth message because the second message and the first message in the cache are used together to generate the fourth message.

[0063] In response to the user's selection, the processing device can recursively expand forward and backward at any transmission node to build a message flow path diagram. Specifically, with the transmission node selected by the user as the initial node, based on the identification information, the processing device can build a transmission relationship link, that is, determine the publisher and subscriber of the transmission message. Based on the timestamp and annotation information, the processing device can build a causal relationship link. By combining each transmission node and the transmission message through the transmission message link and the causal relationship link, a complete message flow path diagram can be obtained.

[0064] The process of any two transmission nodes publishing or receiving messages can be referred to Figure 3 As shown in the figure, in transmission node 1, input message 1 and output message 1 are directly causally linked. After transmission node 1 receives input message 1, it triggers the release of output message 1. If the message is transmitted within the process, it is sent through the ROS C++ client. If the message is transmitted between processes, it is sent through the kernel. In output node 2, input message 1, input message 2, ..., input message M and output message 1 are periodic asynchronous links. Transmission node 2 receives M input messages and caches them, and releases output message 1 through timer callback. Input message 1, input message 2, ..., input message M and output message 2 are partially synchronous links. After all M input messages are received, transmission node 2 releases output message 2.

[0065] In the embodiment of the present application, the user can start to build the message flow path diagram at any position and select different initial nodes as needed. The recursive method can automatically detect all possible causal links to build a complete message flow diagram. The recursive method can effectively handle large and complex networks and improve construction efficiency.

[0066] Figure 4 A schematic diagram of the structure of a message flow processing device provided in an embodiment of the present application. Figure 4 As shown, the device may include:

[0067] The first determining module 410 is configured to determine the transmission relationship link of the transmission message between the transmission nodes based on the identification information of the transmission message.

[0068] The second determining module 420 is configured to determine the causal relationship links between different transmission messages at the transmission node based on the attribute information of the transmission message.

[0069] The construction module 430 is used to construct a message flow path diagram based on the transmission relationship link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message.

[0070] Corresponding to the above embodiments, the present application also provides an electronic device. Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device 500 may include: a processor 501, a memory 502 and a communication unit 503. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present application, it can be a bus structure or a star structure, and can also include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0071] The communication unit 503 is used to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0072] The processor 501 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs, instructions, and / or modules stored in the memory 502, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 501 can only include a central processing unit (CPU). In the embodiment of the present application, the CPU can be a single computing core or multiple computing cores.

[0073] The memory 502 is used to store the execution instructions of the processor 501. The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0074] When the execution instructions in the memory 502 are executed by the processor 501 , the electronic device 500 is enabled to execute part or all of the steps in the above embodiments.

[0075] In a specific implementation, the present application also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the message stream processing method provided by the present application. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0076] In a specific implementation, the present application also provides a computer program product, wherein the computer program product includes executable instructions, and when the executable instructions are executed on a computer, the computer executes part or all of the steps in each embodiment of the message flow processing method provided in the present application.

[0077] The embodiment of the present application also provides a non-temporary computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to execute the message flow processing method provided by the embodiment of the present application.

[0078] The above-mentioned non-temporary computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (Read On ly Memory; hereinafter referred to as: ROM), an erasable programmable read-only memory (Erasab leProgrammab le Read On ly Memory; hereinafter referred to as: EPROM) or flash memory, optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0079] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0080] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0081] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0082] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

Claims

1. A message stream processing method, characterized in that: include: Determining a transmission relationship link of the transmission message between transmission nodes based on identification information of the transmission message; Determining causal relationship links between different transmission messages at transmission nodes based on attribute information of the transmission messages; A message flow path diagram is constructed based on the transmission relationship link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message.

2. The method according to claim 1, characterized in that The transmission node includes a publisher or a subscriber, and the transmission relationship link includes a link between the publisher and the subscriber; The determining, based on the identification information of the transmission message, the transmission relationship link of the transmission message between the transmission nodes includes: A publisher or a subscriber of the transmission message is determined based on the identification information of the transmission message.

3. The method according to claim 2, characterized in that The identification information includes one or more of the following combinations: The subject name of the transmission message; The publisher's global identifier GID; The timestamp of the transmitted message.

4. The method according to claim 3, characterized in that The publisher's GID includes: a pointer value, a process ID or a host ID.

5. The method according to claim 1, characterized in that The determining, based on the attribute information of the transmission message, causal relationship links of different transmission messages at the transmission node comprises: Determining direct causal relationship links between different transmission messages at a transmission node based on timestamps of the transmission messages; An indirect causal relationship link between different transmission messages at a transmission node is determined based on the annotation information of the transmission message.

6. The method according to claim 5, characterized in that The determining, based on the annotation information of the transmission message, indirect causal relationship links between different transmission messages at the transmission node comprises: The periodic asynchronous links of different transmission messages at the transmission node are determined based on the annotation information of the transmission message, and the output message of the periodic asynchronous link is triggered by a timer callback function.

7. The method according to claim 5, characterized in that The determining, based on the annotation information of the transmission message, indirect causal relationship links between different transmission messages at the transmission node comprises: Partial synchronization links of different transmission messages at the transmission node are determined based on the annotation information of the transmission message, and the output messages of the partial synchronization links are triggered by a subscription callback function.

8. A message stream processing device, characterized in that: include: A first determining module, configured to determine a transmission relationship link of the transmission message between transmission nodes based on identification information of the transmission message; A second determination module, configured to determine causal relationship links between different transmission messages at a transmission node based on attribute information of the transmission message; A construction module is used to construct a message flow path diagram based on the transmission relationship link and the causal relationship link, and the message flow path diagram is used to indicate the path information of each transmission message.

9. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images