Data acquisition method and system
Patent Information
- Application Number
- CN202611141031.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-29
- Publication Date
- 2026-10-23
AI Technical Summary
[0003]然而，部分中间件缺乏动态发现能力，消息主题关系依赖静态配置，导致消息查看设备往往需通过预定义映射解析元数据；当消息类型或字段变化时，消息查看设备适配需人工更新，难以在运行时保持兼容，维护成本和调试复杂度较高
[0038]本申请实施例提供的数据获取方法及系统，通过向至少一个中间件节点发送元数据请求，并接收至少一个中间件节点返回的包括消息主题的标识信息和序列化结构定义信息在内的元数据信息，能够使消息查看设备在无动态拓扑发现能力的静态配置中间件环境下动态掌握消息主题及其数据结构，进而根据元数据信息对原始消息数据进行反序列化，得到可读消息内容，降低对预定义映射和人工适配更新的依赖，提高消息查看设备的兼容性、维护效率和消息查看的灵活性。
Smart Images
Figure CN122963834A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving middleware communication, and in particular to a data acquisition method and system. Background Technology
[0002] In the development of intelligent driving systems, middleware is typically used to enable message topic communication between modules, and a matching message viewing device is used to obtain topic metadata and view message content.
[0003] However, some middleware lacks dynamic discovery capabilities, and message topic relationships rely on static configuration, which means that message viewing devices often need to parse metadata through predefined mappings. When message types or fields change, message viewing devices need to be manually updated to adapt, making it difficult to maintain compatibility at runtime, resulting in high maintenance costs and debugging complexity.
[0004] Therefore, how to dynamically acquire message topic metadata and improve message viewing device compatibility in a middleware environment lacking a dynamic discovery mechanism has become a technical problem that needs to be solved. Summary of the Invention
[0005] This application provides a data acquisition method and system to solve the aforementioned technical problems. This solution addresses message viewing scenarios in statically configured middleware environments lacking dynamic topology discovery capabilities. By enabling message viewing devices to acquire identifier information and serialization structure definition information related to the message topic from middleware nodes, and then acquiring the corresponding message data based on the acquired information, the runtime adaptability of message viewing devices to different middleware nodes and message topics is improved when message types or fields change.
[0006] In a first aspect, embodiments of this application provide a data acquisition method, applied to a message viewing device adapted to statically configured middleware nodes without dynamic topology discovery capabilities, the method comprising:
[0007] Send a metadata request to at least one middleware node;
[0008] Receive metadata information returned by at least one middleware node. The metadata information includes at least the identifier information of the message topic and the serialization structure definition information in the corresponding middleware node.
[0009] Based on the metadata information, obtain the original message data corresponding to the message topic.
[0010] In one possible embodiment, after obtaining the readable message content, the method further includes: outputting the readable message content to an interactive interface for display.
[0011] In one possible embodiment, after receiving metadata information returned by at least one middleware node, the method further includes: merging the metadata information to generate a full list of message topics.
[0012] In one possible embodiment, the interactive interface includes a first view and a second view. The first view is used to display the identification information of each message topic, and the second view is used to display the complete content of the target message topic in response to the user's selection of the target message topic.
[0013] In one possible embodiment, the identification information includes at least one of the following: message topic name, message topic information, and message type name.
[0014] Secondly, embodiments of this application provide a method for obtaining message topic metadata, applied to statically configured middleware nodes without dynamic topology discovery capabilities, the method comprising:
[0015] During the instantiation of the publish role and / or subscribe role, obtain the serialization structure definition information corresponding to the message topic;
[0016] Receive messages to view the metadata requests sent by the device;
[0017] Based on the metadata request, the serialization structure definition information and the identification information corresponding to the message topic are encapsulated to generate metadata information;
[0018] Send metadata information to the message viewing device.
[0019] In one possible embodiment, obtaining the serialization structure definition information corresponding to the message topic includes:
[0020] Add dynamic libraries to the communication architecture of middleware nodes;
[0021] During the instantiation of published and / or subscribed roles, serialization structure definition information is obtained through dynamic libraries.
[0022] In one possible embodiment, after obtaining the serialization structure definition information through the dynamic library, the method further includes: converting the serialization structure definition information into a serialized data stream format and caching it in the local storage space of the middleware node.
[0023] Thirdly, embodiments of this application provide a data acquisition system, including a message viewing device and a middleware node. The middleware includes at least one middleware node. The message viewing device is used to execute the data acquisition method provided in the first aspect, and the middleware node is used to execute the data acquisition method provided in the second aspect.
[0024] Fourthly, embodiments of this application provide a message viewing device, including:
[0025] The first sending module is used to send metadata requests to at least one middleware node;
[0026] The first receiving module is used to receive metadata information returned by at least one middleware node. The metadata information includes at least the identifier information of the message topic and the serialization structure definition information in the corresponding middleware node.
[0027] The first acquisition module is used to deserialize the original message data based on metadata information to obtain readable message content.
[0028] Fifthly, embodiments of this application provide a middleware node, including:
[0029] The second acquisition module is used to acquire the serialization structure definition information corresponding to the message topic during the instantiation of the publishing role and / or the subscription role;
[0030] The second receiving module is used to receive metadata requests sent by the message viewing device;
[0031] The processing module is used to encapsulate the serialization structure definition information and the identification information corresponding to the message topic according to the metadata request, and generate metadata information;
[0032] The second sending module is used to send metadata information to the message viewing device.
[0033] Sixthly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0034] The memory stores the instructions that the computer executes;
[0035] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above, or causing the processor to perform the second aspect and / or various possible implementations of the second aspect as described above.
[0036] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect, or, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect.
[0037] Eighthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect, or when executed by a processor, implements the second aspect and / or various possible implementations of the second aspect.
[0038] The data acquisition method and system provided in this application embodiment, by sending a metadata request to at least one middleware node and receiving metadata information returned by at least one middleware node, including the identifier information of the message topic and the serialization structure definition information, enables the message viewing device to dynamically grasp the message topic and its data structure in a static configuration middleware environment without dynamic topology discovery capabilities. Then, based on the metadata information, the original message data is deserialized to obtain readable message content, reducing the dependence on predefined mapping and manual adaptation updates, and improving the compatibility, maintenance efficiency and message viewing flexibility of the message viewing device. Attached Figure Description
[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0040] Figure 1 A schematic diagram of the structure of a data acquisition system provided in this application;
[0041] Figure 2 Flowchart of the data acquisition method provided in this application Figure 1 ;
[0042] Figure 3 This is a schematic diagram of the architecture of a message viewing device provided in an embodiment of this application;
[0043] Figure 4 Flowchart of the data acquisition method provided in this application Figure 2 ;
[0044] Figure 5 Flowchart of the data acquisition method provided in this application Figure 3 ;
[0045] Figure 6 Signaling diagram of a data acquisition method provided in this application;
[0046] Figure 7 A schematic diagram of the message viewing device provided in this application;
[0047] Figure 8 A schematic diagram of the middleware node provided in this application;
[0048] Figure 9A schematic diagram of the structure of the electronic device provided in this application.
[0049] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0051] Intelligent driving system development typically relies on middleware to achieve message topic communication between modules such as perception, planning, and control. Debuggers need to use message viewing devices to obtain topic metadata and view the readable message content corresponding to the original message data in order to verify whether the data interaction between each node is correct.
[0052] In some middleware environments, the communication relationships between nodes are predetermined by static configuration, lacking dynamic topology discovery capabilities during system operation. In such scenarios, message viewing devices typically connect to multiple middleware nodes as independent tools to observe and analyze message topics on each node.
[0053] In existing solutions, message viewing devices often rely on the dynamic discovery capabilities provided by the middleware itself to determine the correspondence between nodes, topics, and message types, and further parse the raw message data based on pre-defined structural information. In an environment with a dynamic discovery mechanism, message viewing devices can easily synchronize topic metadata and display message content; however, in statically configured middleware, due to the lack of such capabilities, message viewing devices typically need to pre-establish a mapping relationship between message topics and message structure definitions, or adapt to different node message types through manual maintenance.
[0054] The drawback of this technical solution is that once the message type, field definition, or topic instance changes, the original mapping relationship may become invalid. The message viewing device may struggle to obtain the latest metadata in a timely manner, thus failing to accurately parse the corresponding original message data. Especially during the development of intelligent driving, topics related to perception results, planned trajectories, or control commands are frequently adjusted with iterations. If the message viewing device still relies on static adaptation, repeated manual updates are required, leading to a break in the debugging process, increased maintenance costs, and reduced runtime compatibility.
[0055] Therefore, how to enable message viewing devices to dynamically obtain available metadata from message topics and obtain readable message content corresponding to the original message data in a static configuration middleware environment without dynamic topology discovery capabilities has become an urgent technical problem to be solved.
[0056] In view of this, a data acquisition method is provided for message viewing devices that adapt to statically configured middleware nodes without dynamic topology discovery capabilities. After the message viewing device sends a metadata request to at least one middleware node, it receives metadata information returned by the middleware node. The metadata information includes at least the identifier information of the message topic and the serialization structure definition information in the corresponding middleware node. Based on the metadata information, the original message data is deserialized to obtain readable message content, thereby improving the message viewing device's adaptability to changes in message topics in statically configured middleware scenarios.
[0057] Figure 1 This is a schematic diagram of the structure of a data acquisition system provided in this application. Figure 1 As shown, it includes a message viewing device and multiple middleware nodes. The message viewing device and each middleware node are connected for communication.
[0058] Each middleware node integrates a server-side module (ToolsServer); the message viewing device integrates a client-side module (ToolsClient). The server-side module and the client-side module communicate via a request-response method, with the client actively requesting metadata information corresponding to the message topic from each middleware node.
[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0060] Figure 2 Flowchart of the data acquisition method provided in this application Figure 1 This is used in message viewing devices that adapt to statically configured middleware nodes that lack dynamic topology discovery capabilities. For example... Figure 2 As shown, the method includes:
[0061] S201. Send a metadata request to at least one middleware node.
[0062] In this application, the data acquisition method is implemented by a message viewing device, which is deployed in a static configuration middleware environment adapted to lack dynamic topology discovery capabilities, and establishes a usable communication connection with at least one static configuration middleware node. A static configuration middleware node refers to a node whose inter-node communication relationships are predetermined by configuration files, startup parameters, or preset routing information, and which does not rely on a dynamic topology discovery mechanism to publish topic relationships during system operation. The message viewing device initiates metadata requests, receives metadata information, and parses the original message data to obtain readable message content. The metadata request is used to request the identifier information and serialization structure definition information of the current message topic from the target middleware node.
[0063] In practice, the message viewing device pre-stores the access information of at least one middleware node. This access information includes the node address, communication port, transmission protocol identifier, and request interface identifier. After startup, the message viewing device first establishes a point-to-point communication channel with each middleware node based on the access information. This communication channel can be a socket-based long connection or a request-response link based on the middleware control channel.
[0064] In this application, a message topic is a logical channel in a middleware node that carries business data communication. It can correspond to runtime data objects such as perception results, planned trajectories, positioning status, or control commands. Identification information is used to distinguish a message topic. When sending metadata requests, the message viewing device does not need to maintain a complete mapping relationship between topics and message structures in advance. Instead, it obtains the currently effective identification information and serialization structure definition information of the node by actively requesting it from the node.
[0065] This step involves the message viewing device actively establishing a request link and sending a metadata request to the static configuration middleware node. This enables the formation of a detection entry point for publishing roles and / or subscribing roles in a system environment that originally lacked dynamic topology discovery capabilities. This establishes the prerequisites for receiving metadata information and obtaining the corresponding original message data.
[0066] In one possible embodiment, the metadata request employs a combination of periodic and event-triggered sending methods. In the periodic triggering method, the message viewing device repeatedly sends metadata requests to the target node at preset time intervals, which can be set to seconds or minutes depending on the debugging scenario. In the event-triggered method, when a node restart, configuration file update, connection re-establishment, or user switching viewing objects is detected, the message viewing device immediately resends the metadata request. Through this process, the message viewing device can continuously monitor the changing status of the node's current message topic and trigger subsequent metadata synchronization processes. It should be understood that the above example is merely illustrative and not limiting; other equivalent request sending mechanisms can also be used to complete metadata requests to at least one middleware node.
[0067] S202. Receive metadata information returned by at least one middleware node. The metadata information includes at least the identifier information of the message topic and the serialization structure definition information in the corresponding middleware node.
[0068] In this application, after the middleware node receives a metadata request from the message viewing device, it reads its own message topic set and generates corresponding metadata information, which is then returned to the message viewing device. The metadata information carries a description of the message topics that the middleware node can identify and subsequently retrieve data from. The identification information identifies the message topic, while the serialization structure definition information describes the data field composition, field order, field type, nesting relationship, array dimensions, length constraints, or version information of the message topic. The serialization structure definition information exists in a transmissible serialized form, which the message viewing device can directly cache, deserialize, or convert into an internal structure description object upon receipt.
[0069] In practice, the message viewing device listens for response data from each middleware node on the communication channel. Upon receiving a response message, it first performs a message integrity check based on the node identifier, response type, length field, and checksum field in the message header, and then parses messages that pass the checksum. If a node returns metadata information for multiple message topics, it can return the information in a list structure, with each list item corresponding to a message topic. Each list item must contain at least topic identifier information and the serialization structure definition information corresponding to that topic. After parsing, the message viewing device associates the node identifier with the topic identifier and writes the serialization structure definition information of each topic into the local metadata cache. The cache can be indexed by node dimension, topic dimension, or topic type dimension to support subsequent fast retrieval.
[0070] In one possible embodiment, the serialized structure definition information can be returned as a serialized byte stream. After receiving it, the message viewing device decodes it according to the agreed encoding to obtain the field name, field type, field offset, or nested object level. In another embodiment, the serialized structure definition information is returned as a serialized and encapsulated middleware internal message definition text. The message viewing device performs syntactic analysis on the text definition and generates a local message structure tree.
[0071] In this application, receiving metadata information returned by at least one middleware node includes not only receiving a single response, but also receiving and recording metadata information from multiple nodes. The message viewing device can establish an independent receiving session for each node, receiving responses in each session and recording the topic metadata returned by the corresponding node. If different nodes return the same topic but with different serialization structure definitions, the message viewing device can distinguish and record them based on the node identifier; if the same node repeatedly returns metadata for the same topic and the serialization structure definition remains unchanged, the message viewing device only updates the timestamp. In this way, the message viewing device can form topic recognition across multiple nodes in a statically configured middleware environment, without relying on the middleware itself to provide dynamic discovery results.
[0072] This step transforms the manually maintained topic structure mapping into runtime factual metadata returned by the nodes by directly receiving the current message topic's identification information and serialization structure definition information from the middleware nodes. Based on this, the message viewing device can identify which topics actually exist, how the data structure of each topic is organized, and whether topic definitions have changed across different nodes. This process addresses the lack of dynamic topology discovery capabilities in statically configured middleware by providing an equivalent metadata acquisition path, enabling the message viewing device to update its local understanding and trigger subsequent data parsing in a timely manner when faced with changes in topic instances, field definitions, or message versions. It should be understood that the above example is merely illustrative and not limiting; other equivalent implementation methods can be adopted for the metadata information's format, return timing, and caching organization without deviating from this technical characteristic.
[0073] S203. Based on the metadata information, deserialize the original message data to obtain readable message content.
[0074] In this application, the raw message data is the message payload actually sent by the middleware node, which is typically transmitted in the form of byte streams, frame data, or serialized message packets defined by the middleware. The message viewing device can complete the reception and storage of the raw message data before this step.
[0075] After receiving metadata information returned by at least one middleware node, the message viewing device deserializes the original message content based on the identification information therein to obtain the readable message content corresponding to the original message data. In practice, deserialization can be performed by parsing logic corresponding to the serialization structure definition information. This parsing logic can be pre-loaded into the message viewing device or dynamically generated based on the metadata information. This ensures that even when the data structure corresponding to the message topic changes, the field parsing can still be completed according to the latest serialization structure definition information to obtain the corresponding readable message content. The readable message content may include field names, field values, and their associated structural information, and can be organized as structured text, tree data, or tabular data for subsequent display.
[0076] This application provides a data acquisition method, comprising: sending a metadata request to at least one middleware node; receiving metadata information returned by at least one middleware node, wherein the metadata information includes at least the identifier information and serialization structure definition information of the message topic in the corresponding middleware node; and deserializing the original message data according to the metadata information to obtain readable message content. In this application, the message viewing device, without relying on the dynamic topology discovery capability of statically configured middleware, actively requests metadata information from middleware nodes and obtains readable message content by deserializing the original message data based on the metadata information. Therefore, when the topic instance, message type, and field definition change, the message viewing device can update its local topic directory and structure description according to the node's returned results, thereby continuing to obtain readable message content consistent with the current node state.
[0077] Figure 3 A schematic diagram of the architecture of a message viewing device provided in an embodiment of this application. See also... Figure 3 The message viewing device 11 includes a lower-level data acquisition layer 111 and an upper-level interface presentation layer 112. The lower-level data acquisition layer 111 is used to dynamically acquire metadata information and raw message data from the system, while the upper-level interface presentation layer 112 includes an interactive command-line rendering component for visually displaying readable message content and user interaction responses.
[0078] based on Figure 3 The data acquisition method provided in this application embodiment also includes steps for visualizing readable message content and responding to user interactions. See also Figure 4 , Figure 4 Flowchart of the data acquisition method provided in this application Figure 2 The method, applicable to message viewing devices adapted to statically configured middleware nodes lacking dynamic topology discovery capabilities, includes:
[0079] S401. Send a metadata request to at least one middleware node.
[0080] S402. Receive metadata information returned by at least one middleware node.
[0081] The metadata information includes at least the identifier information of the corresponding message topic in the middleware node and the serialization structure definition information (Proto Descriptor). Furthermore, the identifier information includes at least one of the following: message topic name (name), message topic information (topic_info), and message type name (message_type).
[0082] In practice, metadata information can be returned by middleware nodes when responding to metadata requests. Identification information can be directly encapsulated in the topic description object, or it can be formed by combining multiple fields. The message topic name, message topic information, and message type name can exist individually, or any two or three of them can exist simultaneously, to adapt to the differences in the granularity of metadata return by different nodes.
[0083] S403. Merge the metadata information to generate a full list of message topics.
[0084] The full message topic list is used to aggregate message topic information returned by multiple middleware nodes and form a unified viewing list for subsequent display and selection. The merge process is used to combine and organize scattered metadata information into a unified set of message topics.
[0085] In actual processing, after receiving metadata information returned by at least one middleware node, the message viewing device first performs association matching on the message topic identifier, message topic name, or message type name in each metadata information. Then, records with the same topic meaning are merged into the same topic item, and duplicate items are deduplicated. For metadata information from different middleware nodes but with the same topic name, consistent message type, or a mapping relationship between identifiers, the device merges them into the same topic record, while retaining the corresponding node source information so that the middleware node to which the topic belongs can be traced during subsequent viewing. After merging, the device generates a full list of message topics covering all acquired message topics and caches this list in the local storage unit or temporary memory area for unified display in the interactive interface.
[0086] By merging metadata information and generating a full list of message topics, the message viewing device can provide a data foundation for forming a unified topic view in a statically configured middleware environment. Subsequently, topic selection and viewing can be completed without comparing the discrete return content of each node, thereby improving the uniformity of topic retrieval and providing a stable index foundation for the acquisition and parsing of raw message data.
[0087] S404. Based on the metadata information, deserialize the original message data to obtain readable message content.
[0088] Optionally, raw message data can be obtained by: establishing a data receiving channel for the message topic; and obtaining the raw message data published by the middleware node to the corresponding message topic through the data receiving channel.
[0089] S405. Output readable message content to the interactive interface for display.
[0090] The interactive interface receives and visualizes readable message content. After deserialization, the message viewing device transmits the parsed readable message content to the interactive interface, which then renders it according to preset display components. The interactive interface can display field names and values, or expand the display according to the message structure hierarchy, allowing users to directly view the complete semantic content corresponding to the original message data. When a user selects a target message topic, the device can immediately output the corresponding readable message content to the interactive interface; as the message topic is continuously updated, the interactive interface can also synchronously refresh the display results to keep the displayed content consistent with the currently received data.
[0091] Optionally, the interactive interface includes a first view and a second view. The first view is used to display the identification information of each message topic, and the second view is used to display the complete content of the target message topic in response to the user's selection of the target message topic.
[0092] A first view can be generated based on a complete list of message topics. This first view serves as an overview area for message topics, displaying the list of topics, topic names, topic types, topic numbers, or other identifying information to distinguish different topic messages, allowing users to quickly browse multiple topics within the same interface. The first view can be a display area in the interactive interface, implemented using a list control, tree control, tab control, or table control. In practical applications, other models can also be selected for this component; this application does not impose any limitations on this.
[0093] In practical implementation, the first view and the second view can be arranged in different display areas within the same window. The first view is located on the left or top to display summary information of the topic, while the second view is located on the right or bottom to display detailed content. Alternatively, they can be presented separately through page switching. In the first view, after the user selects a target message topic, the interface control logic passes the topic identifier to the content query module, which then extracts the corresponding complete content based on the identifier and renders it to the second view. For static configuration middleware scenarios, the topic identifier information displayed in the first view can come from the merged result after receiving metadata information, while the second view displays the original message data associated with the merged result, thus enabling message topics from different nodes to be located and viewed in a unified interface.
[0094] This interactive interface separates the overview information of the message topic from the full content through a layered display. Users can quickly locate the target topic in the first view and then view the full content of that topic in the second view, thus achieving linkage between topic browsing and content viewing. Since the second view only displays the corresponding topic content after the selection action is triggered, the interface displays more concentrated content, making it easier to switch between multiple topics and maintain consistency with the metadata retrieval and original message display process.
[0095] Figure 5 Flowchart of the data acquisition method provided in this application Figure 3 This method, applied to statically configured middleware nodes lacking dynamic topology discovery capabilities, includes:
[0096] S501. During the instantiation of publishing roles and / or subscribing roles, obtain the serialization structure definition information corresponding to the message topic.
[0097] In this application, the statically configured middleware node is the target node running in a middleware environment without dynamic topology discovery capabilities. The serialization structure definition information describes the data organization method of the message subject during transmission, and includes at least one or more of the following: field name, field type, field order, hierarchical relationship, array length description, nested structure description, and serialization encoding rules. This enables subsequent message viewing devices that receive the information to identify message content boundaries and establish field-level mapping relationships based on the structure definition.
[0098] In one possible implementation, obtaining the serialization structure definition information corresponding to the message topic includes: adding a dynamic library to the communication architecture of the middleware node; and obtaining the serialization structure definition information through the dynamic library during the instantiation of the publish role and / or subscribe role.
[0099] In practical implementation, the code implementation in middleware with dynamic discovery mechanisms can be referenced. The logic for obtaining serialization structure definition information can be extracted and encapsulated into an independent dynamic library. This dynamic library is added to the communication architecture of the middleware nodes provided in this application. Each middleware node's binary file needs to link this dynamic library during compilation. Upon receiving the trigger signal for the instantiation of the publishing role and / or subscribing role, the dynamic library extracts the serialization structure definition information from a pre-defined message type description file, reflection information table, or compile-time generated structure metadata based on the message topic name, type identifier, message descriptor, or instance context. This information is then returned to the middleware node for subsequent encapsulation, caching, or external output.
[0100] With this approach, the serialization structure definition information of the message topic can be extracted in a timely manner and associated with the message topic instance when the publishing role and / or subscription role are instantiated. This allows the middleware node to form a usable metadata foundation in a static configuration environment, reducing the reliance on pre-maintained mapping relationships and improving adaptability when the topic structure changes.
[0101] Optionally, after obtaining the serialization structure definition information through the dynamic library, the method further includes: converting the serialization structure definition information into a serialized data stream format and caching it in the local storage space of the middleware node.
[0102] In actual processing, after obtaining the serialization structure definition information, the dynamic library can encapsulate it into a preset serialized data stream format (such as the binary format of Protocol Buffer), and retain the identification information, version information, and verification information corresponding to the message topic. Subsequently, the middleware node writes the serialized data stream format to its local storage space (such as the node's memory cache or file system) based on the identifier, and establishes an index mapping according to the topic identifier, so that when a metadata request for the same topic is received later, the corresponding cached content can be directly located.
[0103] This approach allows serialized structure definition information to be transformed into a unified data stream representation and resided locally on the node after being retrieved from the dynamic library. The node does not need to repeatedly execute the structure parsing and external loading process each time, and can quickly access the cached content during subsequent metadata responses. Therefore, the structure definition corresponding to the message topic can be stably stored on the static configuration middleware node side, supporting subsequent fast reading and encapsulation.
[0104] S502, Receive message to view the metadata request sent by the device.
[0105] S503. Based on the metadata request, encapsulate the serialization structure definition information and the identification information corresponding to the message topic to generate metadata information.
[0106] In this application, the identification information is used to establish a correspondence between a certain structure definition and a specific message topic, and may include at least one or more of the following: message topic name, message topic information, and message type name. In specific implementation, serialized structure definition information can be read from local storage space and encapsulated with the identification information to generate metadata information.
[0107] S504. Send metadata information to the message viewing device.
[0108] In this application, after the metadata information is generated, it is returned to the message viewing device by the middleware node through the established communication channel with the message viewing device.
[0109] On the message viewing device side, after receiving metadata information, an index table can be built based on the identification information, and corresponding deserialization rules can be generated according to the serialization structure definition information. Subsequently, the message viewing device uses the corresponding serialization structure definition information to complete field-level deserialization processing of the original message data, and renders and outputs the parsed field content in the form of tables, tree structures, or field paths.
[0110] This application provides a method for obtaining message topic metadata, including: obtaining serialization structure definition information corresponding to the message topic during the instantiation of a publishing role and / or a subscription role; receiving a metadata request sent by a message viewing device; encapsulating the serialization structure definition information and the identification information corresponding to the message topic according to the metadata request to generate metadata information; and sending the metadata information to the message viewing device, so that the message viewing device can still dynamically obtain available metadata information in a static configuration middleware environment without dynamic topology discovery capabilities, and obtain the corresponding readable message content accordingly.
[0111] Figure 6 This application provides a signaling diagram for a data acquisition method, applicable to a data acquisition system including message viewing devices and middleware nodes. For example... Figure 6 As shown, the method includes:
[0112] S601. During the instantiation of the publish role and / or subscribe role, the middleware node obtains the serialization structure definition information corresponding to the message topic.
[0113] S602, View the device sending metadata requests to the middleware node via messages.
[0114] S603, the middleware node receives messages and views the metadata requests sent by the device.
[0115] S604. The middleware node encapsulates the serialization structure definition information and the identification information corresponding to the message topic according to the metadata request, and generates metadata information.
[0116] S605, the middleware node sends metadata information to the message viewing device.
[0117] S606. Message viewing: The device receives metadata information returned by the middleware node.
[0118] S607. The middleware node deserializes the original message data based on the metadata information to obtain readable message content.
[0119] Figure 7 A schematic diagram of the message viewing device provided in this application, such as Figure 7 As shown, the message viewing device 70 provided in this embodiment includes:
[0120] The first sending module 71 is used to send metadata requests to at least one middleware node;
[0121] The first receiving module 72 is used to receive metadata information returned by at least one middleware node. The metadata information includes at least the identification information of the message topic and the serialization structure definition information in the corresponding middleware node.
[0122] The first acquisition module 73 is used to deserialize the original message data based on metadata information to obtain readable message content.
[0123] In one possible embodiment, the message viewing device 70 is also used to: output readable message content to an interactive interface for display.
[0124] In one possible embodiment, the message viewing device 70 is also used to: merge metadata information to generate a full list of message topics.
[0125] In one possible embodiment, the interactive interface includes a first view and a second view. The first view is used to display the identification information of each message topic, and the second view is used to display the complete content of the target message topic in response to the user's selection of the target message topic.
[0126] In one possible embodiment, the identification information includes at least one of the following: message topic name, message topic information, and message type name.
[0127] The message viewing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0128] Figure 8 A schematic diagram of the middleware node provided in this application is shown below. Figure 8 As shown, the middleware node 80 provided in this embodiment includes:
[0129] The second acquisition module 81 is used to acquire the serialization structure definition information corresponding to the message topic during the instantiation of the publishing role and / or the subscription role;
[0130] The second receiving module 82 is used to receive metadata requests sent by the message viewing device;
[0131] Processing module 83 is used to encapsulate the serialization structure definition information and the identification information corresponding to the message topic according to the metadata request, and generate metadata information;
[0132] The second sending module 84 is used to send metadata information to the message viewing device.
[0133] In one possible embodiment, the second acquisition module 81 is specifically used for:
[0134] Add dynamic libraries to the communication architecture of middleware nodes;
[0135] During the instantiation of a message topic, the serialization structure definition information is obtained through a dynamic library.
[0136] In one possible embodiment, the second acquisition module 81 is further configured to: convert the serialization structure definition information into a serialized data stream format and cache it in the local storage space of the middleware node.
[0137] The middleware node provided in this embodiment can execute the methods provided in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0138] Figure 9 A schematic diagram of the structure of the electronic device provided in this application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes at least one processor 91 and a memory 92. Optionally, the device 90 further includes a communication component 93. The processor 91, memory 92, and communication component 93 are connected via a bus.
[0139] In a specific implementation, at least one processor 91 executes computer execution instructions stored in memory 92, causing at least one processor 91 to perform the above-described method.
[0140] The specific implementation process of processor 91 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0141] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0142] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0143] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0144] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0145] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0146] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0147] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0148] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0150] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0151] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.
[0152] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0153] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A data acquisition method, characterized in that, The method, applicable to message viewing devices adapted to statically configured middleware nodes lacking dynamic topology discovery capabilities, includes: Send a metadata request to at least one middleware node; Receive metadata information returned by the at least one middleware node, wherein the metadata information includes at least the identifier information of the message topic and the serialization structure definition information in the corresponding middleware node; Based on the metadata information, the original message data is deserialized to obtain readable message content.
2. The method according to claim 1, characterized in that, After obtaining the readable message content, the method further includes: outputting the readable message content to an interactive interface for display.
3. The method according to claim 1, characterized in that, After receiving the metadata information returned by the at least one middleware node, the method further includes: merging the metadata information to generate a full list of message topics.
4. The method according to claim 2, characterized in that, The interactive interface includes a first view and a second view. The first view is used to display the identification information of each message topic, and the second view is used to display the complete content of the target message topic in response to the user's selection of the target message topic.
5. The method according to claim 1, characterized in that, The identification information includes at least one of the following: message subject name, message subject information, and message type name.
6. A method for obtaining message topic metadata, characterized in that, The method, applied to statically configured middleware nodes lacking dynamic topology discovery capabilities, includes: During the instantiation of the publish role and / or subscribe role, obtain the serialization structure definition information corresponding to the message topic; Receive messages to view the metadata requests sent by the device; Based on the metadata request, the serialization structure definition information and the identification information corresponding to the message topic are encapsulated to generate metadata information; Send the metadata information to the message viewing device.
7. The method according to claim 6, characterized in that, The step of obtaining the serialization structure definition information corresponding to the message topic includes: Add dynamic libraries to the communication architecture of the middleware nodes; During the instantiation of published and / or subscribed roles, the serialization structure definition information is obtained through the dynamic library.
8. The method according to claim 7, characterized in that, After obtaining the serialization structure definition information through the dynamic library, the method further includes: converting the serialization structure definition information into a serialized data stream format and caching it in the local storage space of the middleware node.
9. A data acquisition system, characterized in that, The method includes a message viewing device and at least one middleware node, the message viewing device being configured to perform the method as described in any one of claims 1 to 5, and the middleware node being configured to perform the method as described in any one of claims 6 to 8.
10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-5, or causing the processor to perform the method as described in any one of claims 6-8.