Flow data acquisition and recharging method and device, electronic equipment and storage medium

Abstract

The invention provides a collection and recharge method and device of circulation data, electronic equipment and a storage medium, in the method, specified nodes and topics needing collection and recharge are flexibly set through a collection configuration file and a recharge configuration file, a user can accurately control the range and size of data collection and the starting time of recharge according to requirements, and the collection and recharge efficiency is improved. The customizability and the practicability of the system are greatly improved, the controllability of recording and playback is good, a high-precision time synchronization mechanism is realized through a time alignment module, the same time sequence relation as original data during collection can be kept in the playback process, nanosecond-level time adjustment is adopted, and the accuracy of the system is improved. It is ensured that data among the multiple designated nodes can be played back according to the correct time sequence, and the problem of playback time sequence disorder caused by time inconsistency in a distributed system is solved.

Description

Technical Field

[0001] This invention relates to the technical field of distributed systems, and in particular to a method, apparatus, electronic device, and storage medium for collecting and reloading flowing data. Background Technology

[0002] In modern distributed systems and communication networks, especially in fields with high requirements for real-time performance and reliability, such as autonomous driving and industrial automation, system debugging, fault reproduction, and performance analysis become particularly important. Traditional data recording and playback methods typically suffer from the following problems: Time precision issues: When playing back data between different devices and processes, inconsistent timestamps can cause timing discrepancies. Large data volume: As system complexity increases, the amount of communication data that needs to be collected grows dramatically, making it difficult to manage efficiently using traditional storage methods; Playback control is difficult: it is difficult to flexibly control the start time, speed, and range of playback; System intrusiveness: Traditional data acquisition methods may interfere with the operation of the original system; Multi-node collaborative replay problem: In a distributed system, how to ensure that data from multiple nodes can be replayed in the correct time sequence.

[0003] In summary, traditional data recording and playback methods suffer from technical problems such as poor controllability of recording and playback and inaccurate timing. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and storage medium for collecting and reloading circulating data, so as to alleviate the technical problems of poor controllability and inaccurate timing of traditional data recording and playback methods.

[0005] In a first aspect, the present invention provides a method for collecting and reinjecting circulating data, comprising: The communication data of specified nodes and topics in the distributed system are recorded according to a predefined collection configuration file, and the communication data with timestamps is written to the data file of the file system and an index file is generated. Injection points are registered according to the predefined backfeed configuration file, and a unified playback timestamp is generated by the time alignment module; The timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

[0006] Furthermore, communication data of specified nodes and topics in the distributed system is recorded according to a predefined collection configuration file, and the communication data, along with a timestamp, is written to a data file in the file system, and an index file is generated, including: The nodes and topics to be collected, the IDs of the collection points corresponding to the nodes and topics, and the data volume limits of the collection points are determined according to the collection configuration file. Create a corresponding file storage directory for each of the aforementioned collection points; The collection point listens to the corresponding communication data in the distributed system according to the node and the topic, captures the communication data and corresponding timestamps of the node and the topic, serializes them, and writes them into the data file; An index file is generated for the data file to record the time range and storage path of the data file.

[0007] Furthermore, injection points are registered according to the predefined backfeed configuration file, and a unified playback timestamp is generated by the time alignment module, including: The data start time, playback start time, playback speed, playback nodes, and topics are determined based on the backfeed configuration file. Based on the replay nodes and topics, register corresponding injection points for different types of communication data, and assign the injection points to different backfeed threads; The unified playback timestamp is dynamically calculated based on the playback start time, the playback speed, the real-world time at the start of the playback task determined by the time alignment module, and the current real-world time.

[0008] Further, the unified playback timestamp is dynamically calculated based on the playback start time, the playback speed, the real-world time at the start of the playback task determined by the time alignment module, and the current real-world time, including: Calculation formula based on unified playback timestamp ,in, This refers to the unified playback timestamp. This indicates the start time of the playback. Represents the current real-world time. This indicates the real-world time at the start of the replay mission. This indicates the playback speed.

[0009] Furthermore, the timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point, including: The data to be replayed is determined from the communication data based on the nodes and topics to be replayed. The planned playback timestamp for each piece of data to be played back is calculated based on the playback start time, the timestamp of the data to be played back, and the data start time. , This indicates the timestamp of the planned replay. This indicates the start time of the playback. This indicates the timestamp of the data to be replayed. Indicates the start time of the data; Continuously compare the planned playback timestamp of each piece of data to be replayed with the unified playback timestamp; When the planned playback timestamp of the data to be played back is less than or equal to the unified playback timestamp, the data to be played back is immediately injected into the currently running system through the corresponding injection point.

[0010] Furthermore, the file storage directory is organized according to a hierarchical structure of system name / session timestamp / node name / topic name / data file / index file; The injection point executes the injection task in parallel through a multi-feedback thread architecture.

[0011] Furthermore, both the data acquisition configuration file and the data reflow configuration file support dynamic loading and hot updates.

[0012] Secondly, the present invention also provides a device for collecting and reinjecting circulating data, comprising: The recording and writing unit is used to record the communication data of specified nodes and topics in the distributed system according to a predefined collection configuration file, and write the communication data with a timestamp to the data file of the file system, and generate an index file; The registration and generation unit is used to register injection points according to a predefined backfeed configuration file, and the time alignment module generates a unified playback timestamp. The real-time comparison and injection unit is used to compare the timestamp of the communication data with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

[0013] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect.

[0014] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method described in the first aspect.

[0015] This invention provides a method for collecting and re-injecting circulating data, comprising: recording communication data of specified nodes and topics in a distributed system according to a predefined collection configuration file, writing the communication data with timestamps to a data file in a file system, and generating an index file; registering injection points according to a predefined re-injection configuration file, and generating a unified playback timestamp by a time alignment module; comparing the timestamps of the communication data with the unified playback timestamps in real time, and injecting the communication data in the data file into the currently running system through the corresponding injection point when the timestamps meet the time conditions. As described above, the data collection and replay method of this invention allows for flexible setting of specified nodes and topics to be collected and replayed through the collection configuration file and replay configuration file. Users can precisely control the scope, size, and start time of data collection and replay according to their needs, greatly improving the customizability and practicality of the system. The recording and playback are highly controllable. A high-precision time synchronization mechanism is achieved through the time alignment module, which can maintain the same time sequence relationship as the original data collection during playback. Nanosecond-level time adjustment ensures that data between multiple specified nodes can be played back in the correct time sequence, solving the problem of playback timing disorder caused by time inconsistency in distributed systems. This alleviates the technical problems of poor controllability and inaccurate timing in traditional data recording and playback methods. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for collecting and reinjecting circulating data, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of nodes, topics, and collection points provided in an embodiment of the present invention; Figure 3 A schematic diagram of the allocation injection point provided in an embodiment of the present invention; Figure 4 A schematic diagram of a data acquisition and reinjection device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Traditional data recording and playback methods suffer from poor controllability and inaccurate timing.

[0020] Based on this, the data collection and re-feedback method of the present invention allows for flexible setting of specified nodes and topics to be collected and replayed through the collection configuration file and the re-feedback configuration file. Users can precisely control the scope, size and start time of data collection and re-feedback according to their needs, which greatly improves the customizability and practicality of the system. The recording and playback are highly controllable. A high-precision time synchronization mechanism is achieved through the time alignment module, which can maintain the same time sequence relationship as the original data collection during playback. Nanosecond-level time adjustment is used to ensure that the data between multiple specified nodes can be played back in the correct time sequence, thus solving the problem of playback time sequence disorder caused by time inconsistency in distributed systems.

[0021] To facilitate understanding of this embodiment, a method for collecting and reinjecting circulating data disclosed in this embodiment of the invention will first be described in detail.

[0022] Example 1: According to an embodiment of the present invention, an embodiment of a method for collecting and reinjecting circulating data is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] Figure 1 This is a flowchart of a method for collecting and reinjecting circulating data according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps: Step S102: Record the communication data of specified nodes and topics in the distributed system according to the predefined collection configuration file, write the communication data with timestamps to the data file of the file system, and generate an index file; Specifically, the above process is executed by the data acquisition module (DataDumper), which is responsible for collecting communication data from the system and writing it to the file system. Its main functions include: Configurable data collection: The ApplicationDataDumpSettings collection configuration file allows for flexible configuration of the nodes (i.e., specified nodes) and topics to be collected.

[0024] ID Management: Assign a unique ID to each collection point to facilitate data management and retrieval.

[0025] File management: Organizes file structure according to nodes and topics, and supports automatic directory creation.

[0026] Data Limits (i.e., Data Volume Limits): Supports setting the maximum data volume and maximum number of records for each topic.

[0027] Caching mechanism: Use SettingsCache to implement configuration caching and hot updates.

[0028] In distributed systems, especially in middleware-based (such as ROS, DDS) driver assistance systems, "nodes" and "topics" are fundamental models for organizing software architecture, such as... Figure 2 As shown. The "collection point" is a logical concept introduced in this invention for efficient management of data collection.

[0029] Definition of a Node: A Node is an independent executable process in a distributed system, serving as the basic unit for performing a specific function. Each node typically handles a specific task, such as sensing, localization, planning, or control. Function: Nodes exchange data and collaborate to complete complex system functions through communication mechanisms (such as publish / subscribe). Characteristics: Independence and specialization.

[0030] For example, in an autonomous driving system, there may be "LiDAR perception nodes", "camera target detection nodes", "fusion positioning nodes", "path planning nodes", etc.

[0031] Definition of a Topic: A topic is a channel or subject for data exchange between nodes. It is an asynchronous communication mechanism that follows a publish / subscribe model. Function: A node can publish to a topic, i.e., send data to that topic; simultaneously, multiple other nodes can subscribe to that topic, i.e., receive data from that topic. A topic is the "pipeline" for data flow. Characteristics: unidirectional, many-to-many.

[0032] For example, a LiDAR perception node might publish a topic named " / perception / lidar_points" to transmit point cloud data; while path planning nodes and obstacle avoidance nodes might simultaneously subscribe to this topic to obtain information about the surrounding environment.

[0033] Definition of Collection Point: A Collection Point is the specific target for data recording by the data acquisition module in this invention. It is a combination of "node" and "topic," precisely specifying which node and which topic in the system will be used to collect data. Function: It is a logical unit for configuration and management. The granularity of data acquisition is "collection point," not the general "node" or "topic." Relationship with Nodes and Topics: One collection point uniquely corresponds to one (node, topic) pair. This means that even if the same topic may be published or subscribed to by multiple nodes, each node that publishes the topic will form an independent collection point.

[0034] Suppose a simplified autonomous driving system consists of the following parts: Nodes: camera_node: Camera sensing node.

[0035] lidar_node: LiDAR sensing node.

[0036] planning_node: Path planning node.

[0037] Topics: image_raw: Used to transmit raw image data. (Published by camera_node) pointcloud: Used for transmitting laser point cloud data. (Published by lidar_node) `control_cmd`: Used to transmit vehicle control commands. (Published by `planning_node`) Based on the above definition, the configurable data collection points are as follows:

[0038] The core of ID management is assigning a unique identifier (i.e., the ID of the collection point) to each collection point. The advantages of this design include: Efficient management: When storing data, you can use IDs to name files or create indexes (such as data_101.bin), which is more efficient than using verbose node and topic names.

[0039] Quick retrieval: When data needs to be replayed, the specific data stream that needs to be replayed can be quickly located by ID.

[0040] Resource allocation: Different parameters can be set for different IDs (i.e., different collection points), such as data volume limits and storage strategies, to achieve fine-grained management.

[0041] The above content will be discussed in more detail below, and will not be repeated here.

[0042] Step S104: Register injection points according to the predefined backfeed configuration file, and generate a unified playback timestamp by the time alignment module; Specifically, the above process is executed by the DataInjector module, which is responsible for reading previously collected communication data and injecting it into the system according to the original timing sequence. Its main functions include: Register injection points: Supports registering different types of injection points to handle data in different formats.

[0043] Multi-threaded backfeedback: DataInjectionThread is used to implement parallel backfeedback by multiple backfeedback threads, improving efficiency.

[0044] Time control: Supports setting the data start time and playback start time for precise time control.

[0045] Thread allocation: Dynamically allocate feedback threads based on node and topic characteristics.

[0046] The concept and function of the injection point mentioned above: It is a key design abstraction in the data backfeed module. It can be understood as an intelligent, configurable "data adapter." Its core function is to establish a bridge between the replayed data files (historical data stored on the hard drive) and the currently running real-time system (such as an autonomous driving software stack). Its sole task is to inject the right data into the right location (i.e., the corresponding topic / channel) in the system at the right time. The data formats flowing in the system are diverse (e.g., image messages, laser point clouds, control commands, etc.). The design of the injection point allows the data backfeed module to be unconcerned about the specific content of the data, only responsible for scheduling and timing, while delegating the details of data parsing and transmission to the specific injection point. This achieves separation of concerns and reduces the complexity of the module.

[0047] "Registering injection points" refers to informing the data injection module what types of data need to be replayed in the system and how to process them. This is achieved through polymorphism and template techniques in object-oriented programming. Registration can be implemented using DataInjectionPointMsg (template class) and DataInjectionPointRaw (class). This is the technical means to achieve "support for different types of injection points" and "unified processing interface".

[0048] Suppose we need to replay three types of data: Image data (a structured ROS message sensor_msgs / Image); Point cloud data (another structured ROS message, sensor_msgs / PointCloud2); Custom binary log data (an unstructured raw data stream); this design allows the core process of the data injection module to be completely generic. It simply requires: "When the time comes, call the injection point's inject() method to send the data out." How exactly the data is sent is determined by the injection point itself.

[0049] The relationship between injection points and backfeed threads is as follows: injection points are "tasks," and backfeed threads are the "mechanism for executing tasks." They are related as "scheduled" and "scheduling policy."

[0050] In a complex system, there may be dozens of injection points that need to replay data simultaneously. If only a single thread is used, the thread needs to frequently switch between different injection points, which is inefficient and cannot meet real-time requirements.

[0051] like Figure 3 As shown, the backfeed manager allocates different injection points to different DataInjectionThreads (backfeedback threads) according to a preset thread allocation strategy. The strategy can be: Assign by node: Assign different topics from the same node to the same backflow thread.

[0052] Distribute by topic hash: Distribute evenly based on the hash value of the topic name.

[0053] Manual binding: Forces certain injection points to be assigned to a specific backflush thread.

[0054] Parallel execution: Each backfeeding thread runs independently, processing only the injection points assigned to it. Internally, each thread sequentially calls the injection points it is responsible for to perform data injection in chronological order.

[0055] Therefore, an injection point can only be executed by one thread (i.e., the backfeed thread), but one thread can execute multiple injection points. The injection point itself doesn't care which thread it executes on; it's only responsible for the "injection" action. Multithreading is a mechanism used by the upper-level scheduler to improve overall playback throughput.

[0056] Timing control is crucial for achieving high-fidelity playback, ensuring that the timing of data injection is completely consistent with the recording. Setting location: Reload configuration file. The data start time and reload start time are set before starting the playback task via the reload configuration file (such as ApplicationDataInjectSettings).

[0057] Data Start Time: This is an absolute point in time on the timeline of the recorded data. It specifies the "starting point" of the recorded data you want to replay. For example, if you recorded one hour of data but only want to replay the segment starting at the 30th minute, then the data start time would be set to the 30th minute. Its purpose is to locate the beginning of the data segment. The playback engine will use this timestamp to quickly locate the first piece of data that should be played back from the stored data file.

[0058] Injection Start Time: This is an absolute point in time in the current system clock during playback. It specifies when you want the playback to begin. For example, you might want the system to start playing back data at 14:00:00 today. Its purpose is to set the start time of the playback task. It allows users to precisely schedule the timing of the playback task.

[0059] The aforementioned time alignment module (TimeSourceUnified) is the core innovation of this invention, solving the time synchronization problem of data playback in distributed systems. Its main functions include: Unified Time Source: Provides a unified time interface to ensure that all parts of the system use the same time.

[0060] Time synchronization: Periodically synchronize with a reference time source to adjust for time deviations.

[0061] High-precision adjustment: Supports time precision adjustment at the nanosecond level.

[0062] Playback time calculation: A unified playback timestamp is calculated using the offset.

[0063] A distributed system can be understood as a system composed of multiple independent computers (or processors) connected by a network, working collaboratively to complete a common task. These computers coordinate with each other, but from the user's perspective, they appear as a single, coherent system. Take an autonomous driving system as an example: a typical autonomous driving system is a complex distributed system. It may contain the following independent computing units (nodes): Sensing computer: processes data from cameras and LiDAR.

[0064] Positioning computer: processes GPS and IMU data to calculate vehicle location.

[0065] Planning computer: Based on the perception and positioning results, it plans the driving route.

[0066] Control computer: Sends control commands to the steering wheel, accelerator, and brake.

[0067] Key challenge: These computers typically have their own local clocks. Due to minute differences in crystal oscillators, their clock speeds vary slightly (i.e., clock drift). Over time, the local times of each computer develop inconsistent time deviations. If each node uses its own local time when replaying data, then "camera images" and "LiDAR point cloud" data, which should occur simultaneously, will be incorrectly replayed at different times, leading to system instability.

[0068] The TimeSourceUnified module was designed to address these challenges. It acts as the system's "atomic clock," providing a unified and reliable time reference.

[0069] The unified time interface is a software function interface, such as `uint64_t getCurrentPlaybackTime()`. Any system component that needs to obtain the current time (such as the data backfeed engine) should no longer read its own local clock, but must call this unified interface. When deciding whether to inject a piece of data, the data backfeed engine calls the `getCurrentPlaybackTime()` interface to obtain the unified "unified playback timestamp" maintained by the time alignment module. This ensures that all components in the system have a completely consistent understanding of the "unified playback timestamp" at the same moment of playback, thereby achieving global timing synchronization.

[0070] Synchronization with a reference time source refers to the time alignment module periodically synchronizing with a high-precision reference time source (such as an NTP server, GPS clock, or a clock recorded at system startup). The purpose of synchronization is to calculate and adjust the offset between the system reference time maintained internally by the time alignment module and the reference time source. Furthermore, this offset is calibrated at the nanosecond level using high-precision timers and algorithms (such as averaging multiple samples). The adjustment is not to the continuous running time, but rather the "deviation value" used for calculation. For example, if the internal clock is found to be 1 millisecond ahead of the reference clock, this 1 millisecond is subtracted when calculating the unified time.

[0071] The calculation of the unified playback timestamp depends on several key parameters: Data start time : User-defined, indicating from which point in the recorded data playback should begin.

[0072] Playback (i.e., recharge) start time : When the user sets the start time for the replay of the task in the "real world".

[0073] Playback speed For example, 1x real-time speed, 2x speed, etc.

[0074] Unified playback timestamp : Dynamically calculated by the time alignment module. Its core calculation formula can be simplified as follows: , Indicates the start time of playback. Represents the current real-world time. This indicates the real-world time at the start of the replay mission. Indicates playback speed. It calculates how long the replay task has actually been running. It's a measurement representing the passage of real-world time. It is an absolute timestamp recorded at the exact moment the playback engine actually starts. It is the current absolute timestamp that can be queried at any time.

[0075] Then, the data refeeding engine checks the timestamp of each recorded data. If its corresponding planned replay timestamp satisfy The data will be injected into the system immediately.

[0076] The essence of data playback is to faithfully transfer a data stream that occurred in the "past" to a time window in the "present" or "future" and replay it. This process is like a precise "time translation" surgery.

[0077] Within the data replay module, for each piece of data to be replayed, its timestamp is used to determine its own characteristics. and the user-configured data start time Refill (i.e., playback) start time Calculate a scheduled playback time. , Indicates the planned replay timestamp. Indicates the start time of playback. Indicates the timestamp of the data to be replayed. This indicates the start time of the data. This calculation is pre-calculated and static. The core loop logic of the playback engine is: Continuously check the current time alignment module. Once the planned replay time for a certain piece of data is reached... The data will be injected into the system immediately.

[0078] The scheduled playback time is the "personal schedule" for each piece of data, and It is the system's "unified clock." The playback engine's job is to continuously compare the "personal schedule" with the "unified clock" and execute the tasks at the designated times.

[0079] Step S106: The timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

[0080] The specifics of this process have already been described above and will not be repeated here.

[0081] This invention provides a method for collecting and re-injecting circulating data, comprising: recording communication data of specified nodes and topics in a distributed system according to a predefined collection configuration file, writing the communication data with timestamps to a data file in a file system, and generating an index file; registering injection points according to a predefined re-injection configuration file, and generating a unified playback timestamp by a time alignment module; comparing the timestamps of the communication data with the unified playback timestamps in real time, and injecting the communication data in the data file into the currently running system through the corresponding injection point when the timestamps meet the time conditions. As described above, the data collection and replay method of this invention allows for flexible setting of specified nodes and topics to be collected and replayed through the collection configuration file and replay configuration file. Users can precisely control the scope, size, and start time of data collection and replay according to their needs, greatly improving the customizability and practicality of the system. The recording and playback are highly controllable. A high-precision time synchronization mechanism is achieved through the time alignment module, which can maintain the same time sequence relationship as the original data collection during playback. Nanosecond-level time adjustment ensures that data between multiple specified nodes can be played back in the correct time sequence, solving the problem of playback timing disorder caused by time inconsistency in distributed systems. This alleviates the technical problems of poor controllability and inaccurate timing in traditional data recording and playback methods.

[0082] The above provides a brief overview of the data collection and reinjection method of the present invention. The specific details involved are described in detail below.

[0083] In an optional embodiment of the present invention, communication data of specified nodes and topics in a distributed system are recorded according to a predefined collection configuration file, and the communication data with timestamps is written to a data file in the file system, and an index file is generated. Specifically, the steps include: (1) Determine the nodes, topics, IDs of the collection points corresponding to the nodes and topics, and data volume limits of the collection points according to the collection configuration file; (2) Create a corresponding file storage directory for each collection point; (3) The collection point listens to the corresponding communication data in the distributed system according to the node and topic, captures the communication data and timestamps corresponding to the node and topic, serializes them, and writes them to the data file; (4) Generate an index file for the data files to record the time range and storage path of the data files.

[0084] For example, the data collection configuration file is as follows: ApplicationDataDumpSettings: collection_points: - node: "camera_node" topic: "image_raw" ID: 101# Assign a unique ID 101 to [Collection Point #1] max_data_size: "1GB" - node: "lidar_node topic: "pointcloud" ID: 102# A unique ID 102 is assigned to [Collection Point #2]. max_records: 10000 - node: "planning_node" topic: "control_cmd" ID: 103# A unique ID 103 is assigned to [Collection Point #3]. The data collection configuration file includes: the nodes to be collected (i.e., the specified nodes), the topics, the IDs of the collection points corresponding to the nodes and topics, and the data volume limits of the collection points.

[0085] The data volume limit for each topic (more precisely, each collection point) is set through specific parameters in the ApplicationDataDumpSettings configuration file.

[0086] Over-limit handling strategy: Intelligent rolling update When the data volume reaches the set limit, the system will not simply report an error or stop, but will execute a preset rolling update strategy to ensure that the data collection task continues without exhausting storage space. The specific strategy is as follows: Triggering conditions: The limiting mechanism is triggered when the total amount of data currently recorded at a certain collection point exceeds max_data_size, or the number of recorded messages exceeds max_records.

[0087] Processing action: The system will automatically delete the oldest part of the data at this collection point (for example, the earliest generated data file or the earliest 10% of the recorded data).

[0088] The core objective is to free up space so that new data can continue to be written. This is similar to the loop recording of a dashcam.

[0089] Advantages: This strategy enables continuous, unattended data collection, making it particularly suitable for autonomous driving test vehicles that require long-term operation.

[0090] Suppose we are collecting data from an autonomous driving system named AD_SYSTEM_V1. This system contains multiple nodes and topics. The directory and file structure is as follows: AD_SYSTEM_V1 / # Root directory: System name ├── 20231026_153045 / # Subdirectory: Start timestamp of the data collection session (accurate to the second) │├── metadata.json# Metadata file: Records global configuration, time base, etc. for this data collection. │├── sensor_node / # Node Directory: Sensing Nodes ││├── front_camera / # Topic Directory: Front-view camera topic │││├── data_1001.bin# Data file: Binary data of collection point ID 1001 │││├── data_1002.bin │││└── index.csv# Index file: Records metadata such as start and end times, size, and corresponding ID for each data file. ││└── lidar / # Topic Directory: LiDAR Topics ││├── data_1003.bin ││└── index.csv │├── localization_node / # Node directory: Locating nodes ││└── gps_pose / # Topic Directory: GPS Positioning Topic ││├── data_1004.bin ││└── index.csv │└── planning_node / # Node Directory: Planning Nodes │└── trajectory / # Topic Directory: Trajectory Planning │├── data_1005.bin │└── index.csv └── 20231027_093212 / # Directory of another data collection session In other words, the file storage directory is organized according to a hierarchical structure of system name / session timestamp / node name / topic name / data file / index file.

[0091] In an optional embodiment of the present invention, the injection point is registered according to a predefined backfeed configuration file, and a unified playback timestamp is generated by the time alignment module, specifically including the following steps: (1) Determine the data start time, playback start time, playback speed, playback nodes and topics based on the data reflow configuration file; (2) Register corresponding injection points for different types of communication data according to the replay nodes and topics, and assign the injection points to different backfeed threads; (3) Calculate the unified playback timestamp dynamically based on the playback start time, playback speed, and real-world time at the start of the playback task determined by the time alignment module.

[0092] Specifically, playback doesn't simply follow the original definitions of nodes and topics; instead, it employs a more precise and controllable process. The goal is to accurately reproduce the data stream, rather than mechanically replaying files. Configuration-driven playback target setting: Users specify the scope of the playback by injecting the configuration file through ApplicationDataInjectSettings. For example, only the `front_camera` and `lidar` topics of the `sensor_node` node can be played back, instead of all data. The start and end times of the playback, as well as the playback speed (e.g., 1x real-time), can also be specified.

[0093] Index-based precise data retrieval: The playback engine locates the corresponding session directory (e.g., 20231026_153045) based on the configuration. Then, instead of directly scanning all data files, the engine reads the index file (index.csv). Using the index, it instantly knows that to replay the front_camera data from 15:30:45.200 to 15:30:46.000 on 2023-10-26, it needs to load the data_1002.bin and data_1003.bin files sequentially. This significantly improves retrieval efficiency.

[0094] The process of registering the corresponding injection points and allocating different backfeeding threads has been introduced in the previous content and will not be repeated here.

[0095] Specifically, a unified playback timestamp is dynamically calculated based on the playback start time, playback speed, and the real-world time at the start of the playback task determined by the time alignment module, as well as the current real-world time. This process includes the following steps: Calculation formula based on unified playback timestamp ,in, Indicates a unified playback timestamp. Indicates the start time of playback. Represents the current real-world time. This indicates the real-world time at the start of the replay mission. Indicates playback speed.

[0096] In an optional embodiment of the present invention, the timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point. Specifically, the following steps are included: (1) Determine the data to be replayed in the communication data based on the replay node and topic; (2) Calculate the planned playback timestamp for each piece of data to be played back based on the playback start time, the timestamp of the data to be played back, and the data start time. , Indicates the planned replay timestamp. Indicates the start time of playback. Indicates the timestamp of the data to be replayed. Indicates the start time of the data; (3) Continuously compare the planned playback timestamp with the unified playback timestamp for each piece of data to be replayed; (4) When the planned playback timestamp of the data to be played back is less than or equal to the unified playback timestamp, the data to be played back is immediately injected into the currently running system through the corresponding injection point.

[0097] In an optional embodiment of the invention, the injection point executes the injection task in parallel using a multi-feedback thread architecture. The injection point decouples the data injection operation from the specific business logic execution function.

[0098] In an optional embodiment of the present invention, both the acquisition configuration file and the recharge configuration file support dynamic loading and hot updating.

[0099] Specifically, SettingsCache is a memory caching component designed to improve system performance and response speed.

[0100] Configuration files (such as ApplicationDataDumpSettings) are typically stored on the hard drive. If the hard drive is read every time the configuration needs to be read (for example, checking the collection point limits after each message is received), the I / O operation will be very slow, becoming a system bottleneck.

[0101] 1. Initial loading: When the system starts, SettingsCache reads the configuration file from the hard drive into memory all at once.

[0102] 2. High-speed access: After that, when any module needs to obtain configuration, it reads directly from this memory cache, which is extremely fast.

[0103] 3. Hot update: If a user modifies the configuration file without restarting the system, SettingsCache can detect the file change and automatically reload it into memory to make the new configuration take effect. This is the "hot update" capability.

[0104] In simple terms, SettingsCache is a "fast lookup copy" of configuration information in memory. It sits between a slow hard drive and a high-speed data acquisition program, solving the performance bottleneck of configuration reading.

[0105] The method of this invention can efficiently acquire, store, and replay communication data according to the original time sequence. It has the following advantages: 1. Precise time synchronization and alignment This invention implements a high-precision time synchronization mechanism through the TimeSourceUnified module (i.e., the time alignment module), which can maintain the same time sequence relationship as the original data acquisition during playback. The system uses nanosecond-level time adjustment to ensure that data between multiple nodes can be played back in the correct time sequence, solving the problem of playback timing disorder caused by time inconsistency in distributed systems.

[0106] 2. Flexible configuration and control The system supports flexible configuration of nodes, topics, and parameters to be collected and replayed through configuration files. Users can precisely control the scope, size, and start time of data replay according to their needs, greatly improving the system's customizability and usability.

[0107] 3. Highly efficient data processing capabilities Multi-threaded parallel processing: DataInjectionThread enables multi-threaded parallel injection, improving data processing efficiency.

[0108] Sampling optimization: During time synchronization, a strategy of multiple sampling (i.e., collecting reference time sources and synchronizing with the reference time sources) and selecting the best sampling result is adopted to ensure the accuracy and stability of time synchronization.

[0109] Caching mechanism: SettingsCache is used to implement configuration caching and hot updates, which improves the efficiency of configuration reading.

[0110] 4. Low-invasive design The system adopts a non-intrusive design concept, and realizes data reinjection by registering injection points, without modifying the original business logic code, thus reducing the impact on the original system.

[0111] 5. Reliable data management File organization optimization: Organize the file structure according to nodes and topics to facilitate data management and retrieval.

[0112] Data volume control: Supports setting the maximum data volume and number of records for each topic, effectively managing storage space.

[0113] Error handling mechanism: The system includes a comprehensive error detection and handling mechanism to ensure the reliability of the data acquisition and backfeed process.

[0114] 6. Improve development and debugging efficiency With the recording and playback functions of this invention, developers can easily reproduce system operation scenarios, accelerate problem localization and debugging processes, and significantly improve development efficiency.

[0115] The main technical points include: Unified time source technology based on time offset Key technical features: A unified time source is implemented through the TimeSourceUnified class, maintaining the deviation between the system time and the reference time to achieve time synchronization with nanosecond-level precision. This technology ensures time accuracy during playback through periodic sampling and dynamic adjustment of the offset.

[0116] Key implementation points: The deviation between the maintenance system time and the reference time; Periodically sample and select the best sampling result; Supports precise mapping between playback time and real-time; A configurable data acquisition and backfeeding mechanism.

[0117] Key technical features: The scope, parameters, and behaviors of data collection and backfeeding are defined through configuration files, supporting dynamic loading and application configuration, and enabling flexible data management.

[0118] Key implementation points: Configuration management based on ApplicationDataDumpSettings and ApplicationDataInjectSettings; Supports fine-grained control over the scope of data collection and backfeeding based on nodes and topics; Configure a caching mechanism to improve the efficiency of configuration reading and application; Multi-threaded parallel data backfeeding architecture.

[0119] Key technical features: Multi-threaded parallel injection is achieved through DataInjectionThread, which dynamically allocates injection threads based on node and topic characteristics to improve data processing efficiency.

[0120] Key implementation points: Thread allocation strategy based on node or callback ID; Supports multiple injection points working together in the same thread; Thread-safe data access and manipulation mechanisms; Templated injection point design.

[0121] Key technical features: Through the DataInjectionPointMsg template class and DataInjectionPointRaw class, a unified processing interface for different types of data is implemented, supporting flexible expansion.

[0122] Key implementation points: Template-based type-safe message processing; Supports unified processing of raw data and structured messages; Decoupling design of injection points and execution functions; Data management technology based on ID mapping.

[0123] Key technical features: A unique ID is assigned to each collection point and recharge point, and the correspondence between data and files is managed through an ID mapping table, thereby improving data retrieval and management efficiency.

[0124] Key implementation points: A mapping mechanism from nodes and topics to unique IDs; Management of the mapping from ID to file index; Highly efficient ID lookup and management algorithms; Precise time calculation and control technology.

[0125] Key technical features: By calculating the offset between the data start time and the re-irrigation start time, precise control of the playback time can be achieved.

[0126] Key implementation points: Algorithm for mapping data time to playback time; Offset-based time calculation method; Synchronization mechanism between real-time and playback time.

[0127] The main inventive points include: 1. Invention Point 1: A nanosecond-level time synchronization method based on a unified time source and dynamic offset calculation Technical problem solved: Playback timing disorder caused by clock asynchrony among multiple nodes in a distributed system.

[0128] Technical approach: The TimeSourceUnified module is introduced to maintain a globally unified "unified playback time". This time is kept synchronized by periodically sampling the system reference time and calculating dynamic offsets, with an accuracy of nanoseconds.

[0129] Technical effect: It ensures that the original time series relationship between the replayed data is strictly maintained, and realizes high-fidelity time series replay across nodes and processes.

[0130] 2. Invention Point Two: A Low-Intrusive Data Acquisition and Feedback Architecture Based on Configuration and Templates Technical problems to be solved: Traditional data recording and playback systems are highly invasive, lack flexibility, and are difficult to adapt to various data types.

[0131] Technical measures: Dynamically define collection and injection behavior using configuration files (ApplicationDataDump / InjectSettings); uniformly process different types of data through templated injection points (DataInjectionPointMsg); manage all data nodes and topics through unique ID mapping.

[0132] Technical benefits: The system is highly flexible and configurable, with almost no intrusion into the original business code, and is easy to extend and maintain.

[0133] 3. Invention Point Three: A Multi-threaded Parallel Feedback and Load Distribution Method Based on Node Characteristics Technical problem to be solved: Low playback efficiency under large data volumes, which cannot meet real-time requirements.

[0134] Technical approach: Create a DataInjectionThread backfeeding thread pool, and dynamically allocate different injection points to different threads for parallel execution based on the node ID, topic ID, or characteristics of the callback function.

[0135] Technical effect: It greatly improves the throughput and efficiency of data backfeeding, and can quickly reproduce complex scenarios.

[0136] 4. Invention Point Four: Playback Control Mechanism Based on Precise Time Mapping The technical problem to be solved: Inability to flexibly control the start and end times, speed, and range of playback.

[0137] Technical means: By calculating the data start time, the re-injection start time, and the time offset, a precise mapping relationship between the data file timestamp and the playback time is established.

[0138] Technical benefits: Users can flexibly specify the starting point of playback and the playback speed (e.g., 1x real-time, 2x speed), achieving precise control over the playback process.

[0139] Example 2: This invention also provides a device for collecting and reinjecting circulating data. This device is mainly used to execute the method for collecting and reinjecting circulating data provided in Embodiment 1 of this invention. The following is a detailed description of the device for collecting and reinjecting circulating data provided in this invention.

[0140] Figure 4 This is a schematic diagram of a data acquisition and reinjection device according to an embodiment of the present invention, as shown below. Figure 4 As shown, the device mainly includes: a recording and writing unit 10, a registration and generation unit 20, and a real-time comparison and injection unit 30, wherein: The recording and writing unit is used to record the communication data of specified nodes and topics in the distributed system according to the predefined collection configuration file, write the communication data with timestamps to the data file of the file system, and generate an index file; The registration and generation unit is used to register injection points according to a predefined backfeed configuration file, and the time alignment module generates a unified playback timestamp. The real-time comparison and injection unit is used to compare the timestamp of the communication data with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

[0141] The present invention provides a data acquisition and backfeeding device for circulating data, comprising: recording communication data of specified nodes and topics in a distributed system according to a predefined acquisition configuration file, writing the communication data with timestamps to a data file in a file system, and generating an index file; registering injection points according to a predefined backfeeding configuration file, and generating a unified playback timestamp by a time alignment module; comparing the timestamps of the communication data with the unified playback timestamps in real time, and when the timestamps meet the time conditions, injecting the communication data in the data file into the currently running system through the corresponding injection point. As described above, the data acquisition and playback device of the present invention allows for flexible setting of designated nodes and topics to be acquired and played back through acquisition configuration files and playback configuration files. Users can precisely control the scope, size, and start time of data acquisition and playback according to their needs, greatly improving the customizability and practicality of the system. The recording and playback are highly controllable. A high-precision time synchronization mechanism is achieved through the time alignment module, which can maintain the same time sequence relationship as the original data acquisition during playback. Nanosecond-level time adjustment ensures that data between multiple designated nodes can be played back in the correct time sequence, solving the problem of playback timing disorder caused by time inconsistency in distributed systems. This alleviates the technical problems of poor controllability and inaccurate timing in traditional data recording and playback methods.

[0142] Optionally, the recording and writing unit is also used to: determine the nodes, topics, IDs of the collection points corresponding to the nodes and topics, and data volume limits of the collection points according to the collection configuration file; create a corresponding file storage directory for each collection point; the collection point listens to the corresponding communication data in the distributed system according to the nodes and topics, captures the communication data and corresponding timestamps corresponding to the nodes and topics, serializes them, and writes them to the data file; and generates an index file for the data file to record the time range and storage path of the data file.

[0143] Optionally, the registration and generation unit is also used to: determine the data start time, playback start time, playback speed, playback nodes and topics according to the backfeed configuration file; register corresponding injection points for different types of communication data according to the playback nodes and topics, and allocate the injection points to different backfeed threads; and dynamically calculate a unified playback timestamp based on the real-world time at the start of the playback task determined by the playback start time, playback speed, and time alignment module, and the current real-world time.

[0144] Optionally, the registration and generation unit is also used to: calculate a formula based on the unified playback timestamp. ,in, Indicates a unified playback timestamp. Indicates the start time of playback. Represents the current real-world time. This indicates the real-world time at the start of the replay mission. Indicates playback speed.

[0145] Optionally, the real-time comparison and injection unit is also used to: determine the data to be replayed in the communication data based on the replay nodes and topics; calculate the planned replay timestamp for each piece of data to be replayed based on the replay start time, the timestamp of the data to be replayed, and the data start time, wherein, , Indicates the planned replay timestamp. Indicates the start time of playback. Indicates the timestamp of the data to be replayed. Indicates the data start time; continuously compares the planned playback timestamp of each piece of data to be replayed with the unified playback timestamp; when the planned playback timestamp of the data to be replayed is less than or equal to the unified playback timestamp, immediately inject the data to be replayed into the currently running system through the corresponding injection point.

[0146] Optionally, the file storage directory is organized according to a hierarchical structure of system name / session timestamp / node name / topic name / data file / index file; the injection point executes the injection task in parallel through a multi-feedback thread architecture.

[0147] Optionally, both the data collection configuration file and the data reload configuration file support dynamic loading and hot updating.

[0148] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0149] like Figure 5 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions that can be executed by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus. The processor 601 executes the machine-readable instructions to perform the steps of the above-described method for collecting and reloading flowing data.

[0150] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned method for collecting and reloading the circulating data.

[0151] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0152] Corresponding to the above-described method for collecting and reloading circulating data, this application also provides a computer-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by a processor, they cause the processor to perform the steps of the above-described method for collecting and reloading circulating data.

[0153] The data acquisition and reinjection device provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0154] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0155] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0158] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the data acquisition and backfilling methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0160] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for collecting and reinjecting circulating data, characterized in that, include: The communication data of specified nodes and topics in the distributed system is recorded according to a predefined collection configuration file, and the communication data with timestamps is written to the data file of the file system and an index file is generated. Injection points are registered according to the predefined backfeed configuration file, and a unified playback timestamp is generated by the time alignment module; The timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

2. The method according to claim 1, characterized in that, The system records communication data of specified nodes and topics in a distributed system according to a predefined data collection configuration file, and writes the communication data, along with a timestamp, to a data file in the file system, and generates an index file, including: The nodes and topics to be collected, the IDs of the collection points corresponding to the nodes and topics, and the data volume limits of the collection points are determined according to the collection configuration file. Create a corresponding file storage directory for each of the aforementioned collection points; The collection point listens to the corresponding communication data in the distributed system according to the node and the topic, captures the communication data and corresponding timestamps of the node and the topic, serializes them, and writes them into the data file; Generate an index file for the data file to record the time range and storage path of the data file.

3. The method according to claim 1, characterized in that, Injection points are registered according to the predefined backfeed configuration file, and a unified playback timestamp is generated by the time alignment module, including: The data start time, playback start time, playback speed, playback nodes, and topics are determined based on the backfill configuration file. Based on the replay nodes and topics, register corresponding injection points for different types of communication data, and assign the injection points to different backfeed threads; The unified playback timestamp is dynamically calculated based on the playback start time, the playback speed, the real-world time at the start of the playback task determined by the time alignment module, and the current real-world time.

4. The method according to claim 3, characterized in that, The unified playback timestamp is dynamically calculated based on the playback start time, the playback speed, the real-world time at the start of the playback task determined by the time alignment module, and the current real-world time, including: Calculation formula based on unified playback timestamp ,in, This refers to the unified playback timestamp. This indicates the playback start time. Represents the current real-world time. This indicates the real-world time at the start of the replay mission. This indicates the playback speed.

5. The method according to claim 1, characterized in that, The timestamp of the communication data is compared with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point, including: The data to be replayed is determined from the communication data based on the nodes and topics to be replayed. The planned playback timestamp for each piece of data to be played back is calculated based on the playback start time, the timestamp of the data to be played back, and the data start time. , This indicates the timestamp of the planned replay. This indicates the playback start time. This indicates the timestamp of the data to be replayed. Indicates the start time of the data; Continuously compare the planned playback timestamp of each piece of data to be replayed with the unified playback timestamp; When the planned playback timestamp of the data to be played back is less than or equal to the unified playback timestamp, the data to be played back is immediately injected into the currently running system through the corresponding injection point.

6. The method according to claim 2, characterized in that, The file storage directory is organized according to a hierarchical structure of system name / session timestamp / node name / topic name / data file / index file; The injection point executes the injection task in parallel through a multi-feedback thread architecture.

7. The method according to claim 1, characterized in that, Both the data acquisition configuration file and the data recharge configuration file support dynamic loading and hot updating.

8. A device for collecting and reinjecting circulating data, characterized in that, include: The recording and writing unit is used to record the communication data of specified nodes and topics in the distributed system according to a predefined collection configuration file, and write the communication data with a timestamp to the data file of the file system, and generate an index file; The registration and generation unit is used to register injection points according to a predefined backfeed configuration file, and the time alignment module generates a unified playback timestamp. The real-time comparison and injection unit is used to compare the timestamp of the communication data with the unified playback timestamp in real time. When the timestamp meets the time condition, the communication data in the data file is injected into the currently running system through the corresponding injection point.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the method of any one of claims 1 to 7.

Images