Timestamp alignment method and module based on dataflow framework

Abstract

The present disclosure relates to a timestamp alignment method and module based on a data flow framework, wherein the method comprises: receiving input data streams obtained within a time window; and obtaining output data streams with timestamp alignment information from the input data streams through data flow driving rules, wherein the data flow driving rules comprise data flow relationships between input data streams and output data streams of each algorithm module. The present disclosure can conveniently and flexibly configure and manage timestamp alignment modules for the entire autonomous driving system, thereby reducing the technical threshold for developing autonomous driving system application layer software based on a ROS 2 type framework.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to timestamp alignment methods and modules based on data flow frameworks. Background Technology

[0002] In autonomous driving systems, time synchronization specifically includes the following operations: unified clock source, hardware synchronization, and software synchronization. As input data streams to the application layer software modules of autonomous driving systems, sensor data typically carries timestamps, such as GPS / GNSS timestamps, camera timestamps, LiDAR timestamps, millimeter-wave radar timestamps, or IMU timestamps. In the development of autonomous driving system application layers, data from many sensors is required, such as LiDAR, camera, GPS / IMU, etc. If the timestamps of the various sensor data received by the autonomous driving domain control computing unit are inconsistent, it can lead to inaccurate obstacle recognition or program coredumps.

[0003] In related technologies, the main solutions for timestamp alignment in data streams are as follows: If an algorithm module receives input data from multiple sensors, during an iteration cycle, the module checks whether the timestamps of the input sensor data are aligned. Based on the alignment, the corresponding logical branch is executed. However, the above solution has the following problems: The entire autonomous driving system has numerous algorithm modules, each requiring its own logic for determining timestamp alignment. Some modules receive input from a single sensor, others from multiple sensors arriving simultaneously, and still others from any one or more of these sensors. This results in different timestamp alignment logic within each module and a different number of sensor timestamps used. Algorithm modules are organized through a data stream framework, and the data streams between different modules are interconnected. Whether the timestamps of the sensor data input to one algorithm module are aligned affects its output data stream, thus impacting all algorithm modules that use this output data stream as their input. In other words, timestamp alignment causes a series of changes in the processing logic within each algorithm module.

[0004] Based on the above analysis of the development status of this technology field, the logic for determining the alignment of sensor data timestamps in existing technical solutions is scattered within various algorithm modules, lacking a global view of the impact of whether or not timestamps are aligned. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a timestamp alignment method and module based on a data flow framework.

[0006] According to a first aspect of the present disclosure, a timestamp alignment method based on a data flow framework is provided, comprising:

[0007] Receive the input data stream acquired within the time window;

[0008] The input data stream is processed by data stream driving rules to obtain an output data stream with timestamp alignment information. The data stream driving rules include the data stream relationship between the input data stream and the output data stream of each algorithm module.

[0009] According to a second aspect of the present disclosure, a timestamp alignment module based on a data stream framework is provided, comprising: an acquisition unit for receiving an input data stream acquired within a time window;

[0010] The timestamp alignment unit is used to process the input data stream through data flow driving rules to obtain an output data stream with timestamp alignment information. The data flow driving rules include the data flow relationship between the input data stream and the output data stream of each algorithm module.

[0011] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the timestamp alignment method based on a dataflow framework provided in the first aspect of the present disclosure.

[0012] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the timestamp alignment method based on a data flow framework provided in the first aspect of the present disclosure.

[0013] According to a fifth aspect of the present disclosure, a vehicle is provided that stores a set of instructions, which are executed by the vehicle to implement the timestamp alignment method based on a data flow framework provided in the first aspect of the present disclosure.

[0014] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: designing timestamp alignment with global view attributes at the framework level enables driving control of various algorithm modules in the data stream at the framework level, making it convenient and flexible to configure and manage the timestamp alignment module for the entire autonomous driving system, and reducing the technical threshold for developing application layer software of autonomous driving systems based on ROS 2-like frameworks.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0017] Figure 1 This is a flowchart illustrating a timestamp alignment method based on a data flow framework according to an exemplary embodiment.

[0018] Figure 2 This is a detailed flowchart illustrating a timestamp alignment method based on a data flow framework, according to an exemplary embodiment.

[0019] Figure 3 This is a schematic diagram illustrating receiving an input data stream within a time window according to an exemplary embodiment;

[0020] Figure 4 This is a schematic diagram illustrating the relationship between the input data stream and the output data stream of an algorithm module according to an exemplary embodiment;

[0021] Figure 5 This is a schematic diagram illustrating the data-driven rules of various algorithm modules according to an exemplary embodiment;

[0022] Figure 6 This is a schematic diagram illustrating case 1 according to an exemplary embodiment;

[0023] Figure 7 This is a schematic diagram illustrating case 2 according to an exemplary embodiment;

[0024] Figure 8 This is a schematic diagram illustrating case 3 according to an exemplary embodiment;

[0025] Figure 9 This is a block diagram illustrating a timestamp alignment module based on a data stream framework according to an exemplary embodiment;

[0026] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment;

[0027] Figure 11 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation

[0028] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0029] It should be noted that the relevant embodiments and accompanying drawings are only for describing and illustrating exemplary embodiments provided by this disclosure, and not all embodiments of this disclosure, nor should this disclosure be understood to be limited to the relevant exemplary embodiments.

[0030] It should be noted that the terms "first," "second," etc., used in this disclosure are only used to distinguish different steps, devices, or modules. These terms do not represent any specific technical meaning, nor do they indicate any order or interdependence between them.

[0031] It should be noted that the terms “a,” “a plurality of,” and “at least one” used in this disclosure are illustrative rather than restrictive. Unless otherwise expressly indicated in the context, they should be understood as “one or more.”

[0032] It should be noted that the term "and / or" used in this disclosure is used to describe the relationship between related objects, and generally indicates that there are at least three relationships. For example, A and / or B can at least indicate: the existence of A alone, the existence of both A and B, and the existence of B alone.

[0033] It should be noted that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of this disclosure is not limited by the order in which the steps are described in the relevant embodiments.

[0034] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0035] Technical Terminology Explanation

[0036] Time synchronization: A unified host provides a reference time to each sensor, and each sensor adds timestamp information to its independently collected data according to its own calibrated time, so as to synchronize the timestamps of all sensors.

[0037] Timestamp: This refers to the sensor timestamp, which is a time standard. Different sensors have different time mechanisms, such as GPS / GNSS timestamps, camera timestamps, LiDAR timestamps, millimeter-wave radar timestamps, and IMU timestamps. Common time standards include UTC.

[0038] System time: Generally refers to the time of the system on the SOC, which is expressed in UNIX or the time used by the UNIX system. It is generally defined as the total number of seconds from 00:00:00 on January 1, 1970 (UTC) to the present.

[0039] Exemplary methods

[0040] Figure 1 This is a flowchart illustrating a timestamp alignment method based on a data flow framework, according to an exemplary embodiment, such as... Figure 1As shown, the timestamp alignment method based on the data flow framework can be used in the data flow framework driven by the timestamp alignment module in the autonomous driving system, including the following steps.

[0041] In step S110, the input data stream acquired within the time window is received; in this embodiment of the invention, the time window needs to be preset according to requirements, and its input data stream is multiple data streams from multiple algorithm modules.

[0042] In step S120, the input data stream is processed by data flow driving rules to obtain an output data stream with timestamp alignment information. The data flow driving rules include the data flow relationship between the input data stream and the output data stream of each algorithm module. In one example, the data flow relationship between the input data streams may include logical AND, logical OR, and other logical relationships.

[0043] In step S120, in one instance, the timestamp alignment information specifically includes a trigger flag that determines whether each algorithm module can be triggered. In another instance, the timestamp alignment information may also be represented by other identifiers.

[0044] In step S120, the data flow relationship specifically includes at least one of the following: the data flow relationship between the input data flow of each algorithm module, the data flow relationship between the input data flow and the output data flow of each algorithm module, and the data flow relationship between each algorithm module. For example, consider algorithm module A and algorithm module B. Algorithm module A includes input data streams a1, a2, a3, and output data stream a4, while algorithm module B includes input data streams b1, b2, b3, and output data stream b4. The specific data flow relationships include: the relationship between input data streams a1, a2, and a3 of algorithm module A; the relationship between input data streams b1, b2, and b3 of algorithm module B; the relationship between input data streams a1, a2, and a3 and output data stream a4; the relationship between input data streams b1, b2, and b3 and output data stream b4; and the data flow relationship between algorithm module A and algorithm module B, such as the relationship between output data stream a4 and input data stream b1.

[0045] In step S120, the data flow driving rules specifically include the following three cases:

[0046] Case 1: If all input data streams of the algorithm module arrive within the time window, the algorithm module is triggered; for example, if input data streams a1, a2, and a3 all arrive within the time window, then algorithm module A is triggered.

[0047] Case 2: If a portion of the input data streams in all input data streams of the algorithm module arrive within the time window, the algorithm module or the corresponding sub-algorithm in the algorithm module will be triggered. For example, if input data streams a1 and a2 arrive within the time window, algorithm module A or the corresponding sub-algorithm in algorithm module A will be triggered.

[0048] Scenario 3: Based on the data flow relationships between various algorithm modules, when the input data flow of an algorithm module is the output data flow of a related algorithm module, the triggering of that algorithm module is determined by whether the output data flow of the related algorithm module arrives within the time window. For example, if output data flow a4 is input data flow b1, the triggering of algorithm module B can be determined by whether output data flow a4 arrives within the time window.

[0049] It should be noted that the above three situations can be used individually or in combination as needed.

[0050] In step S120, in one instance, obtaining an output data stream with timestamp alignment information from the input data stream through data stream driving rules can specifically include the following processing:

[0051] Within the time window, based on the data flow-driven rules, it is determined whether each algorithm module can be triggered and the corresponding trigger flag is marked according to whether the corresponding input data flow is received;

[0052] The trigger flags of each algorithm module determine whether to execute the corresponding iterative calculation and / or the corresponding multi-sensor data stream fusion calculation. Iterative methods are a typical type of numerical computation method. Their basic idea is successive approximation: first, a rough approximation is taken, and then the same recursive formula is used to repeatedly correct this initial value until the predetermined accuracy requirement is met. Multi-sensor data stream fusion calculation refers to: fully utilizing multi-sensor data resources from different times and spaces, using computer technology to analyze, synthesize, manage, and use multi-sensor observation data obtained in time series under certain criteria, to obtain a consistent interpretation and description of the measured object, and thus achieve corresponding decision-making and estimation, enabling the system to obtain more comprehensive information than its individual components.

[0053] In this embodiment of the invention, the time window, data flow relationship, data flow driving rule, and trigger flag can also be set to a dynamically configurable mode. In one instance, the time window, data flow relationship, data flow driving rule, and trigger flag can be configured and managed (i.e., dynamically configured and managed) through C structures, C++ structures, or classes.

[0054] In summary, the technical solutions of this invention propose a method for a data flow framework driven by a timestamp alignment module. This method designs a timestamp alignment method with global view attributes at the framework level, configures and manages sensor timestamp alignment information, and focuses on the logical judgment of whether timestamp alignment is achieved. It also drives and controls each algorithm module in the data flow at the framework level. This invention provides a global design for "timestamp alignment" at the framework level, eliminating the need for timestamp alignment logic judgments to be scattered within individual algorithm modules. This allows application layer development engineers of autonomous driving systems to conveniently and flexibly configure and manage the timestamp alignment module for the entire autonomous driving system, lowering the technical threshold for developing application layer software for autonomous driving systems based on frameworks like ROS 2.

[0055] The technical solutions of the embodiments of the present invention will be illustrated below with reference to the accompanying drawings.

[0056] Figure 2 This is a detailed flowchart illustrating a timestamp alignment method based on a data flow framework according to an exemplary embodiment, specifically including the following processes:

[0057] Step S210: Set a time window Δt. Assume the first initial time is 0, the second time is 0+Δt, and so on. Within each Δt time window, the sensor's input data stream arrives, and the sensor's input data stream has a timestamp. (See below) Figure 3 As shown, assume that within a time window Δt, input data streams ① to ⑤ from 5 sensors arrive.

[0058] Step S220: For each algorithm module, establish the input data stream, output data stream, and data stream relationships between algorithm modules. The technical solutions of this embodiment include, but are not limited to, […]. Figure 4 The algorithm module shown is data flow driven. For example... Figure 4 As shown, algorithm module 1 has an input data stream ① and an output data stream ⑥; algorithm module 2 has an input data stream ①, an input data stream ②, and an output data stream ⑦; algorithm module 3 has an input data stream ③, an input data stream ④, and an output data stream ⑧; and algorithm module 4 has an input data stream ⑤, an input data stream ⑥, and an output data stream ⑨. The input data stream ⑥ of algorithm module 4 is the output data stream ⑥ of algorithm module 1, and the input data stream ① is both the input data stream of algorithm module 1 and the input data stream of algorithm module 2.

[0059] Step S230: Configure the data flow driving rules for each algorithm module.

[0060] Autonomous driving system application layer engineers can flexibly configure the data-driven rules for each algorithm module, such as... Figure 5As shown, algorithm module 2 will only be executed when both input data streams ① and ② arrive simultaneously, thus generating output data stream ⑦; algorithm module 3 can be executed when either input data stream ③ or ④ arrives, or both arrive simultaneously, thus generating output data stream ⑧; algorithm module 4 will only be executed when both input data streams ⑤ and ⑥ arrive simultaneously, thus generating output data stream ⑨. Here, input data stream ⑥ of algorithm module 4 is the output data stream ⑥ of algorithm module 1, and input data stream ① is both the input data stream of algorithm module 1 and the input data stream of algorithm module 2. In practical applications, data-driven rules can be represented using logical AND and / or logical OR.

[0061] Step S240: Within a time window Δt, determine whether the input data streams ① to ⑤ from the five sensors have arrived. Input data streams from any sensor arriving at any moment within the time window Δt are considered timestamp-aligned, allowing for multi-sensor data fusion calculations. Furthermore, timestamp alignment maintenance information, including the time window, data stream relationships, data stream driving rules, and trigger flags, can be dynamically managed.

[0062] like Figure 6 As shown in Case 1, assuming that within a time window Δt, the input data streams ① to ⑤ from the five sensors arrive, algorithm modules 1, 2, 3, and 4 are all triggered to execute, and the trigger flags of algorithm modules 1 to 4 are managed as follows: algorithm module 1 is trigger_TRUE, algorithm module 2 is trigger_TRUE, algorithm module 3 is trigger_TRUE, and algorithm module 4 is trigger_TRUE.

[0063] like Figure 7 In scenario 2, assuming that within a time window Δt, the input data streams ② to ⑤ from the four sensors arrive, but input data stream ① does not arrive, then algorithm module 3 is triggered to execute; algorithm modules 1, 2, and 4 are not triggered to execute, and the trigger flags managing algorithm modules 1-4 are: trigger_FALSE for algorithm module 1, trigger_FALSE for algorithm module 2, trigger_TRUE for algorithm module 3, and trigger_FALSE for algorithm module 4.

[0064] like Figure 8As shown in scenario 3, assuming that within a time window Δt, the input data streams ①, ②, ④, and ⑤ from the four sensors arrive, but input data stream ③ does not arrive, then algorithm modules 1, 2, 3, and 4 are all triggered and executed. The trigger flags for managing algorithm modules 1 through 4 are set as follows: algorithm module 1 is trigger_TRUE, algorithm module 2 is trigger_TRUE, algorithm module 3 is trigger_TRUE_withOnly④ (only when input data stream 4 is received within the time window), and algorithm module 4 is trigger_TRUE.

[0065] It should be noted that the above three scenarios are merely illustrative examples. The technical solutions of this invention include, but are not limited to, the above three scenarios. The technical solutions of this invention can provide users with flexible configuration of data-driven rules and information related to timestamps in management algorithm modules. This can be defined and implemented using C / C++ structures or classes.

[0066] Step S250: When each algorithm module performs iterative calculation or multi-sensor fusion calculation within a time period, it can directly retrieve the corresponding timestamp-related information from the C / C++ structure or class maintained in step S240, such as trigger flags. Different trigger flags trigger different sub-algorithm modules, thereby determining whether the algorithm module triggers iterative calculation or multi-sensor fusion calculation, and which logical branch within the algorithm module is triggered, such as the case of algorithm module 3 in step S240.

[0067] As can be seen from the above technical solution, the technical solution of this invention uses a frame-level timestamp alignment method with a global view attribute to centrally configure and manage the timestamp alignment information of sensor data. The "timestamp alignment method" is designed at the frame level, with all sensor data from the autonomous driving domain serving as input. Internally, the "timestamp alignment" maintains information related to the sensor data streams of all algorithm modules. This information includes the input data stream name, output data stream name, and driving rules of the algorithm module. The driving rules of the algorithm module can be flexibly configured by the user. The output of the "timestamp alignment" is: whether each algorithm module triggers and which logical branch is executed, calculated based on the driving rules of the algorithm module and all timestamped sensor data.

[0068] The above technical solution is based on the "timestamp alignment method," which drives and controls the various algorithm modules in the data flow framework of the autonomous driving system at the framework level. Within one iteration cycle of the algorithm, each algorithm module in the autonomous driving system obtains the necessary timestamp-related information from the "timestamp alignment method." This information includes whether each algorithm module has been triggered, and which logical branch each module executes if triggered.

[0069] The embodiments of the present invention can be used for the development of application layer software for autonomous driving systems, including but not limited to ROS 1, ROS 2, CyberRT or self-developed frameworks.

[0070] The technical solution of this invention can provide an abstract, global, framework-level "timestamp alignment method" for R&D engineers of autonomous driving system application layers such as OEMs / Tier 1s. R&D engineers of autonomous driving system application layers can centrally configure global strategies driven by data flow based on the "timestamp alignment method", thereby driving the entire autonomous driving system data flow framework.

[0071] The technical solution of this invention allows autonomous driving system application layer R&D engineers to focus only on the algorithm logic design in two cases: whether the sensor timestamps are aligned or not. They do not need to judge the timestamp alignment status separately within the algorithm module. Furthermore, it gives autonomous driving system application layer engineers a global view of the impact of whether or not the timestamps are aligned, which greatly reduces the technical threshold for developing autonomous driving systems based on data flow frameworks.

[0072] Exemplary device

[0073] Figure 9 This is a block diagram illustrating a timestamp alignment module based on a data flow framework according to an exemplary embodiment; see reference. Figure 9 The module 900 includes an acquisition unit 910 and a timestamp alignment unit 920.

[0074] The acquisition unit 910 is used to receive the input data stream acquired within the time window; in this embodiment of the invention, the time window needs to be preset according to requirements, and its input data stream is multiple data streams from multiple algorithm modules.

[0075] The timestamp alignment unit 920 is used to obtain an output data stream with timestamp alignment information by passing the input data stream through data stream driving rules. The data stream driving rules include the data stream relationship between the input data stream and the output data stream of each algorithm module.

[0076] In one instance, timestamp alignment information specifically includes trigger flags that determine whether each algorithm module can be triggered. In another instance, timestamp alignment information can also be represented using other identifiers.

[0077] The data flow relationships specifically include at least one of the following: relationships between the input data flows of each algorithm module, relationships between the input and output data flows of each algorithm module, and relationships between different algorithm modules. For example, consider algorithm modules A and B, where algorithm module A includes input data flows a1, a2, a3, and a4, and algorithm module B includes input data flows b1, b2, b3, and b4. Then the data flow relationships specifically include: the relationships between input data flows a1, a2, and a3 of algorithm module A; the relationships between input data flows b1, b2, and b3 of algorithm module B; the relationships between input data flows a1, a2, and a3 and output data flow a4; the relationships between input data flows b1, b2, and b3 and output data flow b4; and the data flow relationships between algorithm modules A and B, such as the relationship between output data flow a4 and input data flow b1.

[0078] Data flow-driven rules specifically include the following three cases:

[0079] Case 1: If all input data streams of the algorithm module arrive within the time window, the algorithm module is triggered; for example, if input data streams a1, a2, and a3 all arrive within the time window, then algorithm module A is triggered.

[0080] Case 2: If a portion of the input data streams in all input data streams of the algorithm module arrive within the time window, the algorithm module or the corresponding sub-algorithm in the algorithm module will be triggered. For example, if input data streams a1 and a2 arrive within the time window, algorithm module A or the corresponding sub-algorithm in algorithm module A will be triggered.

[0081] Scenario 3: Based on the data flow relationships between various algorithm modules, when the input data flow of an algorithm module is the output data flow of a related algorithm module, the triggering of that algorithm module is determined by whether the output data flow of the related algorithm module arrives within the time window. For example, if output data flow a4 is input data flow b1, the triggering of algorithm module B can be determined by whether output data flow a4 arrives within the time window.

[0082] It should be noted that the above three situations can be used individually or in combination as needed.

[0083] In one instance, the timestamp alignment unit 920 is specifically used to: within the time window, based on the data flow-driven rules, determine whether each algorithm module can be triggered and mark the corresponding trigger flag according to whether the corresponding input data flow is received;

[0084] The trigger flags of each algorithm module determine whether to execute the corresponding iterative calculation and / or the corresponding multi-sensor data stream fusion calculation. Iterative methods are a typical type of numerical computation method. Their basic idea is successive approximation: first, a rough approximation is taken, and then the same recursive formula is used to repeatedly correct this initial value until the predetermined accuracy requirement is met. Multi-sensor data stream fusion calculation refers to: fully utilizing multi-sensor data resources from different times and spaces, using computer technology to analyze, synthesize, manage, and use multi-sensor observation data obtained in time series under certain criteria, to obtain a consistent interpretation and description of the measured object, and thus achieve corresponding decision-making and estimation, enabling the system to obtain more comprehensive information than its individual components.

[0085] In this embodiment of the invention, a setting module may also be included, used to set the time window, data flow relationship, data flow driving rule, and trigger flag in a dynamically configurable manner. In one instance, the time window, data flow relationship, data flow driving rule, and trigger flag can be configured and managed (i.e., dynamically configured and managed) through C structures, C++ structures, or classes.

[0086] In summary, the technical solution of this invention proposes a timestamp alignment module driven by a timestamp alignment module. This module is designed at the framework level with a global view attribute, configuring and managing sensor timestamp alignment information, and focusing on the logical judgment of whether timestamps are aligned. It drives and controls each algorithm module in the data stream at the framework level. This invention globally designs the "timestamp alignment module" at the framework level, eliminating the dispersion of timestamp alignment logic within individual algorithm modules. This allows application layer development engineers of autonomous driving systems to conveniently and flexibly configure and manage the timestamp alignment module for the entire autonomous driving system, lowering the technical threshold for developing application layer software for autonomous driving systems based on frameworks like ROS 2.

[0087] Exemplary electronic devices

[0088] Figure 10 This is a block diagram illustrating an electronic device 1000 according to an exemplary embodiment. The electronic device 1000 may be a vehicle controller, an in-vehicle terminal, an in-vehicle computer, or other types of electronic devices.

[0089] Reference Figure 10The electronic device 1000 may include at least one processor 1010 and a memory 1020. The processor 1010 can execute instructions stored in the memory 1020. The processor 1010 is communicatively connected to the memory 1020 via a data bus. In addition to the memory 1020, the processor 1010 may also be communicatively connected to an input device 1030, an output device 1040, and a communication device 1050 via the data bus.

[0090] The processor 1010 can be any conventional processor, such as a commercially available CPU. The processor may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0091] The memory 1020 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0092] In this embodiment of the present disclosure, the memory 1020 stores executable instructions, and the processor 1010 can read the executable instructions from the memory 1020 and execute the instructions to implement all or part of the steps of the timestamp alignment method based on the data flow framework described in any of the exemplary embodiments above.

[0093] Exemplary computer-readable storage media

[0094] In addition to the methods and apparatus described above, exemplary embodiments of this disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in any of the methods in the exemplary embodiments described above.

[0095] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages, and scripting languages ​​(e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0096] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, or any suitable combination thereof.

[0097] Exemplary vehicle

[0098] Figure 11 This is a block diagram illustrating a vehicle 1100 according to an exemplary embodiment. The vehicle 1100 may be a gasoline vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles.

[0099] Reference Figure 11 The vehicle 1100 may include multiple subsystems, such as a drive system 1110, a control system 1120, a sensing system 1130, a communication system 1140, an information display system 1150, and a computing processing system 1160. The vehicle 1100 may also include more or fewer subsystems, and each subsystem may include multiple components, which will not be described in detail here.

[0100] The drive system 1110 includes components that provide power to the vehicle 1100. These include, for example, an engine, an energy source, and a transmission.

[0101] The control system 1120 includes components that provide control for the vehicle 1100. These include, for example, vehicle control, cockpit equipment control, and driver assistance control.

[0102] The perception system 1130 includes components that provide the vehicle 1100 with perception of its surroundings. Examples include a vehicle positioning system, a laser sensor, a voice sensor, an ultrasonic sensor, and camera equipment.

[0103] The communication system 1140 includes components that provide communication connectivity for the vehicle 1100. Examples include mobile communication networks (e.g., 3G, 4G, 5G networks), WiFi, Bluetooth, and vehicle-to-everything (V2X) connectivity.

[0104] The information display system 1150 includes components that provide various information displays for the vehicle 1100. These include, for example, vehicle information displays, navigation information displays, and entertainment information displays.

[0105] The computing processing system 1160 includes components that provide data computing and processing capabilities for the vehicle 1100. The computing processing system 1160 may include at least one processor 1161 and a memory 1162. The processor 1161 can execute instructions stored in the memory 1162.

[0106] Processor 1161 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0107] The memory 1162 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0108] In this embodiment of the disclosure, a set of instructions is stored in the memory 1162, and the processor 1161 can execute the set of instructions to implement all or part of the steps of the timestamp alignment method based on the data flow framework described in any of the exemplary embodiments above.

[0109] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0110] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A timestamp alignment method based on a data flow framework, characterized in that, include: Receive the input data stream acquired within the time window; The input data stream is processed by data flow driving rules to obtain an output data stream with timestamp alignment information. These data flow driving rules include the data flow relationships between the input and output data streams of each algorithm module. Specifically, these data flow relationships include at least one of the following: relationships between the input data streams of each algorithm module, relationships between the input and output data streams of each algorithm module, and relationships between different algorithm modules. The data flow driving rules specifically include: if all input data streams of an algorithm module arrive within the time window, the algorithm module is triggered; if some input data streams of an algorithm module arrive within the time window, the algorithm module or a corresponding sub-algorithm within the algorithm module is triggered; based on the data flow relationships between algorithm modules, when the input data stream of an algorithm module is the output data stream of an associated algorithm module, whether the algorithm module is triggered is determined by whether the output data stream of the associated algorithm module arrives within the time window.

2. The method according to claim 1, characterized in that, The timestamp alignment information includes: a trigger flag that determines whether each algorithm module can be triggered.

3. The method according to claim 2, characterized in that, Specifically, obtaining an output data stream with timestamp alignment information from the input data stream through data stream driving rules includes: Within the time window, based on the data flow-driven rules, it is determined whether each algorithm module can be triggered and the corresponding trigger flag is marked according to whether the corresponding input data flow is received; The trigger flags of each algorithm module determine whether to perform the corresponding iterative calculation and / or the corresponding multi-sensor data stream fusion calculation.

4. The method according to any one of claims 2 to 3, characterized in that, The method further includes: The time window, data flow relationship, data flow driving rules, and trigger flag are configured to be dynamically configurable.

5. The method according to claim 1, characterized in that, The method further includes: The time window, data flow relationship, data flow driving rules, and trigger flags can be configured and managed using C structures, C++ structures, or classes.

6. A timestamp alignment module based on a data flow framework, characterized in that, include: The acquisition unit is used to receive the input data stream acquired within the time window; A timestamp alignment unit is used to process the input data stream through data flow driving rules to obtain an output data stream with timestamp alignment information. The data flow driving rules include the data flow relationships between the input and output data streams of each algorithm module. Specifically, the data flow relationships include at least one of the following: relationships between the input data streams of each algorithm module, relationships between the input and output data streams of each algorithm module, and relationships between different algorithm modules. The data flow driving rules specifically include: if all input data streams of an algorithm module arrive within the time window, the algorithm module is triggered; if some input data streams of an algorithm module arrive within the time window, the algorithm module or a corresponding sub-algorithm within the algorithm module is triggered; based on the data flow relationships between algorithm modules, when the input data stream of an algorithm module is the output data stream of an associated algorithm module, whether the algorithm module is triggered is determined based on whether the output data stream of the associated algorithm module arrives within the time window.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the timestamp alignment method based on the data flow framework as described in any one of claims 1-5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the steps of the timestamp alignment method based on the data flow framework as described in any one of claims 1-5.

9. A vehicle, characterized in that, A set of instructions is stored, which is executed by the vehicle to implement the timestamp alignment method based on a data flow framework as described in any one of claims 1-5.

Images