Time synchronization and latency compensation for simulation test systems

Abstract

A simulation test system and method are provided for time-adjusting a simulated echo signal in response to a periodic electromagnetic signal. The method comprises: generating a scene simulation including a simulated object; generating a simulation frame indicating object transformation information about the simulated object; calculating a position and velocity of a simulated target corresponding to the simulated object using the object transformation information; generating a ray tracing frame corresponding to the simulation frame, the ray tracing frame indicating the calculated position and velocity of the simulated target; predicting a time point in a next feature in a characteristic pattern of the electromagnetic signal; calculating predicted target information of the simulated object at the predicted time point using the calculated position and velocity; and generating an echo signal in response to a next pulse train of the electromagnetic signal using the predicted target information to compensate for latency introduced by the difference between real-world time and simulation time.

Description

Background Art

[0001] Advanced driver assistance systems (ADAS) and autonomous driving systems for vehicles rely on detection and ranging systems that use electromagnetic signals, such as millimeter-wave radar signals and light-based lidar signals. These electromagnetic signals are used to warn of frontal and rear collisions, implement features such as adaptive cruise control and autonomous parking, and ultimately enable autonomous driving on streets and highways. In particular, millimeter-wave automotive radar has an advantage over other sensor systems in that it can operate in most types of weather and in both light and darkness.

[0002] Conventional automotive detection and ranging systems typically have multiple transmitters and multiple receivers on the vehicle. The actual driving environments in which the detection and ranging system can be deployed can be diverse, and many of these driving environments can be complex. For example, the actual driving environment can contain numerous objects, and some of the objects encountered in the actual driving environment have complex reflection, diffraction, and multiple reflection characteristics that affect the return signal in response to the detection and ranging electromagnetic signal. The direct consequence of not correctly sensing and / or decoding the return signal may be the triggering of erroneous warnings or inappropriate reactions, or the failure to trigger a warning or reaction that should have been triggered, which in turn may cause a collision.

[0003] Testing requires ADAS and autonomous driving systems to face safety-critical situations. However, on-road testing is often dangerous and inefficient. Therefore, in-lab testing is preferred for both safety and efficiency. In-lab testing can include driving scenario simulation software and hardware echo signal simulators for different sensors, enabling hardware-in-the-loop (HIL) scenario simulations for multiple sensors.

[0004] Driving scenario simulators and corresponding hardware echo signal simulators take time to generate simulated echo signals for a given sensor. Since the vehicle being tested has multiple detection and ranging electromagnetic signal sensors, HIL multi-sensor driving simulation solutions face two problems. First, the simulated echo signals from different sensors are not synchronized in time. This causes the sensor fusion algorithm on the vehicle to not work properly, because different sensors on the vehicle will receive signals, providing inconsistent information about the simulated environment. Second, the simulated echo signals will arrive at the sensor with a delay compared to the scenario simulation provided by the driving scenario simulator. This delay will cause the decision-making algorithm regarding the vehicle to make decisions later than originally planned, resulting in HIL testing accuracy being less than expected. Therefore, effective technology is needed to ensure that the simulated echo signals from multiple sensors are synchronized and up-to-date in order to provide accurate and reliable sensor testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The exemplary embodiments are best understood from the following detailed description when read in conjunction with the accompanying drawings. It is emphasized that the various features are not necessarily drawn to scale. In fact, for the sake of clarity of discussion, the dimensions may be increased or decreased arbitrarily. Where applicable and feasible, the same reference numerals represent the same elements.

[0006] Figure 1 is a simplified block diagram illustrating a simulation test system for testing vehicle-mounted detection and ranging electromagnetic signals, including time adjustment of simulated echo signals, according to a representative embodiment.

[0007] Figure 2 is a simplified timing diagram illustrating time synchronization and latency compensation of a simulation test system for testing vehicle-mounted detection and ranging electromagnetic signals, according to a representative embodiment.

[0008] Figure 3 is a simplified flow chart illustrating a method for time-adjusting a simulated echo signal in response to a detection and ranging electromagnetic signal to test a vehicle-mounted detection and ranging electromagnetic signal, according to a representative embodiment. DETAILED DESCRIPTION

[0009] In the following detailed description, for the purpose of explanation rather than limitation, representative embodiments of the disclosure details are set forth to provide a thorough understanding of the embodiments according to the present teachings. Descriptions of known systems, devices, materials, operating methods, and manufacturing methods may be omitted to avoid making the description of the representative embodiments difficult to understand. However, such systems, devices, materials, and methods known to those of ordinary skill in the art are within the scope of the present teachings and can be used according to the representative embodiments. It should be understood that the terms used herein are only used to describe specific embodiments and are not intended to be limiting. The defined terms are in addition to the technical and scientific meanings of the defined terms that are generally understood and accepted in the technical field of the present teachings.

[0010] It should be understood that although the terms first, second, third, etc. may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another element or component. Therefore, the first element or component discussed below can be referred to as the second element or component without departing from the teachings of this disclosure.

[0011] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to be limiting. As used in the specification and the appended claims, the singular forms of the terms "a", "an" and "the" include both the singular and the plural forms, unless the context clearly dictates otherwise. In addition, when used in this specification, the terms "include" and / or "comprise" and / or similar terms clarify the presence of the features, elements and / or parts, but do not exclude the presence or addition of one or more other features, elements, parts and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more related enumerated items.

[0012] Unless otherwise specified, when an element or component is referred to as being “connected to,” “coupled to,” or “adjacent to” another element or component, it should be understood that the element or component may be directly connected or coupled to the other element or component, or that intervening elements or components may be present. In other words, these and similar terms include situations where one or more intervening elements or components may be used to connect two elements or components. However, when an element or component is described as being “directly connected” to another element or component, this only includes situations where the two elements or components are connected to each other without any intervening or intervening elements or components.

[0013] The present disclosure is therefore intended to illustrate one or more of the advantages specifically noted below, through its various aspects, embodiments, and / or specific features or subcomponents. For purposes of explanation and not limitation, exemplary embodiments disclosing specific details are set forth to provide a thorough understanding of the embodiments according to the present teachings. However, other embodiments consistent with the present disclosure that depart from the specific details disclosed herein are still within the scope of the appended claims. In addition, descriptions of well-known devices and methods may be omitted so as not to obscure the description of the exemplary embodiments. Such methods and devices are within the scope of the present disclosure.

[0014] According to various embodiments, the driving simulation system can simulate the echo signal in response to the detection and ranging electromagnetic signal emission (e.g., radar and lidar signals) of multiple sensors on the tested vehicle (such as a car or other autonomous vehicle). The embodiment provides a cost-effective high-performance target simulation in the scenario simulation, and the scenario simulation is fully expanded as the number of sensors increases. For example, a common time base is established using precise synchronization technology (such as the Precision Time Protocol (PTP) IEEE 1588 standard (hereinafter referred to as "IEEE 1588"), so that each component of the multi-sensor driving simulation system is synchronized to the sub-microsecond level. As the target information flows through the processing chain from the scenario simulation, the latency caused by the processing and communication operations is accurately monitored. The prediction model executed by the echo signal simulator will be used to project the target information into the near future when the target information is presented to the autonomous vehicle through the simulated echo signal. When the vehicle's sensor is actively detecting the simulated environment, the prediction model is used to project the target information to the precise time (e.g., at the beginning of the radar chirp frame) when the sensor is generating its detection signal.

[0015] As described above, for example, a common time base can be provided according to IEEE 1588 for real-time clock synchronization at a sub-microsecond level. Providing real-time clock synchronization using IEEE 1588 is relatively low cost, highly scalable, easy to program, and requires minimal system communication resources. Other common time bases for real-time clock synchronization can be incorporated (e.g., synchronization via point-to-point trigger cables), provided that the common time base is capable of synchronizing real-time clocks associated with various computer hardware components or other digital electronic devices to within microseconds. Using the common time base, all events of interest can be easily measured, such as the start time of a calculation and the reception time of a communication packet on a network.

[0016] Typically, the driving simulation system according to various embodiments is very accurate because it can accurately simulate a large number of simultaneous targets even in demanding driving scenarios (such as head-on collisions). Moreover, the driving simulation system has consistency because it can maintain the time alignment of the information presented to all sensors in the autonomous vehicle under test and provide loop stability to minimize the loop delay of the HIL. The sensors include, for example, radar sensors, lidar sensors, cameras, and global positioning satellite (GPS) receivers. Considering loop stability, specifically, the driving computer of the vehicle cannot tolerate processing delays in the feedback loop of the driving simulation system. When the vehicle reacts to the signal sensor, it makes decisions and changes, for example, steering, braking, and throttle settings. These changes are immediately detected by the driving simulation system and cause corresponding changes in the simulated scene, which are then detected by the sensors of the vehicle. In order to make performance normal, the delay around this feedback loop must be minimized. By utilizing numerous distributed computing resources (including general-purpose CPUs, GPS, embedded processors, and FPGAs) across the driving simulation system, total computing delay can be minimized by utilizing parallel processing. That is, the overall calculation is appropriately divided into sub-calculations that can be executed in parallel. The sub-calculations can be executed by the driving scenario simulator and / or distributed, for example, to the echo signal emulators of the sensors stimulating the vehicle.

[0017] Regarding accuracy, the computational complexity of the driving scenario simulator is complex, as the scene must be deconstructed to correctly analyze how the vehicle's sensors should respond. For example, radar and lidar signals are very different, yet they are related to the scene image detected by the vehicle's camera. Although running in parallel, these calculations are complex and result in computational delays that must be compensated for by other components of the driving simulation system.

[0018] Regarding consistency, the computational latency of each sensor on a vehicle may differ. For example, image information provided by a camera is relatively fast compared to the corresponding information provided by radar. When presenting this information to the corresponding sensor, it must be corrected to the correct real-world time using forward propagation techniques. Knowing the precise mapping between simulation time and real-world time, as well as the expected time in the near future when the information should be presented to the vehicle's sensors, the desired corrections can be performed. This involves calculating the position and velocity of each target at least in the near future, and may further include calculating the size, shape, and / or acceleration of each target at the near future real-world time.

[0019] Figure 1is a simplified block diagram illustrating a simulation test system for testing vehicle-mounted detection and ranging electromagnetic signals (including time adjustment of simulated return signals) according to a representative embodiment. As will be appreciated by those of ordinary skill in the art having the benefit of this disclosure, the embodiments are applicable to various types of detection and ranging electromagnetic signals, including, for example, vehicle radars and vehicle lidars (referred to as "electromagnetic signals") used in various capacities in current and emerging automotive applications. However, it is emphasized that the simulation test system currently described is not limited to automotive radar and lidar systems, and can be applied to other types of electromagnetic signals (including imaging signals for cameras) and other types of vehicles, including buses, motorcycles, motorized bicycles (e.g., scooters), boats, and aircraft that can employ vehicle-mounted radar systems.

[0020] refer to Figure 1 , the simulation test system 100 is arranged to test multiple devices under test (DUTs), which can be, for example, radar sensors or laser radar sensors located on a vehicle under test 105. Although the representative first DUT 101, the second DUT 102, and the xth DUT 103 indicate DUTs, without departing from the scope of this teaching, the depicted embodiment can be used to test any number of DUTs. The first DUT 101, the second DUT 102, and the xth DUT 103 can be connected to a controller area network (CAN) bus 108 on the vehicle under test 105. The vehicle under test 105 also includes an electronic control unit (ECU) software and / or an ADAS computer (not shown), which provides vehicle control signals such as braking, throttling, and steering to the simulation test system 100 as feedback for implementing scenario simulation, as discussed below. As indicated by the dotted lines, the vehicle under test 105 can be connected to the simulation test system 100 via various types of wired and / or wireless network connections.

[0021] The first DUT 101, the second DUT 102, and the x-th DUT 103 transmit periodic electromagnetic signals such as radar signals or lidar signals to the simulation test system 100, and receive corresponding simulated echo signals generated by the simulation test system 100 in response. The periodic electromagnetic signal may have a periodic characteristic pattern, such as a repetitive pulse train of RF energy or light energy (laser pulses), and include signal frames, each signal frame having at least one feature of the characteristic pattern in the electromagnetic signal. For example, a radar signal may have an RF pulse train pattern in a series of radar frames, wherein each radar frame includes two pulse trains with a chirped frequency. Of course, depending on the design characteristics of the radar signal, the radar frame may include fewer or more chirped frequency pulse trains, which is obvious to a person skilled in the art. The lidar signal may include a signal frame in which the energy pulse train pattern includes laser pulses.

[0022] The simulation test system 100 includes a plurality of echo signal simulators, indicated by a representative first echo signal simulator 111, a second echo signal simulator 112, and a yth echo signal simulator 113. Without departing from the scope of the present teachings, although the number of echo signal simulators in use is generally the same as the number of DUTs under test (i.e., y equals x), the simulation test system 100 may also use more or fewer echo signal simulators than DUTs under test. The first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 are implemented as a combination of hardware and software that are configured to receive electromagnetic signals (e.g., radar or lidar signals) from the first DUT 101, the second DUT 102, and the xth DUT 103, respectively, and transmit corresponding simulated echo signals from the simulated object to the first DUT 101, the second DUT 102, and the xth DUT 103 in response to the received electromagnetic signals. Without departing from the scope of the present teachings, the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 can be any type of known echo signal simulator capable of receiving an electromagnetic signal and generating a corresponding simulated echo signal in response, such as a modulated reflectometry device (MRD). Each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 includes one or more processing units for performing software configuration and IEEE 1588 time synchronization, as discussed below.

[0023] Each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 includes or has access to a prediction model 115, which is configured to predict a time point in the electromagnetic signal and calculate a predicted position and velocity of a simulated target at a predicted time point in the near future based on object transformation information from the scenario simulation. Generally, the prediction model 115 has two functions. First, the prediction model 115 predicts the time point of the next incoming feature (e.g., a radar pulse train) of the frame. Generally, although the electromagnetic signal is periodic, the sensor (DUT) manufacturer may intentionally add some random deviation. Because the electromagnetic signal is periodic, the timing of the next incoming feature can be predicted using an established pattern of features and / or frames including the features, the pattern including the feature length and the time lapse between adjacent occurrences of the features. For example, when the predicted time point is at the beginning of a feature in the electromagnetic signal, the time can be predicted by adding the known time lapse between features to the measured time of the start of the previous occurrence of the feature. The pattern can be established by measuring actual electromagnetic signals or using sensor specifications of the first DUT 101, the second DUT 102 and the x-th DUT 103. The established pattern can then be stored in the corresponding first echo signal simulator 111, the second echo signal simulator 112 and the y-th echo signal simulator 113 for access by the prediction model 115. Secondly, the prediction model 115 propagates and / or extrapolates the position and velocity of the simulated target forward to the prediction time point. That is, by providing the position, velocity and (optional) acceleration of the simulated target at time point t, the prediction model 115 is able to determine the position and velocity of the simulated target at the prediction time point t+ΔT, for example, as known to those skilled in the art by using kinematic formulas, where t is the time associated with the generated object transformation information. In the following reference Figure 2 In the embodiment discussed, for example, ΔT=ΔTa+ΔTb+ΔTc+ΔTd+ΔTe. When propagating the position and velocity forward and / or extrapolating to a predicted point in time, the acceleration is assumed to be constant.

[0024] exist Figure 1In the embodiment of the present invention, a prediction model 115 is shown in each of the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113, for example, which can be executed by one or more processing units. However, it should be understood that the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113 can access a common prediction model without departing from the scope of the present teachings. Each of the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113 includes one or more antennas to receive electromagnetic signals from the first DUT 101, the second DUT 102, and the x-th DUT 103 to illuminate the simulated target, and in response, transmits a simulated echo signal from the simulated target. Although the one or more antennas can be, for example, horn antennas, other types of antennas, such as patch antennas or patch antenna arrays, can also be incorporated.

[0025] Each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 further includes a receiver connected to one or more antennas to receive electromagnetic signals from one of the first DUT 101, the second DUT 102, and the xth DUT 103, respectively. The receiver includes an analog-to-digital converter (ADC) so that each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 provides a corresponding digitized electromagnetic signal for further processing. Each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 further includes a transmitter connected to one or more antennas to transmit the simulated echo signal to the same one of the first DUT 101, the second DUT 102, and the xth DUT 103. The transmitter includes a digital-to-analog converter (DAC), enabling each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 to transmit a corresponding simulated echo signal that simulates reflections from one or more simulated targets. The implementation of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 for receiving and transmitting signals for various media, including radar and lidar, is well known to those skilled in the art. Each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 includes a precise timing hardware timestamp function, such as IEEE 1588.

[0026] The simulation test system 100 further includes a driving scenario simulator 120 implemented as software stored and executed by one or more processing units of a driving scenario simulation server 125; and a ray tracing engine 130 implemented as software stored and executed by one or more processing units of a ray tracing engine server 135. The one or more processing units used for the driving scenario simulation server 125 and the ray tracing engine server 135 may include a network interface card, for example, a network interface card with a precise timing hardware timestamp function (such as IEEE 1588). In one embodiment, each of the first DUT 101, the second DUT 102, and the xth DUT 103 discussed above may have its own duty cycle in terms of transmitting electromagnetic signals, which duty cycle is different from the duty cycle of the driving scenario simulator 120 and the ray tracing engine 130.

[0027] The simulation test system 100 may further include a switch 140 implemented as hardware and / or software to connect the driving scenario simulator 120 and the ray tracing engine 130, and selectively direct target information for generating simulated echo signals from the ray tracing engine 130 to the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113. Therefore, the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 can generate corresponding echo signals using electromagnetic signals received from the first DUT 101, the second DUT 102, and the xth DUT 103, as well as the target information received from the ray tracing engine 130 via the switch 140. The switch 140 has the same precise timing hardware timestamp function as the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113, such as IEEE 1588. The switch 140 may be, for example, an Ethernet switch with IEEE 1588 hardware timestamp feature functionality. Thus, the switch 140 provides general Ethernet data communication and IEEE 1588 time synchronization support.

[0028] The driving scenario simulator 120 is programmed to generate a simulation of a real-world driving scenario, which includes simulated objects in the environment in which the test vehicle 105 is to drive. Simulated objects may include other vehicles, pedestrians, street signs, curbs, utility poles, foreign objects in the vehicle's path, and the like. The driving scenario simulator 120 acquires and / or determines three-dimensional (3D) geometric information and object transformation information (data) based on one or more mathematical models for the scenario simulation, wherein the one or more mathematical models are well known to those skilled in the art. The 3D geometric information is predefined and may be stored in a simulation scenario file, which is loaded by the driving scenario simulator 120 prior to the scenario simulation. The 3D geometric information may include a list of objects in the simulated scenario, as well as information about each simulated object, such as its position and whether each object is stationary or movable, and data indicating the 3D surface of each object. The driving scenario simulator 120 forwards the 3D geometric information to the ray tracing engine 130 upon initialization, and the 3D geometric information does not change after the scenario simulation begins. The object transformation information for the simulated objects indicates the transformation that gives the simulated objects their dynamic position. The object transformation information includes the position of the simulated object and the yaw, pitch and roll of the (movable) simulated object. The driving scene simulator 120 continuously or periodically updates the object transformation information while the scene simulation is running, and the scene simulation provides simulation frames containing the object transformation information at a certain frame rate.

[0029] Since the driving scenario simulator 120 may experience computational jitter due to the underlying operating system, the simulation time is mapped to the real-world time experienced by the vehicle under test. Therefore, each simulation frame has a simulation time in the scenario simulation and a corresponding real-world time, which may be different from the simulation time, as described below with reference to Figure 2 discussed.

[0030] The driving scenario simulator 120 first loads the 3D geometry information of a predefined scene from a file before the scene simulation and outputs object transformation information to the ray tracing engine 130 during the scene simulation. For example, the driving scenario simulator 120 can be configured to generate a series of simulation frames, each of which includes object transformation information for one or more simulated objects. The simulation frames are generated based on a simulation time and frame rate provided by the driving scenario simulator 120. That is, the driving scenario simulator 120 maintains a simulation timebase referenced to an arbitrary start time of the simulation. Thus, for example, simulation frame N is valid at a simulation time of N*frame rate. Therefore, each simulation frame has a corresponding simulation time associated with the frame length. For example, the simulation timebase can be a common timebase provided by IEEE 1588.

[0031] Simulation time is typically different from real-world time, which is the actual time at which the vehicle under test 105, including the first DUT 101, the second DUT 102, and the x-th DUT 103, is operating. Although in practice, simulation time is approximately equal to real-world time, it may differ enough to create an offset between the two times, thereby causing latency or affecting the accuracy of the scenario simulation. Therefore, each of the simulation frames has a real-world time corresponding to the simulation time associated with the frame. This real-world time is the time actually used to generate the simulation frame. In one embodiment, the simulation time is fixed, while the real-world time is variable. Alternatively, both the simulation time and the real-world time can be variable.

[0032] The ray tracing engine 130 is configured to first receive 3D geometry information before scene simulation, then receive object transformation information about the simulated object from the simulation frame and calculate target information during the scene simulation. The calculated target information includes one or more of the updated position, velocity, acceleration, and cross-section (e.g., RCS) of the simulated target corresponding to the simulated object based on the received object transformation information. The simulated target refers to the echo of the target information displayed to show the reflective surface of the corresponding simulated object. Therefore, it should be understood that references to target information include the calculated position, velocity, acceleration, and cross-section of the simulated target. The ray tracing engine 130 generates ray tracing frames corresponding to the simulation frames indicating the target information of the simulated target based on the object transformation information provided in the corresponding simulation frame. The frame rate of the ray tracing frame has no fixed relationship with the frame rate of the simulation frame. The processing unit for the ray tracing engine 130 (e.g., the ray tracing engine server 135) may also include a network interface card, such as a network interface card with precise timing hardware timestamp function (such as IEEE1588). The ray tracing engine 130 may use the scene 3D geometry first received from the driving scene simulator 120 before the active simulation begins and use the signal object transformation information received from the driving scene simulator 120 during the active simulation on a simulation frame-by-simulation frame basis to output target information, including the position, velocity, acceleration, and cross-section of each simulated object.

[0033] The ray tracing engine 130 outputs the ray tracing frames to the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 through the switch 140. The first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 generate simulated echo signals using electromagnetic signals received from the first DUT 101, the second DUT 102, and the xth DUT 103, and target information received from the ray tracing engine 130. As described above, each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 includes (or can access) a prediction model 115 implemented as software stored and executed by one or more processing units, and may include, for example, a network interface card with a precise timing hardware timestamp function (such as IEEE 1588). Therefore, the first to y-th echo signal simulators 111 , 112 , and 113 , the driving scene simulator 120 , and the ray tracing engine 130 operate from a common time base to synchronize real-world time at a sub-microsecond level.

[0034] The driving scenario simulator 120, the ray tracing engine 130, and the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113, together with the first DUT 101, the second DUT 102, and the xth DUT 103, the ECU software of the vehicle under test 105, and the ADAS computer, provide a HIL system for feedback, as known to those skilled in the art. For example, in the HIL system, brake, throttle, and steering signals are fed back from the vehicle under test 105 to the driving scenario simulator 120 to be incorporated into the scenario simulation, such as Figure 1 Indicated by the dotted line in .

[0035] Prediction model 115 is programmed to predict a list of simulation targets to be used by first echo signal simulator 111, second echo signal simulator 112, and yth echo signal simulator 113 to provide simulated echo signals at future times. That is, for each ray tracing frame, prediction model 115 uses features from the imported electromagnetic signal to predict a time point in the near future. "Near future" refers to a time period that is less than a predetermined period of the (periodic) electromagnetic signal, such as the frame length of a frame or the burst length of a burst within a frame. Therefore, while the near future is practically on the order of milliseconds for most types of electromagnetic signals, the timing of the predicted time point determined for the near future depends on the specific characteristics of the electromagnetic signal. For example, the feature may be an energy burst, allowing prediction model 115 to predict the time point in the next burst of the input electromagnetic signal received by one of first echo signal simulator 111, second echo signal simulator 112, and yth echo signal simulator 113 from one of first DUT 101, second DUT 102, and xth DUT 103. When prediction model 115 receives a specific ray tracing frame, it determines the next pulse train as the next complete pulse train to be fed into a corresponding one of first echo signal simulator 111, second echo signal simulator 112, and y-th echo signal simulator 113. While the prediction time point can be the start, middle, or end point of the next pulse train, other time points within the next pulse train or within the frame containing the next pulse train can also be used. The energy pulse train can be, for example, an RF energy pulse train for radar or a laser energy (laser pulse) pulse train for lidar.

[0036] The prediction model 115 then uses the calculated target information of the simulation target determined by the ray tracing engine 130 to calculate the predicted target information of the simulation target at a predicted time point in the near future. That is, the prediction model 115 uses the position, velocity, and acceleration of the simulation target in the latest ray tracing frame to propagate and / or extrapolate the position and velocity of the simulation target forward to the predicted time point, while the acceleration is kept constant. Therefore, the prediction model 115 is driven by an electromagnetic signal (e.g., a periodic pulse train) rather than by the ray tracing engine 130. The predicted target information includes the position, velocity, and (optionally) acceleration of the simulation target, and depending on the scope and purpose of the scene simulation, may further include a cross-section of the simulation target.

[0037] Prediction model 115 indicates predicted target information for a simulated target before one of first echo signal simulator 111, second echo signal simulator 112, and yth echo signal simulator 113 receives the next burst of electromagnetic signals. Prediction model 115 provides a prediction segment that uses target information from the most recent ray tracing frame before target information for the next ray tracing frame becomes available. During each prediction segment, one or more bursts of electromagnetic energy will arrive at first echo signal simulator 111, second echo signal simulator 112, and yth echo signal simulator 113, and prediction model 115 will use target information from the same ray tracing frame to propagate and / or extrapolate the position and velocity of the simulated target to the predicted time point. When prediction model 115 calculates the predicted target information for the simulated target at the predicted time point, it compensates for delays introduced by each of the simulation frame, driving scenario simulator 120, and ray tracing engine 130. It is noteworthy that the operations of the first echo signal simulator 111 , the second echo signal simulator 112 , and the yth echo signal simulator 113 , the driving scene simulator 120 , and the ray tracing engine 130 are synchronized with the precise timing hardware timestamp capability at the sub-microsecond level.

[0038] As described above, each of at least the driving scene simulator 120, the ray tracing engine 130, and the prediction model 115 in the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 is implemented by one or more processing units. As used herein, a processing unit may include one or more computer processors, digital signal processors (DSPs), central processing units (CPUs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or combinations thereof using any combination of hardware, software, firmware, hard-wired logic circuits, or combinations thereof. A processing unit may include its own processing memory to store computer-readable code (e.g., software, software modules, software engines) capable of performing the various functions described herein. For example, the processing memory may store executable by a processing unit (e.g., a computer processor) to perform some or all aspects of the method described herein (including those described below with reference to Figure 3 The software instructions / computer readable code of each step of the method described herein.

[0039] As used herein, the processing memory and any other memory (and database) described herein may be various types of random access memory (RAM), read-only memory (ROM), and / or other storage media, including flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), registers, latches, flip-flops, hard disk, removable disk, magnetic tape, floppy disk, Blu-ray disc, or universal serial bus (USB) drive, or any other form of storage media known in the art that is tangible and non-transitory (e.g., as compared to a transitory propagating signal). Memory may be volatile or non-volatile, secure and / or encrypted, unsecure and / or unencrypted without departing from the scope of the present teachings.

[0040] Figure 2 is a simplified timing diagram illustrating time synchronization and latency compensation for a simulation test system for testing vehicle-mounted detection and ranging electromagnetic signals according to a representative embodiment. Each of rows (1) to (10) shows the time synchronization and latency compensation for a simulation test system for testing vehicle-mounted detection and ranging electromagnetic signals according to a representative embodiment. Figure 1 The signal output by one of the components in question. Rows (1) to (10) relate to a time axis along the horizontal axis, which is indicated in microseconds. The time axis is not an absolute time, but is intended to represent a point in time. For example, for the first point in time at zero on the time axis, when the scenario simulation is running, the absolute timestamp is recorded as simulation time zero. The absolute timestamp at the start of the identified scenario simulation can be an IEEE 1588 timestamp, such as that represented by T 1588_0 instruct.

[0041] refer to Figure 2 , row (1) shows the simulation frames determined by the driving scene simulator 120 in simulation time. In the depicted example, row (1) shows eight representative simulation frames, labeled m, m+1, m+2, m+3, m+4, m+5, m+6, and m+7. Again, for simplicity, it is assumed that the simulation time of each simulation frame is fixed, indicated by simulation frames having the same length. However, it should be understood that the simulation time of the simulation frames may be variable without departing from the scope of the present teachings because the simulation frames are provided by the driving scene simulator 120. The fixed simulation time of each simulation frame is the inverse (1 / f) of the frame rate at which the driving scene simulator 120 generates the simulation frame. As discussed above, the simulation frame includes object transformation information from the scene simulation. The object transformation information includes the position, yaw, pitch, and roll of one or more simulation objects calculated by the driving scene simulator 120. The object transformation information is sent to the ray tracing engine 130, as discussed below. Although in Figure 2Not shown in the figure, but for example before the scene simulation starts, the driving scene simulator 120 also first sends the 3D geometry information to the ray tracing engine 130.

[0042] Row (2) shows the real-world frames output by the driving scene simulator 120 in real-world time. In the depicted example, row (2) shows eight representative real-world frames corresponding to eight representative simulation frames, also labeled m, m+1, m+2, m+3, m+4, m+5, and m+6. That is, the real-world frames are simulation frames adjusted to real-world time. Therefore, like the corresponding simulation frames, the real-world frames also include object transformation information from the scene simulation.

[0043] Because the driving scenario simulator 120 is a non-real-time system, the actual time required to generate a simulation frame in real-world time may vary depending on, for example, the information for any particular frame and the time required to assemble the frame. Moreover, the real-world time of the real-world frames is variable, as indicated by some real-world frames having a different length than other frames and / or having a different length than the corresponding simulation frames. In the described example, for illustrative purposes, the real-world time of real-world frame m is longer than the simulation time of the corresponding simulation frame m by a time difference ΔTa. For example, referring to the timeline, simulation frame m should end at time t4, but actually ends at time t 4.5 End, making the time difference ΔTa 0.5. 4.5 Record IEEE 1588 timestamp T 1588_m , to mark the end of the real-world frame m. In other words, the simulation time lags behind the real-world time, and the time difference ΔTa = T 1588_m -T 1588_0 -m / f, where m is the number of frames, and f is the simulation frequency in the simulation timeline, and m / f is the simulation time elapsed since the start of the simulation.

[0044] Because the driving scene simulator 120 is not a real-time simulator in various embodiments, the simulation frequency f is the frequency required by the driving scene simulator 120. The real-world time the driving scene simulator 120 spends calculating each simulation frame will vary. For complex simulation frames that include information about more simulated objects than less complex simulation frames, the driving scene simulator 120 spends more time on calculations. Therefore, for more complex simulation frames (e.g., the mth frame), the driving scene simulator 120 spends more time than 1 / f (e.g., 2 / f) to calculate the simulation frame. The driving scene simulator 120 can then ignore the next (e.g., the (m+1)th) simulation frame calculation and continue with the subsequent (e.g., the (m+2)th) simulation frame calculation after it completes the mth frame calculation. This process can be referred to as catching up with the real-world timeline.

[0045] Although some driving scene simulators can support simulation rate changes such as half (x0.5) or doubling (x2), the driving scene simulator 120 in the embodiments described herein maintains the same simulation rate (x1). This means that the simulation timeline and the real-world timeline should have the same frequency in ideal cases. Of course, in fact there will be time deviation, which will affect the HIL of the simulation test system 100. Therefore, the driving scene simulator 120 can adjust the simulation timeline during the scenario simulation by skipping one or more next simulation frames. For example, when the simulation frame spends twice (2x) of the allocated time to generate, the driving scene simulator 120 can skip the generation next simulation frame that will originally occur in the simulation time, and generate the simulation frame at a later simulation time, so that the simulation time will not lag behind the real-world time when there is no chance to catch up. In other words, the driving scene simulator 120 can be programmed to skip one or more next simulation frames when it spends too much time to generate the previous simulation frame (for example, when more simulation objects are visible in the previous simulation frame), so that the simulation time can catch up with the real-world time.

[0046] The simulated frames (as time-adjusted actual frames) are sent to the ray tracing engine 130, which generates corresponding ray traced frames. Figure 2 As shown in , the ray tracing engine 130 may not use information from all simulation frames in order to prevent excessive latency from accumulating during processing. The ray tracing engine 130 continuously receives simulation frames, storing the most recent simulation frame in memory. When ready to generate the next ray tracing frame, the ray tracing engine 130 uses only the information in the most recently stored simulation frame from the driving scenario simulator 120 in real-world time. In the depicted example, simulation frames m, m+2, and m+5 are processed by the ray tracing engine 130 as discussed below, while simulation frames m+1, m+3, m+4, and m+6 are not processed. It is worth noting that the driving scenario simulator 120 runs at its own frequency and is independent of the frequency of the ray tracing engine.

[0047] Row (3) shows the ray tracing frames output by the ray tracing engine 130 in real-world time. In the depicted example, row (3) shows three representative ray tracing frames, labeled n, n+1, and n+2. Ray tracing frame n corresponds to simulation frame m, ray tracing frame n+1 corresponds to simulation frame m+2, and ray tracing frame n+2 corresponds to simulation frame m+5. Specifically, with respect to simulation frame m, the ray tracing engine 130 receives simulation frame m, uses object transformation information from simulation frame m to calculate the position, velocity, acceleration, and cross-section (e.g., RCS) of one or more simulation objects, and generates ray tracing frame n, which includes the calculated position, velocity, acceleration, and cross-section as calculated target information. In general, a ray tracing frame has no fixed relationship to any particular simulation frame (or corresponding real-world frame). Rather, the ray tracing engine 130 selects the most recent simulation frame result provided by the driving scene simulator 120 for calculation. As discussed below, the effects of any additional delay will subsequently be compensated by the prediction model.

[0048] Although a ray traced frame may be longer than a corresponding simulation frame (and real-world frame), the actual time required to generate a ray traced frame in real-world time may vary depending on the time required to calculate the corresponding cross-section, position, velocity, and / or acceleration information and to assemble the ray traced frame. In the depicted example, the amount of time ray traced engine 130 takes to perform the calculations and assemble the ray traced frame n is indicated by the time difference ΔTb, while the length of the complete ray traced frame n is indicated by the time difference ΔTc, both in real-world time. More specifically, ray traced frame n is generated at time t 5.0 Start and at time t 15.0 Finish.

[0049] Line (4) shows that Figure 2 Frames of a periodic electromagnetic signal continuously transmitted by one of the first DUT 101, the second DUT 102, or the xth DUT 103 during the relevant time period depicted in . For illustrative purposes, the electromagnetic signal is a periodic radar signal, and the frames are radar frames, where each radar frame includes two radio frequency (RF) energy bursts. For example, the first RF energy burst of each radar frame can be a long-range radar chirp signal (LONG burst) at a first chirp frequency, while the second RF energy burst can be a medium-range radar chirp signal (MED burst) at a second chirp frequency. Of course, it will be apparent to those skilled in the art that the arrangement of periodic radar and other types of electromagnetic signals can be varied to provide unique benefits for any particular situation or to meet application-specific design requirements of various embodiments.

[0050] Specifically, row (4) shows a first radar frame having a long pulse train (p) and a medium pulse train (p), a second radar frame having a long pulse train (p+1) and a medium pulse train (p+1), a third radar frame having a long pulse train (p+2) and a medium pulse train (p+2), and a fourth radar frame having a long pulse train (p+3) and a medium pulse train (p+3). As shown, the radar frame is being transmitted by one of the first DUT 101, the second DUT 102, or the x-th DUT 103, while the driving scenario simulator 120 and the ray tracing engine 130 are providing the simulated / real-world frame and the ray tracing frame. The radar frame is being received by a corresponding one of the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113, as discussed below with reference to row (6).

[0051] Line (5) shows a prediction segment of the prediction model, which calculates predicted target information of the simulated targets at a point in time in the near future based on the calculated target information provided by the ray tracing frame. The predicted target information includes at least the position and velocity of each of the simulated targets from the ray tracing frame, and may further include the size and shape of the simulated targets to provide RCS, for example, depending on the scope and purpose of the target simulation.

[0052] In the depicted example, row (5) shows two representative prediction segments, labeled r and r+1, generated by the prediction model 115 and executed by one or more of the first echo signal simulator 111, the second echo signal simulator 112, or the yth echo signal simulator 113. Prediction segment r corresponds to ray tracing frame n, while prediction segment r+1 corresponds to ray tracing frame n+1. Specifically, with respect to prediction segment r, the prediction model 115 receives ray tracing frame n from the ray tracing engine 130, predicts a time point in the next consecutive pulse train of the electromagnetic signal, and calculates predicted target information of the simulation target at the predicted time point, as discussed below. In the camera simulation test system, the prediction model can be run in the corresponding software that renders the three-dimensional scene, and the corresponding video stream is output after the prediction model.

[0053] A prediction segment may be longer, shorter, or equal to the corresponding ray tracing frame because the real-world time required to generate a prediction segment may vary depending on the number of bursts addressed by a particular prediction segment and the time required to calculate the predicted time point of the simulation target and the predicted target information for each of these bursts. In the depicted example, the amount of time that the prediction model 115 begins performing calculations is indicated by the time ΔTd in real-world time. In the depicted example, the prediction segment r is generated at time t 15.5 start.

[0054] Row (6) shows a periodic electromagnetic signal that starts after the start time of the predicted segment r, and is received by one of the first echo signal simulator 111, the second echo signal simulator 112, or the yth echo signal simulator 113, as transmitted in row (4). In fact, the timing of receiving the periodic electromagnetic signal is substantially matched with the timing of transmitting the periodic electromagnetic signal, and therefore, the pulse train pattern shown in row (6) is substantially the same as the pulse train pattern shown in row (4). That is, row (6) shows the LONG pulse train (p+2) and the MED pulse train (p+2) of the third radar frame, and the LONG pulse train (p+3) and the MED pulse train (p+3) of the third radar frame. The predicted segment is independent of the frame of the periodic electromagnetic signal. For example, in the depicted example, prediction fragment r performs prediction on the LONG pulse string (p+2) and MED pulse string (p+2) in the third frame and the LONG pulse string (p+3) in the first part of the fourth frame, while prediction fragment r+1 performs prediction on the MED pulse string (p+2) in the second part of the fourth frame and the pulse string from the next frame (not shown).

[0055] The prediction segment provided by the prediction model 115 may include calculations for multiple pulse trains of the electromagnetic signal, depending on the length of the prediction segment and the timing of the pulse train pattern in the electromagnetic signal. That is, during this process, the prediction model 115 determines the predicted time point of the next incoming pulse train and performs forward propagation and / or extrapolation of the pulse train on a pulse train by pulse train basis. The ray tracing engine 130 provides only target information to the prediction model 115. For example, the illustrative prediction segment r shown in row (5) provides calculation results for three consecutive pulse trains of the electromagnetic signal shown in row (6). Therefore, the prediction model 115 predicts the first predicted time point T1 in the LONG pulse train (p+2), the second predicted time point T2 in the MED pulse train (p+2), and the third predicted time point T3 in the LONG pulse train (p+3). Because the electromagnetic signal received by one or more of the first echo signal simulator 111, the second echo signal simulator 112, or the yth echo signal simulator 113 is periodic, the prediction model 115 is able to predict these time points. The prediction model 115 calculates the predicted target information for the simulation target at each of the first predicted time point T1, the second predicted time point T2, and the third predicted time point T3. In the depicted embodiment, although each of the first predicted time point T1, the second predicted time point T2, and the third predicted time point T3 is shown as an intermediate point of the corresponding next consecutive pulse train of the received electromagnetic signal, other points in the next pulse train (points in other features of the electromagnetic signal) may also be used without departing from the scope of the present teachings. Typically, the prediction model 115 runs in the background and predicts the predicted time point of the next incoming pulse train. The prediction model 115 then uses the position, velocity, and acceleration information of the latest ray tracing frame to propagate forward and / or extrapolate to obtain position and velocity information at the predicted time point.

[0056] Specifically, regarding the LONG burst (p+2), the prediction model 115 predicts the first prediction time point T1. In the depicted example, the start time of the predicted segment r is time t 15.5 The start and end times of the LONG pulse train (p+2) are t 16.5 and t 18.5 , and the first prediction time point T1 is t 17.5 The prediction model 115 calculates predicted target information of the simulation target at the first prediction time point T1 using the cross section, position, velocity and / or acceleration of the simulation target calculated from the ray tracing frame n, and includes the predicted target information in the first prediction time point T1 and the prediction segment r.

[0057] As described above, the prediction model 115 predicts the first prediction time point T1 and calculates the predicted target information of the simulation target at the first prediction time point T1 before one or more of the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113 receive the LONG pulse train (p+2). Line (7) indicates such processing performed by one or more of the first echo signal simulator 111, the second echo signal simulator 112, and the y-th echo signal simulator 113. That is, the time required for the prediction model 115 to perform the prediction of the first prediction time point T1 and the calculation of the predicted target information of the simulation target is indicated by the first processing time ΔTp1. In the depicted example, although the first processing time ΔTp1 starts at t 16.0 and ends at t 16.5 , the LONG pulse train (p+2) is received at about the same time, but the time required for the first processing time ΔTp1 can vary as long as the processing is completed before the LONG pulse train (p+2) is received. 17.5 Record IEEE1588 timestamp T 1588_T1 , to mark the first prediction time point T1. In the same way, the time required to perform the prediction and associated calculations for the second prediction time point T2 in the MED pulse train (p+2) is indicated by the second processing time ΔTp2, and the time required to perform the prediction and associated calculations for the third prediction time point T3 in the LONG pulse train (p+3) is indicated by the third processing time ΔTp3. Similarly, with respect to the next prediction fragment r+1, the time required to perform the prediction and associated calculations for the fourth prediction time point T4 in the MED pulse train (p+3) is indicated by the fourth processing time ΔTp4.

[0058] Line (8) indicates that the simulation test system 100 determines the total time delay (waiting time) of at least the predicted target information of the simulation target at the first predicted time point T1 in the LONG pulse train (p+2) based on the object transformation information provided in the simulation frame m. As shown, the total time delay from the end point of the simulation frame m output by the driving scene simulator 120 to the LONG pulse train (p+2) received by one or more of the first echo signal simulator 111, the second echo signal simulator 112 or the yth echo signal simulator 113 is the sum of the above-mentioned time differences ΔTa, ΔTb, ΔTc, ΔTd and ΔTe. Time is tracked using a timestamp provided by a common time base (e.g., according to IEEE 1558). In this regard, the total delay time can be determined as the IEEE 1588 timestamp T corresponding to the first predicted time point T1. 1588_T1 The IEEE 1588 timestamp T corresponding to the end point of the simulation frame m 1588_mTherefore, the prediction model gives the prediction target information based on the target transformation information, which is provided before ΔTa+ΔTb+ΔTc+ΔTd+ΔTe. The same is true for determining the total time delay of the second prediction time point T2 and the third prediction time point T3 using the prediction frame r, wherein the time difference between the first prediction time point T1 and the second prediction time point T2 is added to determine the total time delay at the second prediction time point T2, and the time difference between the second prediction time point T2 and the third prediction time point T3 is further added to determine the total time delay at the third prediction time point T3.

[0059] The simulation test system 100 supports pause and resume functionality. For example, when calculating ΔTa+ΔTb+ΔTc+ΔTd+ΔTe, IEEE 1588 absolute timestamps corresponding to the pause and resume times can be included. The duration between the pause and resume times can then be subtracted. This subtraction can be performed for multiple pause / resume operations. During the pause state, the processing logic and prediction model simulation of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 can be notified that the simulation is paused, causing them to stop predicting and use the last received prediction target information.

[0060] Row (9) shows simulated echo signals emitted by one or more of the first echo signal emulator 111, the second echo signal emulator 112, and the yth echo signal emulator 113 in response to the periodic electromagnetic signal received in row (6). The simulated echo signals include echo pulses corresponding to the energy pulses in the periodic electromagnetic signal. Therefore, the simulated echo signals include a long echo pulse train (q+2), a medium echo pulse train (q+2), a long echo pulse train (q+3), and a medium echo pulse train (q+3), which are also arranged in frames and generated in response to the long pulse train (p+2), the medium echo pulse train (p+2), the long pulse train (p+3), and the medium echo pulse train (p+3), respectively. The predicted target information calculated for the first predicted time point T1 is used to generate a LONG echo pulse train (q+2), the predicted target information calculated for the second predicted time point T2 is used to generate a MED echo pulse train (q+2), and the predicted target information calculated for the third predicted time point T3 is used to generate a LONG echo pulse train (q+3).

[0061] Row (10) shows the simulated echo signal shown in row (9) received by one or more of the first DUT 101, the second DUT 102, and the xth DUT 103 on the vehicle under test 105. In practice, the timing of receiving the simulated echo signal substantially matches the timing of transmitting the simulated echo signal, so the echo pulse shown in row (10) is substantially the same as the echo pulse shown in row (9), thereby ignoring the minimum propagation delay of the electromagnetic signal. The vehicle under test 105 reacts to the received simulated echo signal, for example, by adjusting the steering, braking, throttle setting, etc., in response to the scenario simulation generated by the driving scenario simulator 120, just as it would appear when the first DUT 101, the second DUT 102, and the xth DUT 103 receive the simulated echo signal in real-world time.

[0062] It is noteworthy that because the driving scenario simulator 120, the ray tracing engine 130, and each of the first echo signal simulator 111, the second echo signal simulator 112, and the yth echo signal simulator 113 use a common time base with a precise timing hardware timestamp function such as IEEE 1588, they are synchronized at a sub-microsecond level. For example, IEEE 1588 provides a synchronization accuracy of 20 ns to 100 ns.

[0063] Figure 3 is a simplified flow chart illustrating a method for time-adjusting a simulated echo signal in response to a detection and ranging electromagnetic signal to test a vehicle-mounted detection and ranging electromagnetic signal, according to a representative embodiment.

[0064] refer to Figure 3 In block S311, the driving scenario simulator (120) generates a scenario simulation. The scenario simulation includes one or more simulation objects, which are intended to represent objects that the vehicle may encounter during driving, such as other vehicles, pedestrians, road signs, curbs, utility poles, foreign objects in the vehicle's path, etc.

[0065] In box S312, the driving scene simulator generates a simulator frame, thereby providing object transformation information about the simulated objects. The object transformation information includes the position (3D position), yaw, pitch and roll of each of the simulated objects. The simulator frame part depends on the 3D geometric information used for scene simulation, including a list of objects in the simulated scene, and information related to each of the simulated objects, such as position and whether each simulated object is stationary or moving. The 3D geometric information also includes data indicating the 3D surface of each simulated object. Once the scene simulation starts, the 3D geometric information will no longer change. Each simulation frame has a simulation time in the scene simulation and a corresponding real-world time, which may be different from the simulation time. The simulation time is provided by the time simulator of the driving scene simulator. The real-world time of a simulation frame is the amount of time it takes to generate the simulation frame.

[0066] In block S313, object transformation information about the simulated object is received from the simulation frame at the ray tracing engine (130). The ray tracing engine calculates target information as a function of time based on the object transformation information, the target information including the updated position, velocity, acceleration, and cross-section of the simulated object. The target information is also calculated based on 3D geometry information that is first provided to the ray tracing engine by the driving scene simulator before the scene simulation begins. In block S314, the ray tracing engine generates a ray tracing frame corresponding to the simulation frame. The ray tracing frame includes the calculated target information indicating the calculated position, velocity, acceleration, and cross-section of the simulated object.

[0067] In block S315, the prediction model (115) predicts a time point in the next feature (e.g., the next RF energy burst or the next laser pulse) of the periodic detection and ranging electromagnetic signal received at the return signal simulator (111, 112, 113). The electromagnetic signal can be received from a device under test (DUT) on a vehicle under test, such as a radar sensor or a lidar sensor. Although the predicted time point can be, for example, the start, middle, or end point of the next feature, other points within the next feature or within a frame containing the next feature can also be used without departing from the scope of the present teachings. In one embodiment, the time point can be predicted based on the periodicity of the detection and ranging electromagnetic signal. For example, the timing of a point in the next feature can be predicted using an established pattern of features and / or frames containing features, the pattern including feature lengths and time lapses between adjacent occurrences of features. The pattern can be established by measuring the electromagnetic signal or using sensor specifications of the DUT. In block S316, predicted target information for the simulated target is calculated for the predicted time point using the calculated target information received from the ray tracing frame. For example, the calculated target information is projected to a predicted time point as a function of a real-time clock through forward propagation and / or extrapolation.

[0068] In block S317, the prediction model provides a prediction segment indicating predicted target information for the simulation target before the next feature of the electromagnetic signal is received. The prediction segment compensates for latency introduced by the difference between the real-world time of the simulation frame and the simulation time by generating a ray-traced frame and calculating the predicted target information for the simulation target at the predicted time. Simulation frames, ray-traced frames, and prediction segments are generated using a common timebase to synchronize timing at the sub-microsecond level, thereby achieving synchronization of the various frames and segments and ultimately performing latency compensation. For example, the common timebase can be provided according to the IEEE 1588 standard.

[0069] In block S318, the echo signal simulator (111, 112, 113) generates a simulated echo signal in response to the next pulse train of the received electromagnetic signal using the predicted target information of the simulated target. In block S319, the echo signal simulator transmits the simulated echo signal to the DUT (sensor) that transmits the periodic electromagnetic signal. The simulated echo signal indicates the predicted target information so that the vehicle under test can determine at least the position and velocity and cross-section of the simulated target relative to the DUT. Thus, the vehicle under test can react to the received simulated echo signal, for example, by adjusting steering, braking, throttle settings, etc., in response to the scenario simulation generated by the driving scenario simulator, just as it would occur when the DUT receives the simulated echo signal in real-world time.

[0070] According to various embodiments, because off-the-shelf computer industry standard networking equipment that supports precise time standards (such as IEEE 1588) is used to achieve synchronization instead of expensive and additional custom wiring that carries synchronization signals, all components in the simulation system are synchronized at a sub-microsecond level at low cost. The processing latency of each hardware echo signal simulator can be compensated by a predictive model, thereby improving system performance and ensuring data consistency between all sensors. Based on this approach, system simulation pause and resume functions are easy to implement. By maintaining a mapping between simulator time and real-world time, this approach does not require a real-time driving scenario simulator. This approach is scalable with the number of sensors and has good scalability to support a variety of sensors, for example, including radars, lidars, and cameras. Each processing unit in the simulation test system can be programmed so that it can effectively perform operations in real-world time without the need for traditional parallel programming techniques, which are error-prone and require programmers with expertise.

[0071] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0072] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude the inclusion of other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0073] Aspects of the present invention may be implemented as devices, methods, or computer program products. Thus, aspects of the present invention may take the form of a fully hardware implementation, a fully software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." Additionally, aspects of the present invention may take the form of a computer program product implemented in one or more computer-readable media having computer executable code implemented thereon.

[0074] Although representative embodiments are disclosed herein, those skilled in the art will appreciate that many variations are possible in light of the present teachings and still fall within the scope of the appended claims. Accordingly, the present invention is intended to be limited only by the scope of the appended claims.

Claims

1. A simulation test system for time-adjusting a simulated echo signal in response to a reflected periodic detection and ranging electromagnetic signal, the system comprising: a driving scene simulation server programmed to generate a scene simulation including simulated objects and to generate simulation frames indicating object transformation information about the simulated objects, each simulation frame having a simulation time in the scene simulation and a corresponding real-world time different from the simulation time, wherein the real-world time is an amount of time taken to generate the simulation frame; a ray tracing engine server programmed to receive the object transformation information about the simulation object from the simulation frame, calculate target information including at least a position and a velocity of a simulation target corresponding to the simulation object based on the received object transformation information, and generate a ray tracing frame corresponding to the simulation frame, the ray tracing frame indicating the calculated target information; and An echo signal simulator configured to receive a periodic detection and ranging electromagnetic signal from a sensor and generate a simulated echo signal in response, the periodic detection and ranging electromagnetic signal having a feature in a feature pattern, wherein the echo signal simulator includes a prediction model programmed to predict a time point in the next feature of the electromagnetic signal, calculate predicted target information of the simulated target at the predicted time point using the calculated target information, and generate a prediction segment indicating the predicted target information before receiving the next feature of the electromagnetic signal, the predicted target information being used to generate the simulated echo signal, wherein the driving scenario simulation server, the ray tracing engine server, and the echo signal simulator have a common time base for synchronizing the real-world time at a sub-microsecond level.

2. The system of claim 1 , wherein the prediction model compensates for a delay introduced by each of the real-world time of the simulation frame, the driving scenario simulation server, and the ray tracing engine server when calculating the predicted target information of the simulation target at the predicted time point.

3. The system of claim 1, wherein the common time base is provided according to the IEEE 1588 standard. 4 . The system of claim 1 , wherein the driving scenario simulation server is further programmed to skip generating one or more next simulation frames to enable the simulation time to catch up with the real-world time.

5. The system of claim 1, wherein the simulation time of the simulation frame and each subsequent simulation frame is fixed, and the real-world time of the simulation frame and each subsequent simulation frame is variable.

6. The system of claim 1, wherein the simulated time of the simulated frame and each subsequent simulated frame varies, and the real-world time of the simulated frame and each subsequent simulated frame varies.

7. The system of claim 2, wherein the periodic detection and ranging electromagnetic signal is a radar signal and the features in the characteristic pattern comprise bursts of radio frequency (RF) energy.

8. The system of claim 2, wherein the periodic detection and ranging electromagnetic signal is a lidar signal, and the features in the characteristic pattern comprise laser pulses.

9. The system of claim 2, wherein the prediction model is programmed to use a measurement time of a previous one of the features to predict the time point in the next feature of the electromagnetic signal.

10. A method for time-adjusting a simulated echo signal of a detection and ranging electromagnetic signal reflected from a simulated target, the method comprising: generating a scene simulation including a plurality of simulation objects; generating a plurality of simulation frames indicating object transformation information about the simulation object, each simulation frame having a simulation time in the scene simulation and a corresponding real-world time different from the simulation time, wherein the real-world time is an amount of time taken to generate the simulation frame; receiving the object transformation information from one of the plurality of simulation frames, and calculating a position and a velocity of a simulation target corresponding to the simulation object based on the received object transformation information; generating a ray traced frame corresponding to the simulation frame, the ray traced frame indicating the calculated position and velocity of the simulation target; Predicting a time point of a next feature in a characteristic pattern of a received electromagnetic signal; Calculating predicted target information of the simulation target at a predicted time point using the position and velocity of the simulation target calculated from the ray tracing frame; generating a prediction segment indicating the predicted target information as a function of real-world time before receiving a next burst of the electromagnetic signal, wherein the prediction segment compensates for a latency introduced by a difference between the real-world time and the simulation time of the simulation frame by generating the ray traced frame and by calculating the predicted target information at the prediction time; as well as An echo signal is generated in response to the next burst of the received electromagnetic signal using the predicted target information of the simulated target in the predicted segment.

Images