State estimation of target using sensor measurements

Abstract

In some aspects, a computing device may determine sensor measurements associated with a target via one or more sensors of the computing device, where the sensor measurements include relative radial acceleration. The computing device may determine a measurement model based at least in part on the sensor measurements associated with the target including the relative radial acceleration. The computing device may provide the measurement model to a second order Kalman filter. The computing device may determine a state estimate for the target based at least in part on the second order Kalman filter. The computing device may provide a command based at least in part on the state estimate of the target. Numerous other aspects are described.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to U.S. non-provisional patent application No. 18 / 158,267, filed on January 23, 2023, entitled “STATE ESTIMATION OF ATARGET USING SENSOR MEASUREMENTS,” which is hereby expressly incorporated herein by reference. Technical Field

[0003] Aspects of the present disclosure relate generally to sensors, and, for example, to estimating the state of an object using sensor measurements. Background Art

[0004] Sensor fusion and system state estimation can involve combining sensor data from multiple sensors to produce more reliable information with less uncertainty than using each source individually. Direct fusion can involve fusing sensor data from a set of heterogeneous or homogeneous sensors, while indirect fusion can use information sources such as prior knowledge about the environment and human input. Sensor fusion and system state estimation can be achieved using a Kalman filter (which is a predictor-corrector filtering technique) as well as by using other methods. Summary of the Invention

[0005] In some implementations, an apparatus includes: one or more sensors; a memory; and one or more processors coupled to the memory, the one or more processors configured to: determine sensor measurements associated with a target via the one or more sensors, wherein the sensor measurements include relative radial acceleration a r (k); based at least in part on the relative radial acceleration a associated with the target r (k) determining a measurement model based on these sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second-order Kalman filter; and providing a command based at least in part on the state estimate of the target.

[0006] In some implementations, a method performed by a computing device includes determining, via one or more sensors of the computing device, sensor measurements associated with a target, wherein the sensor measurements include a relative radial acceleration a. r (k) e; based at least in part on the relative radial acceleration a associated with the target r(k) determining a measurement model based on these sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second-order Kalman filter; and providing a command based at least in part on the state estimate of the target.

[0007] In some implementations, a non-transitory computer-readable medium storing an instruction set includes one or more instructions that, when executed by one or more processors of a computing device, causes the computing device to: determine sensor measurements associated with a target via one or more sensors of the computing device, wherein the sensor measurements include a relative radial acceleration a r (k); based at least in part on the relative radial acceleration a associated with the target r (k) determining a measurement model based on these sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second-order Kalman filter; and providing a command based at least in part on the state estimate of the target.

[0008] In some implementations, an apparatus includes means for determining sensor measurements associated with a target via one or more sensors of the apparatus, wherein the sensor measurements include relative radial acceleration a r (k); for determining a relative radial acceleration a based at least in part on a velocity associated with the target; r (k) components for determining a measurement model based on these sensor measurements; components for providing the measurement model to a second-order Kalman filter; components for determining a state estimate of the target based at least in part on the second-order Kalman filter; and components for providing commands based at least in part on the state estimate of the target.

[0009] Aspects generally include methods, apparatus, systems, computer program products, non-transitory computer-readable media, user devices, user equipment, and / or processing systems substantially as described with reference to and as illustrated in the accompanying drawings and description.

[0010] The features and technical advantages of the examples according to the present disclosure have been outlined quite broadly above so that the detailed description below may be better understood. Additional features and advantages will be described below. The concepts and specific examples disclosed may be readily used as a basis for modifying or designing other structures for achieving the same purpose of the present disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein (both their organization and method of operation) and the associated advantages will be better understood from the following description when considered in conjunction with the accompanying drawings. Each of the figures in the drawings is provided for the purpose of illustration and description and not as a definition of limitations to the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order that the above-described features of the present disclosure may be fully understood, a more particular description of the invention briefly summarized above may be obtained by reference to various aspects (some of which are illustrated in the accompanying drawings). It should be noted, however, that the drawings illustrate only certain typical aspects of the present disclosure and are not therefore to be considered limiting of its scope, as the description may admit to other equally effective aspects. The same reference numerals in different drawings may identify the same or similar elements.

[0012] Figure 1 is an illustration of an example environment in which the systems and / or methods described herein may be implemented.

[0013] Figure 2 is a diagram illustrating example components of a device according to the present disclosure.

[0014] Figure 3 is a diagram illustrating an example of radial measurement according to the present disclosure.

[0015] Figure 4 is a diagram illustrating an example associated with state estimation of a target using sensor measurements according to the present disclosure.

[0016] Figure 5 is a diagram illustrating an example associated with a radial acceleration vector according to the present disclosure.

[0017] Figure 6 is a diagram illustrating an example associated with a radial velocity vector according to the present disclosure.

[0018] Figure 7 is a diagram illustrating an example associated with yaw angle and yaw rate estimation according to the present disclosure.

[0019] Figure 8 is a diagram illustrating an example associated with calculation of a measurement covariance matrix according to the present disclosure.

[0020] Figure 9 is a flow chart of an example process associated with state estimation of a target using sensor measurements according to the present disclosure. DETAILED DESCRIPTION

[0021] Various aspects of the present disclosure are described more fully below with reference to the accompanying drawings. However, the present disclosure can be embodied in many different forms and should not be interpreted as being limited to any specific structure or function presented throughout the present disclosure. Rather, these aspects are provided so that the present disclosure will be thorough and complete, and the scope of the present disclosure will be fully conveyed to those skilled in the art. It will be appreciated by those skilled in the art that the scope of the present disclosure is intended to cover any aspect of the present disclosure disclosed herein, whether it is realized independently or in combination with any other aspect of the present disclosure. For example, any number of aspects set forth herein may be used to realize a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods that are practiced using other structures, functionality, or structure and functionality in addition to or different from the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the present invention.

[0022] Sensors, such as radar sensors or light detection and ranging (LIDAR) sensors, can sense in a radial manner. The sensor can determine radial range, radial velocity, and / or radial acceleration. However, a Cartesian reference can be used for trajectory, tracking, and positioning. The sensor can measure the azimuth angle θ, but the sensor may not be able to directly measure the angular rate on a frame-by-frame basis. In other words, the sensor can measure radial positioning and motion, while the environmental perception model can use a Cartesian reference. The state variables of the environmental perception model can be defined in a Cartesian coordinate system, while the measurement vectors can be radial. When the target is at a significant range (e.g., greater than 30 meters) or when the directions of travel are not very different (e.g., nearly parallel, which will result in a relatively low angular rate), the radial to Cartesian error within a 50ms cycle time may be relatively small. When very close (e.g., less than 30 meters) or when the directions of travel are significantly different (e.g., cross paths or during sudden acceleration), the radial to Cartesian error may become significant. Existing solutions may rely on motion models and Kalman filtering techniques. However, such existing solutions may rely on motion models that introduce errors due to nonlinearities and assumptions made for simplicity.

[0023] In some embodiments, a computing device may determine sensor measurements associated with a target via one or more sensors associated with the computing device. The computing device may be associated with a first vehicle. The target may be associated with a second vehicle. The sensor measurements may include relative radial acceleration. The relative radial acceleration may be based at least in part on a Doppler chirp rate at a time instance. The computing device may determine a measurement model (or measurement vector) based at least in part on the sensor measurements associated with the target including the relative radial acceleration. The computing device may provide the measurement model to a second-order Kalman filter. The computing device may determine a state estimate of the target based at least in part on the second-order Kalman filter. The state estimate of the target may be a filtered estimate of a state vector. The computing device may provide a command based at least in part on the state estimate of the target. For example, the command may be associated with accelerating the first vehicle, braking the first vehicle, and / or turning the first vehicle.

[0024] In some aspects, a second-order Kalman filter can be used with an unconventional measurement vector, where the measurement vector is compatible with radar and coherent homodyne LIDAR, or prior fusion of measurements. The measurement vector can utilize Doppler modulation rate to improve the accuracy of acceleration state estimates. Furthermore, by using the measurement vector, a computing device can be able to obtain a more accurate environmental perception model, which can be used to more accurately determine the state estimate of the target.

[0025] Figure 1 is a diagram of an example environment 100 in which the systems and / or methods described herein may be implemented. Figure 1 As shown, environment 100 may include sensor 110, computing device 120, first vehicle 130, second vehicle 140, and network 150. Second vehicle 140 may be associated with a target. The devices of environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0026] The sensor 110 may include a radar sensor and / or a LIDAR sensor. The sensor 110 may be configured to capture various measurements, such as radial range, azimuth, relative radial velocity, and / or relative radial acceleration. The sensor 110 may provide the measurements to the computing device 120. The computing device 120 may determine an estimate of the state of the target based at least in part on the measurements. The computing device 120 may employ a Kalman filter (or an extended Kalman filter) that may be used to determine an estimate of the state of the target based at least in part on the measurements. The sensor 110 and the computing device 120 may be associated with a first vehicle 130. For example, the sensor 110 and the computing device 120 may be onboard the first vehicle 130. The second vehicle 140 may move relative to the first vehicle 130. For example, the second vehicle 140 may move at a specific velocity and / or a specific acceleration.

[0027] Figure 1 The number and arrangement of devices and networks shown are provided as examples. In practice, there may be Figure 1 The devices and / or networks shown may include additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently than those shown. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, a set of devices (eg, one or more devices) of environment 100 may perform one or more functions described as being performed by another set of devices of environment 100.

[0028] Figure 2 is a diagram illustrating example components of a device 200 according to the present disclosure. Device 200 may correspond to computing device 120. In some aspects, computing device 120 may include one or more devices 200 and / or one or more components of device 200. Figure 2 As shown, device 200 may include bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, and / or sensor 240 (or multiple sensors). Sensor 240 may be a radar sensor or a LIDAR sensor.

[0029] The bus 205 includes components that allow communication between the various components of the device 200. The processor 210 is implemented in hardware, firmware, or a combination of hardware and software. The processor 210 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some aspects, the processor 210 includes one or more processors that can be programmed to perform functions. The memory 215 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 210.

[0030] The storage component 220 stores information and / or software related to the operation and use of the device 200. For example, the storage component 220 may include a solid-state memory device, a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of non-transitory computer-readable medium, along with a corresponding drive.

[0031] Input components 225 include components that permit device 200 to receive information (such as via user input) (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). Additionally or alternatively, input components 225 may include components for determining the location or position of device 200 (e.g., a global positioning system (GPS) component or a global navigation satellite system (GNSS) component), and / or sensors for sensing information (e.g., an accelerometer, a gyroscope, an actuator, or another type of positioning or environmental sensor). Output components 230 include components that provide output information from device 200 (e.g., a display, a speaker, a tactile feedback component, and / or an audio or visual indicator).

[0032] The communication interface 235 includes a transceiver-like component (e.g., a transceiver and / or a separate receiver and transmitter) that enables the device 200 to communicate with other devices (such as via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection). The communication interface 235 can permit the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 235 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency interface, a universal serial bus (USB) interface, a wireless local area interface (e.g., a Wi-Fi interface), and / or a cellular network interface.

[0033] Device 200 can perform one or more processes described herein. Device 200 can perform these processes based on processor 210 executing software instructions stored by non-transitory computer-readable media (such as memory 215 and / or storage component 220). Computer-readable media is defined herein as non-transitory memory devices. Memory devices include storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0034] The software instructions may be read from another computer-readable medium or another device into the memory 215 and / or storage component 220 via the communication interface 235. The software instructions stored in the memory 215 and / or storage component 220, when executed, may cause the processor 210 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0035] In some aspects, the device 200 includes components for performing one or more processes described herein and / or components for performing one or more operations of the processes described herein. For example, the device 200 may include components for determining sensor measurements associated with the target via one or more sensors of the computing device, wherein the sensor measurements include relative radial acceleration. for determining a relative radial acceleration based at least in part on a velocity associated with the target including the relative radial acceleration means for determining a measurement model based on the sensor measurements; means for providing the measurement model to a second order Kalman filter; means for determining a state estimate of the target based at least in part on the second order Kalman filter, and / or means for providing a command based at least in part on the state estimate of the target. In some aspects, such means may include a combination of Figure 2 One or more components of device 200 are depicted, such as bus 205 , processor 210 , memory 215 , storage component 220 , input component 225 , output component 230 , communication interface 235 , and / or processor 240 .

[0036] Figure 2 The number and arrangement of components shown are provided as examples. In practice, the device 200 may include Figure 2 200. In some embodiments, the present invention relates to a device 200 that is configured to have additional components, fewer components, different components, or differently arranged components than those shown in FIG. Additionally or alternatively, one set of components (e.g., one or more components) of the device 200 may perform one or more functions described as being performed by another set of components of the device 200.

[0037] Sensors (such as radar sensors or LIDAR sensors) can sense in a radial manner. The sensor can determine radial range, radial velocity, and / or radial acceleration. However, a Cartesian reference can be used for trajectory, tracking, and positioning. The sensor can measure the azimuth angle θ, but the sensor may not be able to directly measure the angular rate on a frame-by-frame basis. The radial to Cartesian error over a 50 ms period may be relatively small when the target is at significant range (e.g., greater than 30 meters) or when the directions of travel are not very different (e.g., nearly parallel, which would result in a relatively low angular velocity). The radial to Cartesian error may become significant when very close (e.g., less than 30 meters) or when the directions of travel are significantly different (e.g., crossing paths, during sudden accelerations, or during sudden changes of direction). The directions of travel may be significantly different due to maneuvering through roundabouts, due to intersections, or due to urban clutter. The relative velocity measurement made by Doppler frequency measurement may be radial. The Doppler rate of change (also called the Doppler modulation rate) (df doppler / dt) can be a possible direct measurement that provides radial acceleration and improves motion estimation.

[0038] Figure 3 is a diagram illustrating an example 300 of radial measurement according to the present disclosure.

[0039] like Figure 3 As shown, a first vehicle may move relative to a second vehicle. The second vehicle may move at a specific speed. The first vehicle may include sensors, such as radar sensors and / or LIDAR sensors, to capture radial measurements. The first vehicle may use sensors to measure an azimuth angle θ. The Doppler shift may provide radial velocity. can be where c is the speed of light, f0 is the sensor operating frequency, and f doppler is the measured Doppler frequency. The Doppler velocity provides the radial acceleration. can be In addition, for f0=77GHz radar, And for λ0 = 1550nm coherent LIDAR, where λ0 is the optical wavelength.

[0040] As indicated above, Figure 3 are provided as examples. Other examples can be found in the Figure 3 The examples described are different.

[0041] The target state equation discrete model of constant acceleration can be defined. The target dynamics can be represented by the state vector s = [xv x a x yvy a y ] T Definition, where x indicates the distance in the x direction of the target, v x Indicates the speed in the x direction of the target, a x Indicates the acceleration in the x direction of the target, y indicates the distance in the y direction of the target, v y indicates the velocity in the y direction of the target, and a y Indicates the acceleration in the y direction of the target. The target model may assume that the acceleration is constant during the time interval Δt. The derivative of the acceleration (often also called jerk) may be the process noise represented by a random vector w, where is zero-mean Gaussian noise, and the covariance matrix A discrete-time controlled process (uniform Δt time interval) can be represented by a linear equation.

[0042] A sensor (such as a radar sensor or a coherent LIDAR sensor) can measure the radial range r k , azimuth angle θ k and relative radial velocity Where k is a time instance. The sensor can measure radial range via time-of-flight measurements (at time k*Δt). The sensor can measure azimuth and elevation via angle-of-arrival algorithms (at time k*Δt). The sensor can measure relative radial velocity via Doppler measurements (at time k*Δt).

[0043] Extended Kalman filter implementations may rely on linearization of trigonometric functions and may lack angular rate, which may introduce residual errors and may lead to violations of Kalman error statistics. Therefore, most extended Kalman filter implementations may be self-organizing, and the relationship between measurement inaccuracy and the covariance matrix of the extended Kalman filter may be broken.

[0044] State-of-the-art measurement models (or measurement equations) may be associated with a relatively high degree of nonlinearity. For example, the following may be associated with nonlinearity:

[0045] or

[0046] Such state-of-the-art measurement models can be associated with Taylor linearization of residual terms with propagated errors, poor assumptions about Gaussian zero-mean error statistics, and / or a lost relationship between sensor measurements and the covariance matrix. Such state-of-the-art measurement models can produce residual errors that can reduce the accuracy of the estimates.

[0047] In various aspects of the techniques and apparatus described herein, a computing device may determine sensor measurements associated with a target via one or more sensors associated with the computing device. The computing device may be associated with a vehicle. The target may be associated with another vehicle. The sensor measurements may include relative radial acceleration. The relative radial acceleration may be based at least in part on a Doppler modulation rate at a time instance. The computing device may determine a measurement model (e.g., a measurement vector) based at least in part on the sensor measurements associated with the target, including the relative radial acceleration. The computing device may provide the measurement model to a second-order Kalman filter. The computing device may determine a state estimate of the target based at least in part on the second-order Kalman filter. The state estimate of the target may be a filtered estimate of the state vector. The computing device may provide a command based at least in part on the state estimate of the target. For example, the command may be associated with accelerating the vehicle, braking the vehicle, and / or turning the vehicle.

[0048] Figure 4 is a diagram illustrating an example 400 associated with state estimation of a target using sensor measurements according to the present disclosure.

[0049] As shown at 402, a computing device may determine sensor measurements associated with a target via one or more sensors associated with the computing device. The one or more sensors may include a radar sensor and / or a LIDAR sensor. The computing device may be associated with a first vehicle (e.g., first vehicle 130). The target may be associated with a second vehicle (e.g., second vehicle 140). The sensor measurements may include a relative radial acceleration a. r (k), the relative radial acceleration may be based at least in part on the Doppler modulation rate at the time instance. The sensor measurements may also include a radial range r based at least in part on the time of flight measurement at the time instance. k , an azimuth angle θ based at least in part on the digital beamforming at the time instance k (and possibly the elevation angle), and a relative radial velocity v based at least in part on Doppler measurements at the time instance r (k) Sensor measurements may be directly and independently measured by one or more sensors.

[0050] As shown at reference numeral 404, the computing device may calculate the relative radial acceleration a based at least in part on the data associated with the target. r (k) The sensor measurements are used to determine the measurement model. The measurement model may include a plurality of measurement vectors, and the plurality of measurement vectors may include: and in and The measurement vector may be an unconventional measurement vector based on the instantaneous Doppler modulation rate (which may improve the accuracy of the acceleration state estimation).

[0051] As shown in the reference numeral 406, the computing device may provide the measurement model to a second order Kalman filter. The second order Kalman filter may be implemented using an unconventional measurement vector using an instantaneous Doppler modulation frequency. In other words, the computing device may use a vector having a relative radial acceleration a. r (k) Second-order Kalman filter of the measurements.

[0052] As shown in reference numeral 408, the computing device may determine a state estimate of the target based at least in part on a second-order Kalman filter. The second-order Kalman filter may provide a filtered estimate of the state vector. The state estimate of the target may be an estimate of the acceleration state of the target. The state estimate of the target may be calculated by s=[xv x a x yv y a y ] T Represents, where x indicates the distance in the x direction of the target, v x Indicates the relative velocity in the x direction of the target, a x Indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y direction of the target. The state estimate of the target as determined at least in part based on a second order Kalman filter may be based at least in part on: x associated with the corresponding modified sensor measurements k 、y k 、 and and Where k indicates a time instance. The computing device may determine the state estimate of the target by excluding linearization of trigonometric functions and avoiding nonlinearities associated with linearization of trigonometric functions.

[0053] As shown at 410, the computing device may provide a command based at least in part on the target's state estimate. The command may be associated with accelerating, braking, and / or turning the first vehicle. For example, based on the acceleration state estimate associated with the second vehicle, the computing device may provide a command to maneuver the first vehicle so that the first vehicle avoids a collision with the second vehicle.

[0054] As indicated above, Figure 4 are provided as examples. Other examples can be found in the Figure 4 The examples described are different.

[0055] The target state equation discrete model of constant acceleration can be defined. The target dynamics can be represented by the state vector s = [xv x a x yv y a y ] T Definition, where x indicates the distance in the x direction of the target, v x Indicates the speed in the x direction of the target, a x Indicates the acceleration in the x direction of the target, y indicates the distance in the y direction of the target, v y indicates the velocity in the y direction of the target, and a y Indicates the acceleration in the y direction of the target. The target model may assume that the acceleration is constant during the time interval Δt.

[0056] A sensor (such as a radar sensor or a coherent LIDAR sensor) can directly and independently measure the radial range r k , azimuth angle θ k , relative radial velocity v r (k) and relative radial acceleration a r (k). The sensor may measure radial range by time-of-flight measurement (at time k*Δt). The sensor may measure azimuth by digital beamforming (at time k*Δt). The sensor may measure relative radial velocity by Doppler measurement (at time k*Δt). The sensor may measure relative radial acceleration by Doppler modulation rate (at time k*Δt). Radial acceleration may not always be available and may be considered optional. The Doppler modulation rate may be measured using a coherent optical 1550 nanometer (nm) LIDAR sensor or a 77 GHz radar sensor. However, when the acquisition time increases beyond 100 ms, the Doppler modulation rate may also be measured using a 77 GHz radar.

[0057] Figure 5 is a diagram illustrating an example associated with a radial acceleration vector according to the present disclosure.

[0058] like Figure 5 As shown, the radial velocity vector (Cartesian and polar coordinates) can be defined as follows: The measured radial velocity corresponds to The change from polar coordinates to Cartesian coordinates gives v r =cos(θ).v x +sin(θ).v y , and multiply both sides by r to get rv r=(r.cos(θ)).v x +(r.sin(θ)).v y , which becomes (by defining x and y) rv r =xv x +yv y .

[0059] As indicated above, Figure 5 are provided as examples. Other examples can be found in the Figure 5 The examples described are different.

[0060] Figure 6 is a diagram illustrating an example 600 associated with a radial velocity vector according to the present disclosure.

[0061] like Figure 6 As shown, the radial acceleration vector can be defined as follows: The measured radial acceleration corresponds to The change from polar coordinates to Cartesian coordinates gives a r =cos(θ).a x +sin(θ).a y , and multiply both sides by r to get ra r =(r.cos(θ)).a x +(r.sin(θ)).a y , which becomes ra r =xa x +ya y .

[0062] As indicated above, Figure 6 are provided as examples. Other examples can be found in the Figure 6 The examples described are different.

[0063] In some aspects, the radial velocity vector (which may be represented by rv r =xv x +yv y ) and the radial acceleration vector (which can be represented by ra r =xa x +ya y The measurement model (z k ) may depend on the choice of the modified measurement vector according to:

[0064]

[0065] where a r Measurements are available.

[0066] In some aspects, the measurement model (z k ) can be used a r And the relationship between the measurement and state vector is bilinear with respect to the independent variable. k ) can be used with a second-order Kalman filter to provide a closed and relatively simple form, and can preserve the Gaussian properties required by the second-order Kalman filter.

[0067] In some aspects, it is possible to define a r Second-order measurement equations for acceleration measurement. For example, the measurement model (z k ) includes the acceleration term a r The nonlinear function of can be defined as follows:

[0068]

[0069] Where s=[xv x a x yv y a y ] T The nonlinearity may be a second order polynomial, which may mean that a second order Taylor expansion provides the exact form. A second order Kalman filter may rely on a Taylor expansion according to: where e 1 =[1 0 0 0] T , e 2 =[01 0 0] T , e 3 =[0 0 1 0] T And e 4 =[0 0 0 1] T , and where H is the Jacobian matrix (4×6) and H j is the j of h(s) th 4 Hessian matrices (6×6) of components.

[0070] also, And H 1 =0;H 2 =0; The Hessian matrix is ​​constant. The measurement equation can be exact and the Taylor-formed residuals can be null.

[0071] In some aspects, a second-order Kalman filter may utilize a r The Kalman filter formulation can be based on Expanded to the second order, where Cov(w k )=Q k And Cov(ε k )=Rk The process noise w and the measurement noise E can be assumed to be white noise with zero mean. The second-order Kalman filter can be obtained by The state s is estimated from the measurement z by minimizing the covariance of the estimation error. The time update can be regular because the radar target state model can be linear, which can be based on and The measurement update using the second-order measurement model can be obtained by Indicates that And the Kalman gain K k is based on of.

[0072] also, And the error covariance update can be based on Furthermore, the next measurement update can be performed to return to the time update loop.

[0073] Figure 7 is a diagram illustrating an example 700 of yaw angle and yaw rate estimation according to the present disclosure.

[0074] As shown at 702, the yaw angle can be derived according to the following

[0075]

[0076] As shown at 704, the yaw rate Can be partially eliminated by substitution To export, this gives Elimination by permutation Can produce Make

[0077] As indicated above, Figure 7 are provided as examples. Other examples can be found in the Figure 7 The examples described are different.

[0078] In some aspects, for a second-order Kalman filter, a measurement vector debias and a measurement covariance matrix may be defined. With respect to debiasing the measurement vector, the sensor measurement noise ("error") may be independent, Gaussian, and zero-mean. The variance (σ) of the measurement may be reported by the sensor and estimated based at least in part on the signal-to-noise ratio (SNR) and system parameters. The sensor measurement noise may be defined for range, relative velocity, azimuth, and relative acceleration, respectively, according to the following formulas:

[0079]

[0080] Where B is the bandwidth, λ is the wavelength, θ -3dB is the -3dB beamwidth, and is the SNR. Variance of relative acceleration The measurement covariance matrix may be positively correlated with the sensor measurement variance.

[0081] In some respects, a second-order Kalman filter can be expected to have zero mean error. of Noisy measurements caused by zero-mean independent noise When the measurement vector of the conversion in the second-order Kalman filter can be m )=z, it is unbiased.

[0082] In some aspects, with respect to debiasing the measurement vector, the measurement noise can be zero-mean. However, the transformed measurements x = r.cos(θ) and y = r.sin(θ) are no longer zero-mean because E(cos(θ)) ≠ 0. Consider Among them (ε r ; ε θ ) is zero-mean independent noise and (r m θ m ) represents the noisy measurement, and (r;θ) represents the true value. After simplification:

[0083] (biased). The modified and unbiased measurement vector can be defined as:

[0084] In reality:

[0085] (Thus, unbiased). In addition,

[0086] After simplification, (Hence, unbiased).

[0087] In some aspects, the measurement vector z m permuted by a modified unbiased vector Can produce The expected value of the measurement can be equal to the unbiased true value. Therefore, the unbiased modified measurement vector It can be based on the following:

[0088]

[0089] The updated system can be defined as follows:

[0090] Figure 8is a diagram illustrating an example 800 of calculation of a measurement covariance matrix according to the present disclosure.

[0091] like Figure 8 As shown in , the measurement covariance matrix (R) can be clearly defined and calculated. Figure 8 The measurement covariance matrix R shown may be based at least in part on:

[0092]

[0093] In some respects, when considering the noise of the sensor When is independent and zero-mean, each component of the measurement covariance matrix can be expanded with respect to the sensor measurement accuracy. For example, E(ε r )=0; E(ε θ )=0; Each error variance may depend on the measured SNR and is reported by the sensor.

[0094] As indicated above, Figure 8 are provided as examples. Other examples can be found in the Figure 8 The examples described are different.

[0095] In some aspects, the classical formulation of the measurement vector may suffer from several drawbacks. The classical formulation may be associated with Taylor linearization of residual terms with propagated errors. The classical formulation may be associated with poor assumptions about Gaussian zero-mean error statistics. The classical formulation may be associated with a missing relationship between the sensor measurements and the covariance matrix.

[0096] In some aspects, a new formulation of the measurement vector can be associated with several advantages. The new formulation can utilize direct sensor measurements (e.g., direct sensor measurements and nothing else). The new formulation can provide a direct relationship between the sensor error variance (dependent on the SNR) and the covariance matrix of the second-order Kalman filter. The new formulation can be associated with a second-order linearization that is explicit in an exact form with zero residuals, which can result in more accurate estimates. The new formulation can be associated with measurement errors that are Gaussian with zero mean (as required by Kalman optimization). The new formulation can be well suited for radial measurements made by sensors such as radar sensors or LIDAR sensors. The measurement model can be scaled to combined radar and LIDAR measurements. The new formulation can utilize direct measurement of the target relative acceleration using the Doppler modulation rate (which may be particularly relevant for coherent LIDAR). In other words, the target relative acceleration can be directly measured based, at least in part, on the Doppler modulation rate. The new formulation can eliminate the need for cascaded filters, which can reduce latency and enable simpler calibration.

[0097] Figure 9is a flow chart of an example process 900 associated with estimating the state of a target using sensor measurements. In some implementations, Figure 9 One or more process blocks of are performed by a computing device (e.g., computing device 120). In some implementations, Figure 9 One or more process blocks of are performed by another device or a group of devices separate from or including the computing device, such as a sensor (e.g., sensor 110) and / or a vehicle (e.g., vehicle 130). Additionally or alternatively, Figure 9 One or more process blocks of may be performed by one or more components of device 200, such as processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, and / or sensor 240.

[0098] like Figure 9 As shown, process 900 may include determining sensor measurements associated with a target via one or more sensors of a computing device, wherein the sensor measurements include a relative radial acceleration a r (k) (Block 910). For example, the computing device may determine sensor measurements associated with the target via one or more sensors of the computing device, where the sensor measurements include relative radial acceleration a r (k), as described above.

[0099] like Figure 9 As further shown, process 900 may include determining a target based at least in part on a relative radial acceleration a associated with the target. r (k) to determine a measurement model (block 920). For example, the computing device may determine a measurement model based at least in part on sensor measurements associated with the target including a relative radial acceleration a. r (k) is used to determine the measurement model, as described above.

[0100] like Figure 9 As further shown, process 900 may include providing the measurement model to a second-order Kalman filter (block 930).For example, the computing device may provide the measurement model to the second-order Kalman filter, as described above.

[0101] like Figure 9 As further shown, process 900 may include determining a state estimate of the target based at least in part on a second-order Kalman filter (block 940). For example, the computing device may determine the state estimate of the target based at least in part on a second-order Kalman filter, as described above.

[0102] like Figure 9As further shown, process 900 may include providing a command based at least in part on the target's state estimate (block 950).For example, the computing device may provide a command based at least in part on the target's state estimate, as described above.

[0103] Process 900 may include additional implementations, such as any single implementation or any combination of implementations described below and / or in conjunction with one or more other processes described elsewhere herein.

[0104] In a first embodiment, the sensor measurement further comprises: determining a radial range r based at least in part on a time-of-flight measurement at a time instance. k , an azimuth angle θ based at least in part on the digital beamforming at the time instance k , and a relative radial velocity v based at least in part on Doppler measurements at the time instance r (k).

[0105] In a second embodiment, alone or in combination with the first embodiment, the measurement model includes a plurality of modified measurement vectors, and the plurality of modified measurement vectors includes: and where σ is the symbol of variance.

[0106] In a third embodiment, alone or in combination with one or more of the first and second embodiments, the state estimate of the target is given by s=[xv x a x yv y a y ] T Indicates that x indicates the relative distance in the x direction of the target, v x Indicates the relative velocity in the x direction of the target, a x Indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y direction of the target.

[0107] In a fourth embodiment, alone or in combination with one or more of the first to third embodiments, the state estimate of the target, as determined at least in part based on a second-order Kalman filter, is based at least in part on: x associated with the modified sensor measurement k 、y k 、 and where k indicates a time instance.

[0108] In a fifth implementation, alone or in combination with one or more of the first through fourth implementations, the sensor measurements are directly and independently measured by one or more sensors of the computing device.

[0109] In the sixth embodiment, alone or in combination with one or more of the first to fifth embodiments, the relative radial acceleration a r (k) The variance associated with is based at least in part on the SNR and system parameters, and the measurement covariance matrix is ​​based at least in part on the relative radial acceleration a r (k) Associated variance.

[0110] In a seventh implementation, alone or in combination with one or more of the first to sixth implementations, determining a state estimate of a target excludes linearization of a trigonometric function and avoids nonlinearities associated with linearization of a trigonometric function.

[0111] In an eighth implementation, alone or in combination with one or more of the first to seventh implementations, the one or more sensors include one or more radar sensors or LIDAR sensors.

[0112] In a ninth implementation, alone or in combination with one or more of the first through eighth implementations, the computing device is associated with a vehicle, and the target is associated with another vehicle.

[0113] although Figure 9 An example block diagram of process 900 is shown, but in some implementations, process 900 includes Figure 9 The depicted blocks may include additional blocks, fewer blocks, different blocks, or blocks arranged in a different manner. Additionally or alternatively, two or more of the blocks of process 900 may be performed in parallel.

[0114] A summary of some aspects of the disclosure is provided below.

[0115] Aspect 1: An apparatus comprising: one or more sensors; a memory; and one or more processors coupled to the memory, the one or more processors being configured to: determine sensor measurements associated with a target via one or more sensors of a computing device, wherein the sensor measurements include a relative radial acceleration a r (k); based at least in part on the relative radial acceleration a associated with the target r(k) determines a measurement model based on the sensor measurements; provides the measurement model to a second-order Kalman filter; determines a state estimate of the target based at least in part on the second-order Kalman filter; and provides a command based at least in part on the state estimate of the target.

[0116] Aspect 2: The apparatus of aspect 1, wherein the sensor measurement further comprises: a radial range r based at least in part on a time-of-flight measurement at a time instance k , an azimuth angle θ based at least in part on the digital beamforming at the time instance k , and a relative radial velocity v based at least in part on the Doppler measurement at the time instance r (k).

[0117] Aspect 3: The apparatus according to aspect 2, wherein the measurement model comprises a plurality of modified measurement vectors, and wherein the plurality of modified measurement vectors comprises: and where σ is the symbol of variance.

[0118] Aspect 4: The apparatus according to any one of aspects 1 to 3, wherein the state estimate of the target is given by s=[xv x a x yv y a y ] T where x indicates the relative distance in the x direction of the target, and v x Indicates the relative velocity of the target in the x direction, a x indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, and v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y-direction of the target.

[0119] Aspect 5: The apparatus of aspect 4, wherein the state estimate of the target as determined at least in part based on the second order Kalman filter is based at least in part on: x associated with modified sensor measurements k 、y k 、 and where k indicates the time instance.

[0120] Aspect 6: The apparatus of any one of aspects 1 to 5, wherein the sensor measurements are directly and independently measured by the one or more sensors of the device.

[0121] Aspect 7: The device according to any one of aspects 1 to 6, wherein the relative radial acceleration a r (k) The variance associated with the relative radial acceleration is based at least in part on a signal-to-noise ratio (SNR) and system parameters, and wherein the measurement covariance matrix is ​​based at least in part on the relative radial acceleration a r (k) the associated variance.

[0122] Aspect 8: An apparatus according to any one of aspects 1 to 7, wherein the one or more processors are configured to determine the state estimate of the target by excluding linearization of trigonometric functions and avoiding nonlinearities associated with the linearization of trigonometric functions.

[0123] Aspect 9: The apparatus of any one of aspects 1 to 8, wherein the one or more sensors include one or more of: a radar sensor or a light detection and ranging (LIDAR) sensor.

[0124] Aspect 10: The apparatus of any one of aspects 1 to 9, wherein the apparatus is associated with a vehicle, and wherein the target is associated with another vehicle.

[0125] Aspect 11: A method configured to perform one or more operations recited in one or more of aspects 1 to 10.

[0126] Aspect 12: A system configured to perform one or more operations recited in one or more of aspects 1 to 10.

[0127] Aspect 13: An apparatus comprising means for performing one or more operations recited in one or more of Aspects 1 to 10.

[0128] Aspect 14: A non-transitory computer-readable medium storing an instruction set, the instruction set comprising one or more instructions that, when executed by a device, cause the device to perform one or more operations recited in one or more of aspects 1 to 10.

[0129] Aspect 15: A computer program product comprising instructions or codes for performing one or more operations recited in one or more of Aspects 1 to 10.

[0130] While the foregoing disclosure provides illustration and description, it is not intended to be exhaustive or to limit the aspects to the precise forms disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of these aspects.

[0131] As used herein, the term "component" is intended to be broadly interpreted as a combination of hardware and / or hardware and software. Whether referred to as software, firmware, middleware, microcode, hardware description language or other names, "software" should be broadly interpreted as meaning instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures and / or functions, etc. As used herein, a "processor" is implemented in a combination of hardware and / or hardware and software. It will be apparent that the systems and / or methods described herein can be implemented by a combination of different forms of hardware and / or hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit various aspects. Therefore, no reference is made herein to specific software code to describe the operation and behavior of the systems and / or methods, as those skilled in the art will appreciate that software and hardware can be designed to implement the systems and / or methods based at least in part on the description herein.

[0132] As used herein, "satisfying a threshold" may mean that a value is greater than a threshold, greater than or equal to a threshold, less than a threshold, less than or equal to a threshold, equal to a threshold, not equal to a threshold, etc., depending on the context.

[0133] Although specific combinations of features are set forth in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of various aspects. Many of these features may be combined in a manner not specifically stated in the claims and / or not disclosed in the specification. The disclosure of various aspects includes each dependent claim combined with each other claim in the claim set. As used herein, a phrase referring to "at least one of" a list of items refers to any combination of these items (which includes a single member). As an example, "at least one of a, b, or c" is intended to encompass a, b, c, a+b, a+c, b+c, and a+b+c, as well as any combination of multiple identical elements (e.g., a+a, a+a+a, a+a+b, a+a+c, a+b+b, a+c+c, b+b, b+b+b, b+b+c, c+c, and c+c+c, or any other arrangement of a, b, and c).

[0134] Any element, action or instruction used herein should not be interpreted as key or necessary, unless explicitly described as such. In addition, as used herein, the articles "one" and "a kind of" are intended to include one or more projects, and can be used interchangeably with "one or more". In addition, as used herein, the article "said" is intended to include one or more projects connected with the article "said", and can be used interchangeably with "one or more". In addition, as used herein, the terms "set" and "group" are intended to include one or more projects, and can be used interchangeably with "one or more". If only want to refer to a project, then use the phrase "only one" or similar terms. In addition, as used herein, the terms "have", "have" etc. are intended to be open terms, which do not limit the elements they modify (for example, "an element with" A can also have B). In addition, the phrase "based on" is intended to represent "at least partially based on", unless explicitly stated otherwise. Furthermore, as used herein, the term "or" when used in a series is intended to be open-ended and used interchangeably with "and / or" unless explicitly stated otherwise (e.g., if used in conjunction with "either" or "only one of").

Claims

1. A device, comprising: one or more sensors; Memory; and one or more processors coupled to the memory and configured to: Determining sensor measurements associated with the target via the one or more sensors, wherein the sensor measurements include a relative radial acceleration a r (k); Based at least in part on information associated with the target including the relative radial acceleration a r (k) determining a measurement model based on the sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second-order Kalman filter; as well as Commands are provided based at least in part on the state estimate of the target.

2. The apparatus of claim 1 , wherein the sensor measurements further comprise: A radial range r based at least in part on a time-of-flight measurement at a time instance k , Based at least in part on the azimuth angle θ of the digital beamforming at the time instance k ,and A relative radial velocity v based at least in part on the Doppler measurement at the time instance r (k).

3. The apparatus of claim 2 , wherein the measurement model comprises a plurality of modified measurement vectors, and wherein the plurality of modified measurement vectors comprises: and where σ is the symbol of variance.

4. The apparatus according to claim 1, wherein the state estimate of the target is given by s=[xv x a x yv y a y ] T where x indicates the relative distance in the x direction of the target, and v x Indicates the relative velocity of the target in the x direction, a x indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, and v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y-direction of the target.

5. The apparatus of claim 4 , wherein the state estimate of the target as determined based at least in part on the second order Kalman filter is based at least in part on: x associated with modified sensor measurements. k 、y k 、 and where k indicates the time instance. The apparatus of claim 1 , wherein the sensor measurements are directly and independently measured by the one or more sensors.

7. The device according to claim 1, wherein the relative radial acceleration a r (k) The variance associated with the relative radial acceleration is based at least in part on a signal-to-noise ratio (SNR) and system parameters, and wherein the measurement covariance matrix is ​​based at least in part on the relative radial acceleration a r (k) the associated variance.

8. The apparatus of claim 1, wherein the one or more processors are configured to determine the state estimate of the target by excluding linearization of trigonometric functions and avoiding nonlinearities associated with the linearization of trigonometric functions.

9. The apparatus of claim 1, wherein the one or more sensors comprise one or more of: a radar sensor or a light detection and ranging (LIDAR) sensor.

10. The device of claim 1, wherein the device is associated with a vehicle, and wherein the target is associated with another vehicle.

11. A method performed by a computing device, the method comprising: Determining sensor measurements associated with a target via one or more sensors of the computing device, wherein the sensor measurements include a relative radial acceleration a r (k); Based at least in part on information associated with the target including the relative radial acceleration a r (k) determining a measurement model based on the sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second order Kalman filter; as well as Commands are provided based at least in part on the state estimate of the target.

12. The method of claim 11, wherein the sensor measurements further comprise: A radial range r based at least in part on a time-of-flight measurement at a time instance k , Based at least in part on the azimuth angle θ of the digital beamforming at the time instance k ,and A relative radial velocity v based at least in part on the Doppler measurement at the time instance r (k).

13. The method of claim 12, wherein the measurement model comprises a plurality of modified measurement vectors, and wherein the plurality of modified measurement vectors comprises: and where σ is the symbol of variance.

14. The method according to claim 11, wherein the state estimate of the target is given by s=[xv x a x yv y a y ] T where x indicates the distance in the x direction of the target, and v x Indicates the velocity of the target in the x direction, a x indicates the acceleration in the x direction of the target, y indicates the distance in the y direction of the target, and v y indicates the velocity in the y direction of the target, and a y Indicates the acceleration in the y-direction of the target.

15. The method of claim 14, wherein the state estimate of the target as determined based at least in part on the second order Kalman filter is based at least in part on: x associated with modified sensor measurements k 、y k 、 and where k indicates the time instance.

16. The method of claim 11, wherein the sensor measurements are directly and independently measured by the one or more sensors of the computing device.

17. The method according to claim 11, wherein the relative radial acceleration a r (k) The variance associated with the relative radial acceleration is based at least in part on a signal-to-noise ratio (SNR) and system parameters, and wherein the measurement covariance matrix is ​​based at least in part on the relative radial acceleration a r (k) the associated variance.

18. The method of claim 11, wherein determining the state estimate of the target excludes linearization of trigonometric functions and avoids high-order nonlinearities associated with the linearization of trigonometric functions.

19. The method of claim 11, wherein the one or more sensors include one or more of: a radar sensor or a light detection and ranging (LIDAR) sensor.

20. The method of claim 11, wherein the computing device is associated with a vehicle, and wherein the destination is associated with another vehicle.

21. A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: One or more instructions that, when executed by one or more processors of a computing device, cause the computing device to: Determining sensor measurements associated with a target via one or more sensors of the computing device, wherein the sensor measurements include a relative radial acceleration a r (k); Based at least in part on information associated with the target including the relative radial acceleration a r (k) determining a measurement model based on the sensor measurements; providing the measurement model to a second-order Kalman filter; determining a state estimate of the target based at least in part on the second-order Kalman filter; as well as Commands are provided based at least in part on the state estimate of the target.

22. The non-transitory computer readable medium of claim 21 , wherein the sensor measurements further comprise: A radial range r based at least in part on a time-of-flight measurement at a time instance k , Based at least in part on the azimuth angle θ of the digital beamforming at the time instance k ,and A relative radial velocity v based at least in part on the Doppler measurement at the time instance r (k).

23. The non-transitory computer-readable medium of claim 22, wherein the measurement model comprises a plurality of modified measurement vectors, and wherein the plurality of modified measurement vectors comprises: and where σ is the symbol of variance.

24. The non-transitory computer readable medium of claim 21, wherein the state estimate of the target is given by s=[xv x a x yv y a y ] T where x indicates the relative distance in the x direction of the target, and v x Indicates the relative velocity of the target in the x direction, a x indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, and v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y-direction of the target.

25. The non-transitory computer readable medium of claim 24, wherein the state estimate of the target as determined based at least in part on the second order Kalman filter is based at least in part on: x associated with modified sensor measurements k 、y k 、 and where k indicates the time instance.

26. A device comprising: Means for determining sensor measurements associated with a target, wherein the sensor measurements include relative radial acceleration a r (k); for determining a relative radial acceleration a based at least in part on a r (k) determining a component of a measurement model by measuring said sensor; means for providing said measurement model to a second order Kalman filter; means for determining a state estimate of the target based at least in part on the second order Kalman filter; and Means for providing commands based at least in part on the state estimate of the target.

27. The apparatus of claim 26, wherein the sensor measurements further comprise: A radial range r based at least in part on a time-of-flight measurement at a time instance k , Based at least in part on the azimuth angle θ of the digital beamforming at the time instance k ,and A relative radial velocity v based at least in part on the Doppler measurement at the time instance r (k).

28. The apparatus of claim 27, wherein the measurement model comprises a plurality of modified measurement vectors, and wherein the plurality of modified measurement vectors comprises: and where σ is the symbol of variance.

29. The apparatus according to claim 26, wherein the state estimate of the target is given by s=[xv x a x yv y a y ] T where x indicates the relative distance in the x direction of the target, and v x Indicates the relative velocity of the target in the x direction, a x indicates the relative acceleration in the x direction of the target, y indicates the relative distance in the y direction of the target, and v y indicates the relative velocity in the y direction of the target, and a y Indicates the relative acceleration in the y-direction of the target.

30. The apparatus of claim 29, wherein the state estimate of the target as determined based at least in part on the second order Kalman filter is based at least in part on: x associated with modified sensor measurements k 、y k 、 and where k indicates the time instance.

Images