System and method for time synchronization between vehicle and central server

Abstract

The present disclosure provides a system and method for time synchronization between a vehicle and a central server. A method for synchronizing time between a central server and an automated vehicle (AV) includes exchanging a series of time synchronization (TS) messages; and generating a set of time offsets between a pair of TS messages in the series of TS messages. The method further includes altering a system time of the AV using a time drift estimate defined by a set of time evaluation models that are a function of at least one of a plurality of time inputs or a set of layer time offset values; and controlling the AV using the system time and one or more marshalling instructions, the one or more marshalling instructions providing a driving command from the central server.

Description

[0001] Cross-references

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 635,037, filed April 17, 2024, the disclosure of which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates to a system and / or method for synchronizing time for shunting automated vehicles. Background Art

[0004] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0005] An automated vehicle (AV) is configured to drive autonomously without the need for a human operator. Typically, a destination is provided to the AV, and using a defined algorithm, the AV autonomously drives to the destination using a travel route.

[0006] In some applications, the AVs may also be controlled by an external system that transmits driving commands to the AVs to shunt the AVs through an area. In a non-limiting example, the Society of Automotive Engineers (SAE) has published a series of guidelines under SAE J3292 for supporting automated shunts of vehicles in various applications, such as, but not limited to, manufacturing plants, warehouses, parking lots, and / or garages. Summary of the Invention

[0007] In one aspect, the present disclosure relates to a method for synchronizing time between a central server and an automated vehicle (AV). The method includes obtaining a plurality of time inputs that can be used to synchronize a clock of the AV; and exchanging a series of time synchronization (TS) messages, wherein each TS message is defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer. The method also includes generating a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets being defined for at least one of the plurality of layers for the pair of TS messages. The method also includes modifying a system time of the AV using a time drift estimate defined by a set of time estimation models, the set of time estimation models being a function of at least one of the plurality of time inputs and the set of time offsets; and controlling the AV using the system time and one or more train instructions that provide driving commands from the central server.

[0008] In another aspect, the present disclosure relates to a vehicle system for an autonomous vehicle (AV) that is marshaled by a central server. The vehicle system includes one or more computing devices configured to: obtain a plurality of time inputs available to synchronize the AV with the central server; exchange a series of time synchronization (TS) messages with the central server, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; alter a system time of the AV using a time drift estimate defined by a set of time estimation models, the set of time estimation models a function of at least one of the plurality of time inputs and the set of time offsets; and control the AV to execute a driving command using the system time in response to receiving a marshaling message from the central server having the driving command.

[0009] In yet another aspect, the present disclosure relates to a system for autonomously controlling an autonomous vehicle (AV). The system includes an infrastructure server associated with a facility and including one or more computing devices. The one or more computing devices are configured to: obtain a plurality of time inputs available to synchronize the AV with the infrastructure server; exchange a series of time synchronization (TS) messages with the AV, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; transmit a message including a command to alter a system of the AV to an updated system time, the updated system time determined using a time drift estimate calculated using a set of time estimation models, the set of time estimation models a function of at least one of the plurality of time inputs and the set of time offsets; and transmit one or more marshaling instructions having one or more driving commands to the AV to control movement of the AV based on the updated system time. BRIEF DESCRIPTION OF DRAWINGS

[0010] So that the disclosure can be well understood, various forms thereof will now be described, by way of example, with reference to the drawings in which:

[0011] Figure 1 A plurality of automated vehicles and an automated vehicle marshaling central server (AVM CS) for marshaling the automated vehicles according to the present disclosure are shown;

[0012] Figure 2 is a block diagram of an automated vehicle and AVM CS according to the present disclosure;

[0013] Figure 3 is a flowchart of a time synchronization routine performed by a clock synchronization module; and

[0014] Figure 4 is a flowchart of a time synchronization message exchange between an AV and an AVM CS.

[0015] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. DETAILED DESCRIPTION

[0016] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. The drawings are not necessarily to scale; some features can be exaggerated or minimized for purposes of illustration. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.

[0017] When marshaling AVs through an area, the system clock of the AVs should be synchronized with the system clock of the automated vehicle marshaling central server (AVM CS). For example, the time reference of the AVs and the AVM CS can conform to Coordinated Universal Time (UTC), and for data elements of messages transmitted between the AVs and the AVM CS, the reference clock is accurate within 1-msec accuracy of the UTC reference, with a timestamp reference used in messages in the end-to-end architecture of the automated vehicle marshaling (AVM) system. In addition to the timestamp, the exchanged messages include a timestamp confidence that provides details about the time confidence when the timestamp was generated.

[0018] In a non-limiting example, an AVM system can be employed to move AVs through production line end-of-line testing using an overhead vision system coupled with the AVM CS for route selection and motion control. Providing route selection and motion control information can require robust time synchronization between the AVM CS and the AVs for the AVs to effectively move through space without interfering with each other and / or objects, and / or delaying cycle time.

[0019] A method for controlling AVs includes a time-based trajectory control method. If the AVs or the AVM CS do not know the time at which they are operating; this can result in interference between the AVs or with the environment.

[0020] In one form, the present disclosure relates to a system and / or method for monitoring, estimating, and adjusting timestamps and time confidence of AVs and / or AVM CSs using a multi-layer approach that analyzes time-based input data from external sources and from exchanged messages. This provides consistent time synchronization for AVs performing low-speed and high-speed automated features especially when time synchronization signals from external sources (e.g., global navigation satellite systems (GNSS)) can be degraded.

[0021] Reference is made to Figure 1 and Figure 2 In non-limiting examples, autonomous vehicles 100A, 100B, 100C, 100D (collectively, “autonomous vehicles 100”) are provided in a facility 102 in which the vehicles 100 are undergoing various system processes, such as calibration, software configuration, and / or testing. In one form, the autonomous vehicles 100 are driven to various stations 104A, 104B, 104C, 104D, and 104E (collectively, “stations 104”) under the control of an automated vehicle marshaling central server (AVM CS) 106. That is, the autonomous vehicles 100, which are automated vehicles (AVs), drive in the facility based on driving commands from the AVM CS 106, which tracks the position and orientation of the vehicles 100 using a vision system 110 (which can be part of the AVM CS 106) Figure 2 ) that is part of the AVM CS 106.

[0022] Hereinafter, the autonomous vehicles 100 are referred to as automated vehicles (AVs) 100. While the AVs 100 are shown as four-wheeled passenger vehicles, the present disclosure can be applicable to other types of AVs 100, such as but not limited to automated guided vehicles, vehicles having one or more wheels, vehicles having tracks that move by wheels, and other vehicles that can be controlled autonomously.

[0023] In one form, the vision system 110 includes a plurality of vision sensors 112 (e.g., cameras) that capture images of various areas of the facility 102, where the images are then processed by a vision module 113 to recognize, for example, the AVs 100 as well as other features in the facility 102.

[0024] In addition to the vision system 110, the AVM CS 106 includes a communication system 120 and an AVM controller 122.

[0025] In one form, the communication system 120 is configured to support wireless and / or wired communication with, for example, the vision sensor 112, the AV 100, and other devices and / or controllers in the facility 102. In one form, the communication system 120 is configured to establish wireless communication using any suitable wireless communication protocol. In a non-limiting example, the communication system 120 includes a WiFi module 124 configured to communicate using a WiFi protocol, a Bluetooth® (BT) module 126 configured to communicate using a BT protocol, a GNSS module 128 configured to obtain and process GNSS signals from one or more satellites 127, and a cellular module 129 configured to communicate using one or more cellular protocols and a cellular tower network 131.

[0026] The AVM controller 122 is configured to control the marshalling of one or more AVs 100 in the facility 102. As part of the marshalling control, the AVM controller 122 is configured to include a central server (CS) internal clock 130, which can also be provided as a system clock for the AVM CS 106, and a CS clock synchronization module 132 having a connected marshalling system timer (CMST) model 136. As detailed herein, the CS clock synchronization (sync) module 132 is configured to monitor and adjust the time of the CS internal clock (Clk) 130 to provide accurate time synchronization with, for example, the AV 100 to enable accurate marshalling of the AV 100.

[0027] In one form, the AV 100 is configured to include a communication system 150 and a vehicle system 150 having a drive controller 154 for controlling the driving operations of the AV 100.

[0028] The communication system 150 is configured to support wired and wireless communication with external devices or systems using various suitable technologies and / or wireless protocols. In a non-limiting example and as with the communication system 120 of the AVM CS 106, the communication system 150 includes a WiFi module 160, a BT module 162, a GNSS module 164, and a cellular module 166. The communication system 150 also includes a vehicle network interface 168 (e.g., a gateway module) configured to connect to one or more vehicle communication networks (e.g., a controlled area network (CAN) and / or a local interconnect network (LIN)) to communicate with other systems and / or controllers in the AV 100, such as but not limited to the vehicle system 152.

[0029] ​In one form, the drive controller 154 is configured to autonomously drive the AV 100 by controlling a drive system (not shown) of the AV 100. The drive controller 154 controls the AV 100 to a desired destination using, for example, a drive control algorithm stored and executed by the drive controller 154 and / or using drive commands from the AVM CS 106. In some embodiments, the drive controller 154 includes AV modules 169 that support vehicle-to- world (V2X) messages 200 and AVM features 202. In non-limiting examples, the V2X messages 200 include a vehicle-to-infrastructure (V2I) message generator 203, a vehicle marshaling message (VMM) generator 204, and a basic safety message (BSM) generator 206. The AVM features include different marshaling programs for the AV 100, such as, but not limited to: factory marshaling 210, warehouse marshaling 212, valet parking 214, low-speed autonomy 216, and high-speed autonomy 218.

[0030] As with the AVM CS 106, the vehicle system 152 includes an internal clock (AV internal Clk) 170 that is a system clock of the AV 100 and an AV clock synchronization (sync) module 172. The AV clock sync module 172 includes a CMST model 174 that operates in a similar manner as the CMST model 134 to provide accurate time synchronization with, for example, the AVM CS 106 to enable accurate marshaling.

[0031] In one form, the AVM controller 122 of the AVM CS 106 is configured to communicate with the AV 100 using infrastructure marshaling messages (IMMs) that are wirelessly transmitted using the communication system 120. The vehicle system 152 of the AV 100 is configured to communicate with the AVM CS 106 using vehicle marshaling messages (VMMs) that are wirelessly transmitted using the AV communication system 150.

[0032] Referring to Figure 3 An example time synchronization routine 300 performed by the CS clock sync module 132 and the AV clock sync module 172 (collectively, the “clock sync modules 132, 172”) is provided.

[0033] At operation 302, the CK sync modules 132, 172 obtain inputs for time synchronization. In non-limiting examples, Table 1 provides a list of inputs that can be received and the type of information provided from the inputs. Among the inputs, GNSS, internal computer clock, NTP / PTP, and / or marshaling messages (VMM and / or IMM) can be referred to as time inputs that can be used to synchronize the clock of the AV 100. Figure 3Example inputs including IMM&VMM 302A, system time 302B, NTP / PTP 302C, and GNSS 302D are shown, but the inputs should not be limited to these examples and can include any combination or all of the inputs in Table 1.

[0034] Table 1: Time Synchronization Inputs

[0035]

[0036]

[0037] As part of the inputs, the CLK synchronization module 132, 172 is configured to perform a defined time synchronization message exchange. Referring to Figure 4 , the AV 100 transmits a time synchronization request as a VMM to the AVM CS 106 at time Tl (e.g., TSVMMReqTrnsmiT = Tl). Like all marshaled messages, the request includes an application layer, a transmission time, a security layer, and a radio link layer, with each layer including a timestamp.

[0038] The AVM CS 106 receives the request at time T2 (e.g., TSVMMReqReceiveTime = T2). The AVM CS 106 processes the request received at time T2 and generates a time synchronization response to the AV 100 and transmits the response at time T3 (e.g., TSVMMRspnTrnmtTime = T3). The AV 100 receives the response at time T4 (e.g., TSVMMRspnReceiveTime = T4).

[0039] While Figure 4 Time synchronization messages originating from the AV 100 are shown, but similar exchanges can originate from the AVM CS 106.

[0040] Using the timestamp information from the exchanged messages, the CLK synchronization module 132, 172 can determine time-related information about the transmission and reception of the messages. In a non-limiting example, an over-the-air (OTA) time offset provides the time it takes for a message to arrive at a recipient after being transmitted and is provided as a difference between the reception time and the transmission time (e.g., T2-T1 or T4-T3).

[0041] In another example, the CK synchronization module 132, 172 further determines the time taken by the AV 100 to receive the response after the transmission of the request, referred to as the round trip time and determined as the difference between the time of receipt of the response and the time of transmission of the request (e.g., T4-T1). These and other time-based analyses are performed by the CMST model 134, 174 to monitor time drift, detect drift in time stamps, and adjust the time of the system clock, if applicable.

[0042] With continued reference to Figure 3 , the CMST model 134, 174 can perform the steps of operation 304 to adjust the time of the AV 100. At operation 304A, the CMST model 134, 174 determines whether an input is received to estimate the time stamp accuracy.

[0043] At operation 304B, the CMST model 134, 174 uses the VMM and / or IMM to calculate offsets related to the time stamps provided at the application layer, processing, and round trip time. In a non-limiting example, the offsets include: an over-the-air (OTA) request application layer time offset (OTAReqOffsetAppLayer) provided in Equation 1; an OTA response application layer time offset (OTARspnOffsetAppLayer) provided in Equation 2; an AVM CS 106 processing time (CSProcTime) provided in Equation 3; and a round trip time (RndTrpTimeOnVehAppLayer) of the AV 100 for a time synchronization message request (RndTrpTimeOnVehAppLayer) provided in Equation 4. In the equations, “conf” is a level of confidence of the provided time information, which can be a different value for each data (e.g., the confidence value for TSVMMReqReceiveTime can be different from the confidence value for STSVMReqTrnsmtTime). The offsets of Equations 1, 2, 3, and 4 can generally be referred to as OTA request (req.) and response (resp) offsets, and characterize the application layer.

[0044] Equation 1... OTAReqOffsetAppLayer = TSVMMReqReceiveTime + conf

[0045] - TSVMMReqTrnsmtTime + conf

[0046] Equation 2... OTARspnOffsetAppLayer = TSVMMRspnReceiveTime + conf

[0047] - TSVMMRspnTrnmtTime + conf

[0048] Equation 3.... CSProcTime = TSVMMRspnTrnmtTime + conf

[0049] - TSVMMReqReceiveTime + conf

[0050] Equation 4.... RndTrpTimeOnVehAppLayer = TSVMMRspnReceiveTime +

[0051] conf - TSVMMReqTrnsmtTime + conf

[0052] The CMST model 134, 174 further calculates the offset of one or more layers of the VMM and IMM. In one form, each VMM and IMM includes an application layer, a transport layer, a security layer, and a radio link layer. Each layer includes a timestamp that can be used to determine the layer offset. In a non-limiting example, the CMST model 134, 174 uses Equations 5, 6, and 7 to calculate an AVM transport offset (AVMOffsetTrnprtLay), an AVM security timestamp (AVMOffsetSecLay), and an AVM radio link offset (AVMOffsetRadLay), respectively. While the layer offsets are provided as the difference between the IMM timestamp and the VMM timestamp for the respective layer, other techniques such as, but not limited to, phase-locked loop modeling can be used to determine the layer offset. The offset of one or more layers can provide insight into potential latency variations that occur at the hardware level at the AV 100.

[0053] Equation 5.... AVMOffsetTrnprtLay = (IMMTransLayTS - VMMTransLayTS

[0054] Equation 6.... AVMOffsetSecLay = IMMSecLayTS - VMMSecLayTS

[0055] Equation 7.... AVMOffsetRadLay = IMMRadLayTS - VMMRadLayTS

[0056] The CMST model 134, 174 next determines the accuracy of the current automated vehicle time using a time accuracy validator (TAV) model that considers time data from different sources, including time from the system clock of the AV (or AVM CS). In a non-limiting example, the inputs can include: GNSS time, system time, NTP / PTP time, IMM time stamp (e.g., application layer time stamp), and VMM time stamp (e.g., application layer time stamp). Thus, the TAV model can be represented by the following function 1. Function 1... avm_time_accuracy_validator_model(gnss_time, system_time,

[0057] ntp / ptp_time, imm_and_vmm_timestamp)

[0058] The CMST model 134, 174 further calculates the confidence of the current time that the AV needs to operate relative to the time that is being operated using a time confidence validator (TCV) model and various confidence values from other sources. In a non-limiting example, the TCV model is a function of GNSS time confidence, system time confidence, NTP / PTP time confidence, IMM time stamp confidence, and VMM time stamp confidence. Thus, the TCV model can be represented by the following function 2.

[0059] Function 2... avm_time_conf_validator_model(gnss_time_confidence,

[0060] system_time_confidence, ntp / ptp_time_confidence,

[0061] imm_and_vmm_timestamp_confidence)

[0062] The CMST model 134, 174 further calculates the drift of the total time by monitoring different time sources to improve control-trajectory-time based operations. In a non-limiting example, the CMST model 134, 174 employs a time drift monitor (TDM) model that monitors the following time sources: GNSS time, system time, NTP / PTP time, IMM time synchronization, and VMM time synchronization. In the TDM, the IMM time synchronization and VMM time synchronization are provided by Equations 1-7. Thus, the TDM model can be represented by the following function 3.

[0063] Function 3... avm_time_drift_monitor_model(gnss_time, system_time,

[0064] ntp / ptp_time, imm_and_vmm_message_timesync)

[0065] The CMST model 134, 174 further calculates the total timestamp that will be adjusted using a time drift adjuster (TDA) model, as provided in Equation 8. The TDA model uses the outputs of the TAV model, the TCV model, and the TDM model to determine a time drift estimate (e.g., “avm_time_drift_monitor_output”) and correct the current timestamp (e.g., “current_avm_vehicle_time”) and adjust the confidence of the AV’s system time, where the adjusted system time is provided as “avm_time_drift_adjuster” in Equation 8.

[0066] Equation 8... avm_time_drift_adjuster = current_avm_vehicle_time + / -

[0067] avm_time_drift_monitor_output

[0068] In some implementations, the CMST model 134, 174 determines whether to obtain a drift at operation 304C. For example, if the drift determined at 304B is zero, no adjustment is needed. However, if the drift is greater than zero or less than zero, the AV’s system time is changed / adjusted at operation 306. If the CMST model 134 of the AVM CS 106 determines the drift, the AVM CS 106 can transmit a command to the AV 100 to cause the AV 100 to adjust its system time. If the CMST 174 of the AV 100 determines the drift, the AV clock synchronization module 172 directly adjusts the AV internal clock 170 (e.g., system clock).

[0069] Once the timestamp is adjusted, at 306, the various modules of the AVM CS 106 and the AV 100 use the time of the system clock to group the AV 100. In a non-limiting example, the AV 100 uses the time as a timestamp for messages transmitted to external systems (e.g., V2X messages) and / or messages transmitted to AVM features for use in grouping the AV 100 in the facility 102.

[0070] In non-limiting examples, the TAV model, the TCV model, the TDM model, and / or the TDA model are defined using dynamic neural network training, a rolling average of the time and time confidence of the computation, and / or a recursive weighted least squares method.

[0071] At one time, the binding is directly to the hardware being used. Another time offset is more related to the latency of the communication protocol being used.

[0072] Unless specifically stated otherwise as apparent from the foregoing disclosure, all numerical values are understood to be modified in all instances by the term "about" or "approximately" to permit for variations that can occur in equivalents and processing formats to accomplish substantially similar results. Such variations are to be understood to be within the scope of the disclosure.

[0073] In this application, the terms "controller" and / or "module" (e.g., CS clock synchronization module 132 with CMST model 134, vision module 113, modules 124, 126, 128, 129 of communication system 120, driving controller 154 with AV module 169, AV clock synchronization module 172 with CMST model 174, and / or modules 160, 162, 164, 166 of communication system 150) can refer to, be part of, or include an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.

[0074] The term memory is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory propagating signals or electromagnetic waves propagating through a medium (such as on a carrier); thus, the term computer-readable medium can be considered tangible and non-transitory. Non-limiting examples of non-transitory, tangible computer-readable media are nonvolatile memory circuits (such as flash memory circuits, erasable programmable read-only memory (EPROM) circuits, or mask read-only circuits), volatile memory circuits (such as static random access memory (SRAM) circuits or dynamic random access memory (DRAM) circuits), magnetic storage media (such as analog magnetic tapes or digital magnetic tapes or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0075] The apparatus and methods described in this application can be partially or entirely implemented by special purpose computers configured to create a general purpose computer that performs one or more specific functions embodied in the computer program. The functional blocks, flowchart components, and other elements described above (e.g., CS clock synchronization module 132 (e.g., with CMST model 134), vision module 113, modules 124, 126, 128, 129 of communication system 120, driving controller 154 with AV module 169, AV clock synchronization module 172 with CMST model 174, and / or modules 160, 162, 164, 166 of communication system 150) serve as software specifications, which can be converted by routine work of a skilled artisan or programmer into a computer program.

[0076] As used herein, the phrase at least one of A, B, and C should be interpreted as using the non-exclusive logical OR to represent logic (A or B or C), and should not be interpreted as representing "at least one of A, at least one of B, and at least one of C."

[0077] The description of the present disclosure is merely exemplary in nature and, thus, variations that do not depart from the essence of the present disclosure are intended to be within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

[0078] According to the present disclosure, a method for synchronizing time between a central server and an automated vehicle (AV) includes obtaining a plurality of time inputs usable to synchronize a clock of the AV; exchanging a series of time synchronization (TS) messages, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generating a set of time offsets between a pair of TS messages of the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; altering a system time of the AV using a time drift estimate defined by a set of time estimate models, the set of time estimate models a function of at least one of the plurality of time inputs and the set of time offsets; and controlling the AV using the system time and one or more marshaling instructions, the one or more marshaling instructions providing driving commands from the central server.

[0079] According to one embodiment, each of the plurality of layers includes a timestamp confidence of a timestamp associated with the layer.

[0080] According to one embodiment, the plurality of time inputs are obtained using at least one of an internal computer clock of the AV, an internal computer clock of the central server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshaled message transmitted by at least one of the AV or the central server.

[0081] According to one embodiment, the above invention features further comprise using information from the series of TS messages to generate at least one of an over-the-air (OTA) request time offset, an OTA response time offset, a central server processing time, or an AV message round trip time.

[0082] According to one embodiment, the set of time evaluation models comprises at least one of: a time accuracy verifier model configured to determine an accuracy of a current time used by at least one of the AV or the central server; a time confidence verifier model configured to determine a confidence of a time of the AV to be taken relative to a time being used using one or more confidence inputs received from the plurality of time inputs; or a time drift monitor model configured to track a time drift of a total time.

[0083] According to one embodiment, the set of time evaluation models comprises a time drift adjuster configured to determine the time drift estimate using an output of the at least one of the time accuracy verifier model, the time confidence verifier model, or the time drift monitor model.

[0084] According to one embodiment, the above invention features further comprise: transmitting, by the AV, a time synchronization request to the central server, the series of TS messages comprising the time synchronization request; and calculating, by the AV, a round trip time offset using a timestamp of the time synchronization request and a timestamp associated with a response responsive to receiving the response to the time synchronization from the central server, the set of time offsets comprising the round trip time offset.

[0085] According to the present invention, there is provided a vehicle system for an autonomous vehicle (AV) that is marshaled by a central server, the vehicle system having: one or more computing devices configured to: obtain a plurality of time inputs available to synchronize the AV with the central server; exchange a series of time synchronization (TS) messages with the central server, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; alter a system time of the AV using a time drift estimate defined by a set of time evaluation models, the set of time evaluation models a function of at least one of the plurality of time inputs and the set of time offsets; and control the AV to execute a driving command using the system time in response to receiving a marshaling message from the central server having the driving command.

[0086] According to one embodiment, the plurality of time inputs is obtained using at least one of an internal computer clock of the AV, an internal computer clock of the central server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshaling message transmitted by at least one of the AV or the central server.

[0087] According to one embodiment, the one or more computing devices are further configured to generate at least one of an over-the-air (OTA) request time offset, an OTA response time offset, a central server processing time, or an AV message round trip time using information from the series of TS messages.

[0088] According to one embodiment, the set of time evaluation models includes at least one of: a time accuracy verifier model configured to determine an accuracy of a current time used by at least one of the AV or the central server; a time confidence verifier model configured to determine a confidence of a time to be taken by the AV relative to a time being used using one or more confidence inputs received from the plurality of time inputs; or a time drift monitor model configured to track a time drift of a total time.

[0089] According to one embodiment, the set of time evaluation models includes a time drift adjuster configured to determine the time drift estimate using an output of the at least one of the time accuracy verifier model, the time confidence verifier model, or the time drift monitor model.

[0090] According to one embodiment, the one or more computing devices are further configured to: transmit a time synchronization request to the central server, the series of TS messages including the time synchronization request; and calculate a round trip time offset using a timestamp of the time synchronization request and a timestamp associated with a response in response to receiving the response to the time synchronization from the central server, the set of time offsets including the round trip time offset.

[0091] According to the present invention, there is provided a system for autonomously controlling an autonomous vehicle (AV), the system having an infrastructure server associated with a facility and including one or more computing devices configured to: obtain a plurality of time inputs available for synchronizing the AV with the infrastructure server; exchange a series of time synchronization (TS) messages with the AV, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; transmit a message including a command to change a system of the AV to an updated system time, the updated system time determined using a time drift estimate calculated using a set of time evaluation models, the set of time evaluation models a function of at least one of the plurality of time inputs and the set of time offsets; and transmit one or more marshalling instructions having one or more driving commands to the AV to control movement of the AV based on the updated system time.

[0092] According to one embodiment, the plurality of time inputs are obtained using at least one of an internal computer clock of the AV, an internal computer clock of the infrastructure server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshalling message transmitted by at least one of the AV or the infrastructure server.

[0093] According to one embodiment, the one or more computing devices are further configured to generate at least one of an over-the-air (OTA) request time offset, an OTA response time offset, a central server processing time, or an AV message round trip time using information from the series of TS messages.

[0094] According to one embodiment, the set of temporal evaluation models includes at least one of: a temporal accuracy verifier model configured to determine an accuracy of a current time used by at least one of the AV or the infrastructure server; a temporal confidence verifier model configured to determine a confidence of a time to be adopted by the AV relative to a time being used using one or more confidence inputs received from the plurality of temporal inputs; or a temporal drift monitor model configured to track a temporal drift of a total time.

[0095] According to one embodiment, the set of temporal evaluation models includes a temporal drift adjuster configured to determine the temporal drift estimate using an output of the at least one of the temporal accuracy verifier model, the temporal confidence verifier model, or the temporal drift monitor model.

Claims

1. A method for synchronizing time between a central server and an automated vehicle (AV), comprising: obtaining a plurality of time inputs available for synchronizing a clock of the AV; exchanging a series of time synchronization (TS) messages, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer; generating a set of time offsets between a pair of TS messages of the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages; altering a system time of the AV using a time drift estimate defined by a set of time evaluation models, the set of time evaluation models a function of at least one of the plurality of time inputs and the set of time offsets; and controlling the AV using the system time and one or more marshaling instructions, the one or more marshaling instructions providing driving commands from the central server.

2. The method of claim 1, wherein each of the plurality of layers includes a timestamp confidence of a timestamp associated with the layer.

3. The method of claim 1, wherein the plurality of time inputs are obtained using at least one of an internal computer clock of the AV, an internal computer clock of the central server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshaling message transmitted by at least one of the AV or the central server.

4. The method of claim 1, further comprising generating at least one of an over-the-air (OTA) request time offset, an OTA response time offset, a central server processing time, or an AV message round trip time using information from the series of TS messages.

5. The method of claim 1, wherein the set of time evaluation models includes at least one of: a time accuracy verifier model configured to determine an accuracy of a current time used by at least one of the AV or the central server; a time confidence verifier model configured to determine a confidence of a time to be adopted by the AV relative to a time being used using one or more confidence inputs received from the plurality of time inputs; or a time drift monitor model configured to track a time drift of a total time.

6. The method of claim 5, wherein the set of time evaluation models includes a time drift adjuster configured to determine the time drift estimate using an output of the at least one of the time accuracy verifier model, the time confidence verifier model, or the time drift monitor model.

7. The method of claim 1, further comprising: transmitting, by the AV to the central server, a time synchronization request, the series of TS messages including the time synchronization request; and a round trip time offset calculated by the AV using a timestamp of the time synchronization request and a timestamp associated with a response from the central server in response to receiving the time synchronization request, the set of time offsets including the round trip time offset.

8. A vehicle system for an autonomous vehicle (AV) that is marshaled by a central server, the vehicle system comprising: one or more computing devices configured to: obtain a plurality of time inputs available to synchronize the AV with the central server, exchange a series of time synchronization (TS) messages with the central server, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer, generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages, alter a system time of the AV using a time drift estimate defined by a set of time evaluation models, the set of time evaluation models a function of at least one of the plurality of time inputs and the set of time offsets, and control the AV to execute a driving command using the system time in response to receiving a marshaled message from the central server with the driving command.

9. The system of claim 8, wherein the plurality of time inputs are obtained using at least one of an internal computer clock of the AV, an internal computer clock of the central server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshaled message transmitted by at least one of the AV or the central server.

10. The system of claim 8, wherein the one or more computing devices are further configured to generate at least one of an over-the-air (OTA) request time offset, an OTA response time offset, a central server processing time, or an AV message round trip time using information from the series of TS messages.

11. The system of claim 8, wherein the set of time evaluation models includes at least one of: a time accuracy verifier model configured to determine an accuracy of a current time used by at least one of the AV or the central server; a time confidence verifier model configured to determine a confidence of a time to be taken by the AV relative to a time being used using one or more confidence inputs received from the plurality of time inputs; or a time drift monitor model configured to track a time drift of a total time.

12. The system of claim 11, wherein the set of time evaluation models includes a time drift adjuster configured to determine the time drift estimate using an output of the at least one of the time accuracy verifier model, the time confidence verifier model, or the time drift monitor model.

13. The system of claim 8, wherein the one or more computing devices are further configured to: transmit a time synchronization request to the central server, the series of TS messages including the time synchronization request; and calculate a round trip time offset using a timestamp of the time synchronization request and a timestamp associated with a response in response to receiving the response to the time synchronization from the central server, the set of time offsets including the round trip time offset.

14. A system for autonomously controlling an autonomous vehicle (AV), comprising: an infrastructure server associated with a facility and including one or more computing devices configured to: obtain a plurality of time inputs available for synchronizing the AV with the infrastructure server, exchange a series of time synchronization (TS) messages with the AV, each TS message defined by a plurality of layers, the plurality of layers including at least two of an application layer, a transport layer, a security layer, or a radio link layer, generate a set of time offsets between a pair of TS messages in the series of TS messages, the set of time offsets defined for at least one of the plurality of layers for the pair of TS messages, transmit a message including a command to change a system of the AV to an updated system time, the updated system time determined using a time drift estimate calculated using a set of time evaluation models, the set of time evaluation models a function of at least one of the plurality of time inputs and the set of time offsets, and transmit one or more marshaling instructions having one or more driving commands to the AV to control movement of the AV based on the updated system time.

15. The system of claim 14, wherein the plurality of time inputs are obtained using at least one of an internal computer clock of the AV, an internal computer clock of the infrastructure server, a network time protocol, a precision time protocol, a global navigation satellite system, one or more timestamps of a marshaling message transmitted by at least one of the AV or the infrastructure server.

Images