System and method for cloud-based scalable positioning system

Abstract

A system and method includes identifying a plurality of vehicles as a first vehicle cluster, the first vehicle cluster having a front end and a rear end, each of the front end of the first vehicle cluster and the rear end of the first vehicle cluster performing a respective latency masking model, respective sensor data collected from a sensor system of the vehicle is received from each vehicle in the first vehicle cluster, and the sensor data is classified as one of ordered and disordered. The system and method also include processing the ordered sensor data using the front end executing the respective latency masking model, and processing the out-of-order sensor data using the back end executing the respective latency masking model, and generating a location estimate for each vehicle in the first vehicle cluster.

Description

[0001] introduce

[0002] The information provided in this section is intended to generally present the background of this disclosure. The work of the inventors listed herein (within the scope described in this section) and aspects of the specification that may otherwise not be considered prior art at the time of filing are neither expressly nor implied to be prior art to this disclosure.

[0003] This disclosure generally relates to cloud-based scalable positioning systems for vehicles. Vehicle positioning systems are an essential part of modern transportation, enabling the real-time tracking and management of multiple vehicles. These systems typically utilize a combination of Global Positioning System (GPS), cellular network, and other sensor data to determine the precise location of each vehicle. This information is then transmitted to a central server, where it can be used for various applications such as fleet management, navigation, and traffic monitoring. For example, in fleet management, operators can monitor the location and status of all vehicles, optimize routes, and improve efficiency. Similarly, navigation systems use real-time positioning data to provide drivers with accurate direction and traffic updates. These systems are crucial for ensuring the safety and efficiency of transportation networks.

[0004] However, the algorithms used to process positioning data in autonomous driving systems are highly computationally intensive. These algorithms must constantly analyze massive amounts of data from multiple sensors to make real-time driving decisions. This computational demand leads to latency in processing data packets, especially when data is transmitted over networks with varying latency. Furthermore, data packets from individual vehicles may arrive out of order, further complicating the processing and integration of this information. These challenges underscore the need for more efficient and scalable solutions to meet the growing demands of autonomous vehicle positioning systems. Summary of the Invention

[0005] One aspect of this disclosure provides a computer-implemented method for a cloud-based scalable positioning system. When executed on data processing hardware, the computer-implemented method causes the data processing hardware to perform various operations, including identifying a plurality of vehicles as a first vehicle cluster, the first vehicle cluster having a front end and a back end. Here, each of the front end and the back end of the first vehicle cluster executes a corresponding latency masking model. The operations also include receiving corresponding sensor data collected by the sensor systems of each vehicle in the first vehicle cluster, and classifying the sensor data into one of ordered and unordered. The operations also include processing the ordered sensor data using the front end of the first vehicle cluster executing the corresponding latency masking model, processing the unordered sensor data using the back end of the first vehicle cluster executing the corresponding latency masking model, and generating a position estimate for each vehicle in the first vehicle cluster.

[0006] Embodiments of this disclosure may include one or more of the following optional features. In some embodiments, classifying the sensor data into one of ordered and disordered categories includes adding the disordered sensor data to a batch of disordered sensor data. In these embodiments, processing the disordered sensor data may include processing the batch of disordered sensor data at a predetermined time threshold.

[0007] In some examples, each of the wait-time masking models is configured to receive the sensor data as input and generate the position estimate for each vehicle in the first vehicle cluster as output. In these examples, each of the wait-time masking models may execute a corresponding position prediction model. Here, the sensor data for each vehicle in the first vehicle cluster may include one or more of the following: raw sensor data detected by the respective vehicle's sensor system, proximity sensor data detected by other vehicles in the first vehicle cluster, and basic sensor data detected by a base station communicating with the first vehicle cluster.

[0008] In some implementations, the operation further includes receiving location predictions for each vehicle in the first vehicle cluster. In these implementations, the operation may also include identifying one or more of the sensor data as missing based on corresponding time data of the sensor data, and inserting the received vehicle location predictions into a factor graph of the first vehicle cluster based on the corresponding time data of the identified missing sensor data. In some examples, the operation further includes identifying that a first vehicle in the first vehicle cluster has joined a second vehicle cluster, and extracting the sensor data associated with the first vehicle from the front end of the first vehicle cluster. In these examples, the operation may also include reconstructing the factor graph of the second vehicle cluster to include the first vehicle using the sensor data associated with the first vehicle extracted from the front end of the first vehicle cluster.

[0009] Another aspect of this disclosure provides a system for cloud-based scalable positioning, comprising data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform various operations, including identifying a plurality of vehicles as a first vehicle cluster, the first vehicle cluster having a front end and a back end. Here, each of the front end and the back end of the first vehicle cluster executes a corresponding latency masking model. The operations also include receiving corresponding sensor data collected by the sensor systems of each vehicle in the first vehicle cluster, and classifying the sensor data into one of ordered and unordered. The operations also include processing ordered sensor data using the front end of the first vehicle cluster executing the corresponding latency masking model, processing unordered sensor data using the back end of the first vehicle cluster executing the corresponding latency masking model, and generating a position estimate for each vehicle in the first vehicle cluster.

[0010] This aspect may include one or more of the following optional features. In some embodiments, classifying the sensor data into one of ordered and disordered categories includes adding the disordered sensor data to a batch of disordered sensor data. In these embodiments, processing the disordered sensor data may include processing the batch of disordered sensor data at a predetermined time threshold.

[0011] In some examples, each of the wait-time masking models is configured to receive the sensor data as input and generate the position estimate of each vehicle in the first vehicle cluster as output. In these examples, each of the wait-time masking models may execute a corresponding position prediction model. In some implementations, the operation further includes identifying that a first vehicle in the first vehicle cluster has joined the second vehicle cluster, and extracting the sensor data associated with the first vehicle from the front end of the first vehicle cluster. In these implementations, the operation may further include reconstructing a factor graph of the second vehicle cluster to include the first vehicle using the sensor data associated with the first vehicle extracted from the front end of the first vehicle cluster.

[0012] Another aspect of this disclosure provides a method for waiting time masking, which, when executed on data processing hardware, causes the data processing hardware to perform various operations including receiving sensor data of a vehicle, the sensor data including spatial and temporal data, and receiving a predicted position of the vehicle. The operations also include generating a factor map of the vehicle's position, and performing factor map optimization by correlating the spatial and temporal data of the sensor data to predict the vehicle's position.

[0013] This aspect may include one or more of the following optional features. In some embodiments, the vehicle's sensor data includes one or more of the following: raw sensor data detected by the sensor system of the respective vehicle, proximity sensor data detected by other vehicles in the first vehicle cluster, and basic sensor data detected by a base station communicating with the first vehicle cluster. In these embodiments, generating the factor map for the vehicle's location may include identifying one or more of the sensor data as missing based on corresponding time data of the sensor data, and inserting the received vehicle location prediction into the factor map based on the corresponding time data of the missing sensor data.

[0014] This disclosure provides the following examples:

[0015] Example 1. A computer-implemented method, when executed on data processing hardware, causes the data processing hardware to perform operations, the operations including:

[0016] Multiple vehicles are identified as a first vehicle cluster, the first vehicle cluster having a front end and a back end, each of the front end and the back end of the first vehicle cluster performing a corresponding wait time masking model.

[0017] Receive corresponding sensor data collected by the sensor systems of each vehicle in the first vehicle cluster;

[0018] The sensor data is classified into either ordered or disordered;

[0019] The ordered sensor data is processed using the front end of the first vehicle cluster that executes the corresponding wait time masking model;

[0020] The backend of the first vehicle cluster, which executes the corresponding latency masking model, is used to process the disordered sensor data; and

[0021] Generate a location estimate for each vehicle in the first vehicle cluster.

[0022] Example 2. According to the method of Example 1, classifying the sensor data into one of ordered and disordered includes adding the disordered sensor data to a batch of disordered sensor data.

[0023] Example 3. The method according to Example 2, wherein processing the disordered sensor data includes processing the disordered sensor data batch at a predetermined time threshold.

[0024] Example 4. According to the method of Example 1, each of the waiting time masking models is configured to receive the sensor data as input and generate the position estimate of each vehicle in the first vehicle cluster as output.

[0025] Example 5. According to the method described in Example 4, each of the waiting time masking models performs a corresponding location prediction model.

[0026] Example 6. According to the method of Example 5, the sensor data of each vehicle in the first vehicle cluster includes one or more of the following:

[0027] Raw sensor data detected by the sensor system of the corresponding vehicle;

[0028] Proximity sensor data detected by other vehicles in the first vehicle cluster; and

[0029] Basic sensor data detected by the base station communicating with the first vehicle cluster.

[0030] Example 7. The method according to Example 1, wherein the operation further includes receiving a location prediction for each vehicle in the first vehicle cluster.

[0031] Example 8. The method according to Example 7, wherein the operation further includes:

[0032] Based on the corresponding time data of the sensor data, one or more of the sensor data are marked as missing; and

[0033] Based on the corresponding time data of the identified missing sensor data, the received vehicle location prediction is inserted into the factor graph of the first vehicle cluster.

[0034] Example 9. The method according to Example 1, wherein the operation further includes:

[0035] The first vehicle in the first vehicle cluster has joined the second vehicle cluster; and

[0036] The sensor data associated with the first vehicle is extracted from the front end of the first vehicle cluster.

[0037] Example 10. The method according to Example 9, wherein the operation further includes reconstructing a factor map of the second vehicle cluster to include the first vehicle using sensor data associated with the first vehicle extracted from the front end of the first vehicle cluster.

[0038] Example 11. A system comprising:

[0039] Data processing hardware; and

[0040] Memory hardware communicating with the data processing hardware, the memory hardware storing instructions, which, when executed on the data processing hardware, cause the data processing hardware to perform operations, the operations including:

[0041] Multiple vehicles are identified as a first vehicle cluster, the first vehicle cluster having a front end and a back end, each of the front end and the back end of the first vehicle cluster performing a corresponding wait time masking model.

[0042] Receive corresponding sensor data collected by the sensor systems of each vehicle in the first vehicle cluster;

[0043] The sensor data is classified into either ordered or unordered;

[0044] The ordered sensor data is processed using the front end of the first vehicle cluster that executes the corresponding wait time masking model;

[0045] The backend of the first vehicle cluster, which executes the corresponding latency masking model, is used to process the disordered sensor data; and

[0046] Generate a location estimate for each vehicle in the first vehicle cluster.

[0047] Example 12. The system according to Example 11, wherein classifying the sensor data into one of ordered and disordered includes adding the disordered sensor data to a batch of disordered sensor data.

[0048] Example 13. The system according to Example 12, wherein processing the disordered sensor data includes processing the disordered sensor data batch at a predetermined time threshold.

[0049] Example 14. The system according to Example 11, wherein each of the waiting time masking models is configured to receive the sensor data as input and generate the position estimate of each vehicle in the first vehicle cluster as output.

[0050] Example 15. The system according to Example 14, wherein each of the waiting time masking models performs a corresponding location prediction model.

[0051] Example 16. The system according to Example 11, wherein the operation further includes:

[0052] The first vehicle in the first vehicle cluster has been added to the second vehicle cluster; and

[0053] The sensor data associated with the first vehicle is extracted from the front end of the first vehicle cluster.

[0054] Example 17. The system according to Example 16, wherein the operation further includes reconstructing a factor map of the second vehicle cluster to include the first vehicle using sensor data associated with the first vehicle extracted from the front end of the first vehicle cluster.

[0055] Example 18. A computer-implemented method, when executed on data processing hardware, causes the data processing hardware to perform operations, the operations including:

[0056] Receive sensor data from the vehicle, including spatial data and temporal data;

[0057] Receive the vehicle's predicted location;

[0058] Generate a factor map of the vehicle's position; and

[0059] Factor graph optimization is performed by correlating the spatial and temporal data from the sensor data to predict the vehicle's location.

[0060] Example 19. The method according to Example 18, wherein the sensor data of the vehicle includes one or more of the following:

[0061] Raw sensor data detected by the vehicle's sensor system;

[0062] Proximity sensor data detected by other vehicles approaching the vehicle; and

[0063] Basic sensor data detected by a base station communicating with the vehicle.

[0064] Example 20. The method according to Example 18, wherein generating the factor map for the vehicle's location includes:

[0065] Based on the corresponding time data of the sensor data, one or more of the sensor data are marked as missing; and

[0066] Based on the corresponding time data of the missing sensor data, the received vehicle position prediction is inserted into the factor graph.

[0067] Details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will become apparent from the description, the drawings, and the claims. Attached Figure Description

[0068] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0069] Figure 1 This is a schematic diagram of an example system for cloud-based scalable positioning.

[0070] Figure 2 yes Figure 1 A schematic diagram of an example component.

[0071] Figure 3 yes Figure 1 A schematic diagram of an example component.

[0072] Figure 4A and 4B Is Figure 1 A schematic diagram of vehicles moving between clusters of connected systems.

[0073] Figure 5 yes Figure 1 A schematic diagram of an example component.

[0074] Figure 6 This is a flowchart illustrating an example operational setup for a cloud-based scalable positioning system.

[0075] Figure 7 This is a flowchart of an example operation setup for the waiting time masking method.

[0076] Throughout the accompanying figures, the corresponding figure labels indicate the relevant parts. Detailed Implementation

[0077] The example configuration will now be described more fully with reference to the accompanying drawings. The example configuration is provided so that this disclosure will be comprehensive and will fully convey the scope of this disclosure to those skilled in the art. Specific details, such as examples of specific components, devices, and methods, are set forth to provide a comprehensive understanding of the configuration of this disclosure. It will be apparent to those skilled in the art that the specific details are not required, the example configuration may be embodied in many different forms, and the specific details and example configuration should not be construed as limiting the scope of this disclosure.

[0078] The terminology used herein is for describing specific exemplary configurations only and is not intended to be limiting. As used herein, the singular articles “a,” “an,” and “the” may also be intended to include plural forms unless the context explicitly indicates otherwise. The terms “comprising,” “including,” and “having” are inclusive and therefore specify the presence of features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein should not be construed as necessarily requiring them to be performed in the specific order discussed or illustrated, unless explicitly identified as such. Additional or alternative steps may be employed.

[0079] When an element or layer is referred to as “on another element or layer,” “joined to,” “connected to,” “attached to,” or “coupled to” another element or layer, it may be located directly on, joined to, connected to, attached to, or coupled to that other element or layer, or there may be intermediate elements or layers present. Conversely, when an element is referred to as “directly on another element or layer,” “directly joined to,” “directly connected to,” “directly attached to,” or “directly coupled to” another element or layer, there may be no intermediate elements or layers present. Other terms used to describe relationships between elements should be interpreted in a similar manner (e.g., “between” vs. “directly between,” “adjacent” vs. “directly adjacent,” etc.). As used herein, the term “and / or” includes any and all combinations of one or more of the related listed items.

[0080] The terms “first,” “second,” “third,” etc., are used herein to describe various elements, components, regions, layers, and / or sections. These elements, components, regions, layers, and / or sections should not be limited by these terms. These terms are used only to distinguish one element, component, region, layer, or section from another. Unless the context explicitly indicates otherwise, terms such as “first,” “second,” and other numerical terms do not imply order or sequence. Therefore, the first element, component, region, layer, or section discussed below may be referred to as the second element, component, region, layer, or section without departing from the teachings of the example configuration.

[0081] In this application, the term "module" is replaced by the term "circuit" as defined below. The term "module" may refer to, be part of, or include the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor (shared, dedicated, or grouped) that executes code; memory (shared, dedicated, or grouped) that stores code executed by the processor; other suitable hardware components that provide the aforementioned functionality; or combinations of some or all of the foregoing, such as in a system-on-a-chip.

[0082] The term "code" as used above can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, and / or objects. The term "shared processor" covers a single processor that executes some or all of the code from multiple modules. The term "group processor" covers a processor that, in combination with additional processors, executes some or all of the code from one or more modules. The term "shared memory" covers a single memory that stores some or all of the code from multiple modules. The term "group memory" covers memory that, in combination with additional memory, stores some or all of the code from one or more modules. The term "memory" can be a subset of the term "computer-readable medium." The term "computer-readable medium" does not include transient electrical and electromagnetic signals propagating through a medium, and therefore can be considered tangible and non-transient memory. Non-limiting examples of non-transient memory include tangible computer-readable media, including non-volatile memory, magnetic storage devices, and optical storage devices.

[0083] The apparatus and methods described in this application may be implemented, in part or in whole, by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer programs may also include and / or depend on stored data.

[0084] A software application (i.e., a software resource) can refer to computer software that instructs a computing device to perform a task. In some examples, a software application may be referred to as an "application," "app," or "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and game applications.

[0085] Non-transient memory can be a physical device used for temporary or permanent storage of programs (e.g., instruction sequences) or data (e.g., program state information) for use by a computing device. Non-transient memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used in firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change memory (PCM), and magnetic disks or magnetic tapes.

[0086] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0087] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0088] The processes and logic described in this specification can be executed by one or more programmable processors (also known as data processing hardware) that execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic can also be executed by special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, or operably coupled to receive data from or transfer data to, or both, such as magnetic disks, magneto-optical disks, or optical disks. However, a computer does not necessarily need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0089] To provide interaction with a user, one or more aspects of this disclosure can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touchscreen) for displaying information to the user and optionally having a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending web pages to a web browser on the user's client device in response to a request received from a web browser.

[0090] refer to Figure 1In some embodiments, system 100 includes a plurality of vehicles 10 (10a-c) that form a cluster 20 and communicate with each other and with base station 30 via network 40. Additionally, system 100 includes a remote system 60 that communicates with the cluster 20 of vehicles 10 and / or base station 30 via network 40. Network 40 may include a wireless local area network (WLAN) that facilitates communication and interoperability between vehicles 10, base station 30, and remote system 60 within the environment of vehicle 10. Therefore, network 40 may include Wi-Fi (e.g., IEEE 802.11), low-rate wireless personal area networks (e.g., IEEE 802.15.4), WiMAX, 3G, 4G, LTE, 5G, DSL, Bluetooth, NFC, or any other wireless standard, or Ethernet (e.g., IEEE 802.3). Vehicle 10 and / or base station 30 may additionally include one or more access points (APs) (not shown) configured to facilitate wireless communication between vehicle 10, base station 30 and / or remote system 60.

[0091] As shown, vehicle 10 and / or remote system 60 in cluster 20 perform connection to cluster system 200. Figure 2 The connected cluster system 200 is configured to correlate spatial and temporal data, define a graph-based optimization problem, and compute the position of each vehicle 10 in the cluster 20 while mitigating network latency and the impact of asynchronous data. In short, and as described further below, conventional positioning systems typically estimate vehicle positions by solving computational graphs (e.g., factor graphs, attitude graphs, etc.) using sensor measurements from individual vehicles 10. In these positioning systems, computational graphs are used to identify the positions of individual vehicles 10 using individual sensor data collected by the vehicles 10. Furthermore, these positioning systems may receive out-of-order sensor data, requiring continuous updates to the factor graphs, which can delay position estimation.

[0092] In contrast, the connected clustering system 200, performed by the cluster 20 of vehicles 10, is configured to receive sensor data 202 from multiple vehicles 10, identify which of the multiple vehicles 10 are close to each other, and cluster these vehicles 10 together to utilize vehicle-to-vehicle (V2V) measurements that accurately map one or more vehicles 10 about each other. Here, due to the inherent mobility of the vehicles 10, the connected clustering system 200 dynamically identifies the cluster 20 of vehicles 10 that are close to each other at any given time, improving the flexibility and scalability of the connected clustering system 200 while limiting the computational complexity of the increasing number of interconnected vehicles 10. For example, the connected clustering system 200 generates a position estimate 332 for each vehicle 10 in the cluster 20, which will be used by downstream applications of the vehicles 10 and by other vehicles 10 in the cluster 20. Advantageously, the connected cluster system 200 utilizes batch and parallel processing by executing a front-end system 210 and a separate back-end system 220. The front-end system 210 provides a fast response time for the time-sensitive data 202 of the location estimate 332, while the back-end system 220 performs periodic background processing of the disordered and / or delayed sensor data 202 to ensure the accuracy of the location estimate 332. Here, the front-end system 210 can generate / provide an initial factor map for the vehicle cluster 20, and after the back-end system 220 performs its periodic background processing, it updates the initial factor map using the background factor map generated by the back-end system 220. As described further in detail below, each of the front-end system 210 and the back-end system 220 is configured to perform graph recalculation of the corresponding factor map. During this graph recalculation, each corresponding masking model 300a, 300b can update the belief (i.e., probability or estimate) associated with each variable (i.e., location estimate 322) in the factor map. The graph is recalculated / updated after a factor node or variable node in the factor graph changes, or when new sensor data 202 becomes available.

[0093] In the example shown, the connected cluster system 200 is implemented within vehicles 10a-10c. However, the connected cluster system 200 can be implemented in any other propulsion system (such as, but not limited to, motorcycles, trucks, off-road vehicles, agricultural equipment, trains, airplanes, etc.). Each vehicle 10a-10c includes corresponding data processing hardware 12a-12c and memory hardware 14a-14c for storing instructions, which cause the data processing hardware 12 to perform operations when executed on the data processing hardware 12. Each vehicle 10a-10c also includes one or more corresponding sensors 16a-16c configured to capture / receive sensor data 202. The one or more sensors 16a-16c may include one or more of the following: long-range radar sensors, camera sensors capable of capturing image data, global positioning systems (GPS), speedometers, odometers, accelerometers, wireless ranging devices, inertial measurement units (IMUs), etc. Sensor data 202 may include dynamic characteristics of vehicle 10 (such as speed, yaw and acceleration) and wireless measurements (such as time of flight (TOF), angle of arrival (AoA) and so on), and may be transmitted via network 40 at 10 Hz to connected cluster system 200 (i.e., 5G wireless transmission).

[0094] Remote system 60 (e.g., a server, a cloud computing environment) also includes data processing hardware 62 and memory hardware 64 for storing instructions, which, when executed on data processing hardware 62, cause data processing hardware 62 to perform operations. In some examples, the execution of the connected cluster system 200 is shared between cluster 20 of vehicle 10 and remote system 60. In other examples, remote system 60 executes connected cluster system 200, where remote system 60 operates as a central host / controller. In additional examples, connected cluster system 200 executes on one or more vehicles 10 in cluster 20 (i.e., shared among multiple vehicles 10).

[0095] like Figure 1As shown, the connected cluster system 200 receives corresponding sensor data 202 from each of the vehicles 10a-10c as input. The sensor data 202 may include spatial data 204 and corresponding temporal data 206 (e.g., timestamps). Based on the relative proximity of the vehicles 10a-10c, the connected cluster system 200 can group the vehicles 10a-10c into a first cluster 20. Here, the connected cluster system 200 can utilize various types of sensor data 202 from each vehicle 10 within the cluster 20. For example, the corresponding sensor data 202 for each vehicle 10 may include one or more of the following: raw sensor data 202 detected by the sensor system 16 of the corresponding vehicle 10, proximity sensor data 202 detected by the corresponding sensor systems 16 of other vehicles 10 in the cluster 20 of vehicles 10, and basic sensor data 202 detected by a base station 30 communicating with the vehicles 10.

[0096] Upon receiving sensor data 202, the front-end system 210 of the connected cluster system 200 filters the sensor data 202 by classifying it into one of ordered sensor data 202I and unordered sensor data 202O. Here, the front-end system 210 can immediately process the ordered sensor data 202I to use the corresponding latency masking model 300a. Figure 3 To generate a fast-response position estimate 332 for each vehicle 10 in the cluster 20 of vehicles 10a-10c. In parallel, the backend system 220 receives out-of-order sensor data 202O and packages the out-of-order sensor data 202O into batches 242 to avoid redundant updates to the position estimate 332 for each vehicle 10 in the cluster 20. After each predetermined time threshold (e.g., every ten (10) time periods) has elapsed, the backend system 220 uses the corresponding wait time masking model 300b. Figure 3 The system processes batches 242 of unordered sensor data 202O to generate high-throughput location estimates 332 for each vehicle 10 in the cluster 20 of vehicles 10a-10c. In some implementations, the connected cluster system 200 may use the location estimate 332 generated by the back-end system 220 instead of the fast-response location estimate 332 generated by the front-end system 210.

[0097] refer to Figure 2 and 3The front-end system 210 may include a data filtering module 230 and a waiting time masking model 300a, while the back-end system 220 includes a batch processing module 240 and a waiting time masking model 300b. The data filtering module 230 is configured to receive sensor data 202 from each vehicle 10 in the cluster 20 and classify the sensor data 202 into one of ordered sensor data 202I and unordered sensor data 202O. For example, the data filtering module 230 may classify the sensor data 202 based on its corresponding time data 206, which indicates the timestamp of the corresponding sensor data 202. In these cases, the time data 206 may indicate whether the corresponding sensor data 202 is still synchronized with the current sensor data 202 being processed by the connected cluster system 200 and / or the common time concept of the connected cluster systems 200. In some implementations, the data filtering module 230 may identify high-priority sensor data 202 (e.g., safety-critical sensor data 202) as ordered sensor data 202I to ensure that the latency masking model 300a executed by the front-end system 210 processes the higher-priority sensor data 202. Conversely, the data filtering module 230 may classify time-consuming sensor data 202 as unordered sensor data 202O and output the unordered sensor data 202O to the back-end system 220, which executes the latency masking model 300b in the background using batch processing to maintain the fast response time of the latency masking model 300a of the front-end system 210. In some cases, when the upload time of sensor data 202 to the connected cluster system 200 exceeds a threshold (e.g., 0.2 seconds), the data filtering module 230 identifies the sensor data 202 as unordered sensor data 202O.

[0098] Subsequently, sensor data 202 identified as disordered sensor data 202O can be received as input to batch processing module 240. Batch processing module 240 can maintain / save the incoming disordered sensor data 202O in batch 242 for processing by waiting time masking model 300b. Here, batch processing module 240 is configured to receive disordered sensor data 202O from data filtering module 230, add disordered sensor data 202O to batch 242 of disordered sensor data 202O, and only trigger / start the execution of waiting time masking model 300b to process batch 242 of disordered sensor data 202O at a predetermined time threshold. For example, waiting time masking model 300b can process batch 242 of disordered sensor data 202O only every N seconds, where N can include any number of seconds, such as but not limited to two (2), five (5), ten (10) seconds, etc.

[0099] refer to Figure 3The diagram illustrates a latency masking model 300. It should be understood that although the connected cluster system 200 executes latency masking models 300a and 300b on the front-end system 210 and back-end system 220 respectively, the underlying architecture of each of the latency masking models 300a and 300b is the same. Therefore, Figure 3 The waiting time masking model 300 describes both the waiting time masking model 300a executed by the front-end system 210 and the waiting time masking model 300b executed by the back-end system 220 in the same way.

[0100] As shown, the waiting time masking model 300 includes an embedding module 310, a motion prediction model 320, and a factor graph module 330. Furthermore, the waiting time masking model 300 has access to an embedded state data storage device 340, which resides on the corresponding memory hardware 14 of one or more vehicles 10 and / or the memory hardware 64 of the remote system 60. The embedding module 310 is configured to receive sensor data 202 (e.g., ordered sensor data 202I and / or disordered sensor data 202O) as input and generate an embedding state 312 corresponding to the sensor data 202 as output. For example, the generated embedding state 312 output at each time step can be stored in the embedded state data storage device 340. Here, the sensor data 202 includes spatial data 204 and corresponding temporal data 206. It is noteworthy that, because the waiting time masking model 300 may receive incomplete sensor data 202 (i.e., one or more received sensor data 202 are missing and / or delayed), the waiting time masking model 300 is configured to use historical embedded states 312 stored in the embedded state data storage device 340 to predict the position of the vehicle 10 based on the historical embedded states 312 at any given time. When the embedded states 312 of one or more sensor data 202 are missing / not counted, the historical embedded states 312 can be used to generate factor maps.

[0101] Specifically, the motion prediction model 320 of the waiting time masking model 300 is configured to receive the previous embedding state 312 of vehicle 10 and / or the previous embedding state 312 of other vehicles 10 in cluster 20 as input, and generate a position prediction 322 of vehicle 10 as output. Here, the motion prediction model 320 uses the historical embedding state 312 of vehicle 10 in cluster 20 to infer the optimal position estimate of vehicle 10. Subsequently, the factor graph module 330 receives the position prediction 322 of vehicle 10 and the embedding state 312 of vehicle 10 as input, and generates a factor graph for the position estimate 332 of vehicle 10 in cluster 20. Here, the factor graph module 330 uses the position prediction 322 and the embedding state 312 of vehicle 10 and / or other vehicles 10 in cluster 20 to generate the factor graph. In some instances, when generating each node of the factor graph, the factor graph module 330 concatenates the embedding state 312 with the position prediction 322.

[0102] In some cases, the factor graph module 330 identifies one or more missing sensor data 202 based on the time data 206 corresponding to the sensor data 202. In these cases, when vehicle 10 has missing and / or delayed sensor data 202, the factor graph module 330 can insert a location prediction 322 corresponding to the time data 206 of the missing and / or delayed sensor data 202 to populate the factor graph, and can further minimize any errors in the factor graph using any embedding state 312 of the sensor data 202 of vehicle 10 collected by other vehicles 10 in cluster 20. When missing and / or delayed sensor data 202 arrives, the factor graph module 330 can update the historical factor graph of the embedding state 312 to make the next location calculation most accurate.

[0103] When generating a factor map using embedded state 312 and position prediction 322, factor map module 330 performs factor map optimization by associating spatial data 204 and temporal data 206 of sensor data 202 to predict the position estimate 332 for each vehicle 10 in cluster 20. Subsequently, the position estimate 332 and factor map can be stored in embedded state data storage device 340 for future prediction by motion prediction model 320 and / or updated by delayed sensor data 202.

[0104] Refer again Figure 2The waiting time masking model 300a of the front-end system 210 receives ordered sensor data 202I classified by the data filtering module 230 as input and generates a position estimate 332 for each vehicle 10 in the cluster 20 as output. In parallel, at each time threshold period, the batch processing module 240 of the back-end system 220 feeds batches 242 of unordered sensor data 202O to the waiting time masking model 300b. The waiting time masking model 300b then generates its own position estimate 332 for each of the multiple vehicles 10 in the cluster 20 as output. The front-end system 210 can receive the position estimates 332 generated by the waiting time masking model 300b of the back-end system 220, and can update / recalculate the current factor graph and perform factor graph optimization to generate an updated / recalculated position estimate 332 for each of the multiple vehicles 10 in the cluster 20.

[0105] refer to Figure 4A , 4B And 5, due to the mobility of vehicle 10, the connected cluster system 200 dynamically adjusts and / or reforms the composition of vehicles 10 within a specific cluster 20. For example, as Figure 4A and 4B As shown, environments 400a and 400b may include a first cluster 20a and a second cluster 20b. Furthermore, as... Figure 5 As shown, each cluster 20a, 20b can have a corresponding front-end system 210a, 210b and a corresponding back-end system 220a, 220b (220a not shown). In environment 400a (i.e., Figure 4A In the cluster system 200, the connected cluster system 200 is identified as a first cluster 20a comprising three (3) vehicles 10 (vehicles 10a, 10b, and 10c), and a second cluster 20b comprising three (3) vehicles 10 (10d, 10e, and 10f). However, as Figure 4B As shown, environment 400b has changed so that vehicle 10a is now closer to vehicles 10d-10f. Here, the connected cluster system 200 can update the first cluster 20a to remove vehicle 10a, leaving only two (2) vehicles 10b and 10c. Furthermore, the connected cluster system 200 updates the second cluster 20b to include vehicles 10a that are now closely adjacent to three (3) vehicles 10d-10f, so that the second cluster 20b now includes four (4) vehicles 10a, 10d-10f.

[0106] like Figure 5As shown, after the connected cluster system 200 identifies the updated first cluster 20a and the updated second cluster 20b, the connected cluster system 200 extracts sensor data 202a and embedding state 312a from the front-end system A 210a to the back-end system B 220b. The sensor data 202a and embedding state 312a correspond to vehicles 10a that have migrated from the first cluster 20a to the second cluster 20b. Here, the back-end system B 220b can perform graph reconstruction (i.e., via its corresponding masking wait time model 300b) to include vehicles 10a in the factor graph and generate a position estimate 332 for vehicles 10a in the second cluster 20b. Subsequently, the back-end system B 220b can provide the resulting position estimate 332 and a factor graph including the additional vehicles 10a that have joined cluster 20a as input to the front-end system B 210b, wherein the front-end system B 210b also updates the position estimate 332 for each vehicle 10a, 10d-10f in cluster 20b.

[0107] Figure 6 This includes a flowchart of an example operational setup for a method 600 for a cloud-based scalable positioning system. See also... Figure 1-5 To describe method 600. Data processing hardware (e.g., Figure 1 The data processing hardware 12a-12c, 62) can perform operations stored in memory hardware (e.g., Figure 1 Instructions on the memory hardware 14a-14c, 64) are used to execute the example operation arrangement of method 600.

[0108] At operation 602, method 600 includes identifying a plurality of vehicles 10 (10a-c) as a first cluster 20 of vehicles 10. The first cluster 20 of vehicles 10 includes a front-end system 210 and a back-end system 220, each of the front-end system 210 and the back-end system 220 of the first cluster 20 of vehicles 10 executing a corresponding latency masking model 300a, 300b. At operation 604, method 600 includes receiving corresponding sensor data 202 collected by the sensor system 16 of each vehicle 10 in the first cluster 20 of vehicles 10.

[0109] Method 600 also includes, at operation 606, classifying sensor data 202 into one of ordered sensor data 202I and unordered sensor data 202O. At operation 608, method 600 includes processing ordered sensor data 202I using a front-end system 210 of a first cluster 20 of vehicles 10 executing a corresponding latency masking model 300a. At operation 610, method 600 also includes processing unordered sensor data 202O using a back-end system 220 of the first cluster 20 of vehicles 10 executing a corresponding latency model 300b. Method 600 also includes, at operation 612, generating a position estimate 332 for each vehicle 10 in the first cluster 20 of vehicles 10.

[0110] Figure 7 This includes a flowchart of an example operational setup for method 700 for waiting time masking. See also... Figure 1-5 To describe method 700. Data processing hardware (e.g., Figure 1 The data processing hardware 12a-12c, 62) can perform operations stored in memory hardware (e.g., Figure 1 Instructions on the memory hardware 14a-14c, 64) are used to execute the example operation arrangement of method 700.

[0111] At operation 702, method 700 includes receiving sensor data 202 of vehicle 10. Here, sensor data 202 includes spatial data 204 and temporal data 206. At operation 704, method 700 also includes receiving a position prediction 222 for vehicle 10. Method 700 further includes, at operation 706, generating a factor map for a position estimate 332 of vehicle 10. At operation 708, method 700 also includes performing factor map optimization by correlating the spatial data 204 and temporal data 206 of sensor data 202 to predict the position estimate 332 of vehicle 10.

[0112] Several embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other embodiments are within the scope of the following claims.

[0113] The foregoing description is provided for illustrative purposes only. It is not intended to be exhaustive or limiting of this disclosure. Individual elements or features of a particular configuration are generally not limited to that particular configuration, but are (where applicable) interchangeable and can be used in selected configurations, even if they are not specifically shown or described. These elements or features can also be varied in many ways. Such variations should not be considered a departure from this disclosure, and all such modifications are intended to be included within the scope of this disclosure.

Claims

1. A computer-implemented method, when executed on data processing hardware, causes the data processing hardware to perform operations, the operations including: Multiple vehicles are identified as a first vehicle cluster, the first vehicle cluster having a front end and a back end, each of the front end and the back end of the first vehicle cluster performing a corresponding wait time masking model. Receive corresponding sensor data collected by the sensor systems of each vehicle in the first vehicle cluster; The sensor data is classified into either ordered or disordered; The ordered sensor data is processed using the front end of the first vehicle cluster that executes the corresponding wait time masking model; The backend of the first vehicle cluster, which executes the corresponding wait time masking model, is used to process the disordered sensor data; as well as Generate a location estimate for each vehicle in the first vehicle cluster.

2. The method of claim 1, wherein classifying the sensor data into one of ordered and disordered includes adding the disordered sensor data to a batch of disordered sensor data.

3. The method of claim 2, wherein processing the disordered sensor data includes processing the disordered sensor data batch at a predetermined time threshold.

4. The method of claim 1, wherein each of the waiting time masking models is configured to receive the sensor data as input and generate the position estimate of each vehicle in the first vehicle cluster as output.

5. The method of claim 4, wherein each of the waiting time masking models performs a corresponding location prediction model.

6. The method of claim 5, wherein the sensor data of each vehicle in the first vehicle cluster includes one or more of the following: Raw sensor data detected by the sensor system of the corresponding vehicle; Proximity sensor data detected by other vehicles in the first vehicle cluster; and Basic sensor data detected by the base station communicating with the first vehicle cluster.

7. The method of claim 1, wherein the operation further comprises receiving a location prediction for each vehicle in the first vehicle cluster.

8. The method of claim 7, wherein the operation further comprises: Based on the corresponding time data of the sensor data, one or more of the sensor data will be marked as missing; as well as Based on the corresponding time data of the identified missing sensor data, the received vehicle location prediction is inserted into the factor graph of the first vehicle cluster.

9. The method according to claim 1, wherein the operation further comprises: The first vehicle in the first vehicle cluster has joined the second vehicle cluster; as well as The sensor data associated with the first vehicle is extracted from the front end of the first vehicle cluster.

10. The method of claim 9, wherein the operation further comprises reconstructing a factor map of the second vehicle cluster to include the first vehicle using sensor data associated with the first vehicle extracted from the front end of the first vehicle cluster.

Images