Computer-implemented method, vehicle and computer-readable medium

Abstract

This application relates to computer-implemented methods, vehicles, and computer-readable media. Further, techniques are described for identifying sensor data from a sensor of a first vehicle, the sensor data including information related to poses of at least two other vehicles on a roadway. The techniques further include determining a geometry of a portion of the roadway based at least in part on the information related to the poses of the at least two other vehicles. The techniques further include comparing the geometry of the portion of the roadway to map data to identify a match between the portion of the roadway and a portion of the map data. The techniques further include determining a pose of the first vehicle relative to the map data based at least in part on the match.

Description

Technical Field

[0001] This application relates to positioning based on surrounding vehicles. Background Technology

[0002] Autonomous vehicles (AVs) will use at least one sensor in a process referred to herein as localization to identify their location. However, in dense urban areas, at least one sensor may be obstructed. When one or more sensors are obstructed, the AV will be unable to extract sufficient information about its surroundings for localization. Summary of the Invention

[0003] According to one aspect of the invention, a computer-implemented method includes: using at least one processor to identify sensor data from sensors of a first vehicle, wherein the sensor data includes information relating to the attitudes of at least two other vehicles on a road; using the at least one processor to determine, at least in part, the geometry of a portion of the road based on the information relating to the attitudes of the at least two other vehicles; using the at least one processor to compare the geometry of the portion of the road with map data to identify a match between the portion of the road and a portion of the map data; and using at least in part, determining the attitude of the first vehicle relative to the map data based on the match.

[0004] According to another aspect of the invention, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors, cause a vehicle to: identify sensor data from sensors of a first vehicle, wherein the sensor data includes information relating to the attitudes of at least two other vehicles on the road; determine the geometry of a portion of the road based at least in part on the information relating to the attitudes of the at least two other vehicles; compare the geometry of the portion of the road with map data to identify a match between the portion of the road and a portion of the map data; and determine the attitude of the first vehicle relative to the map data based at least in part on the match.

[0005] According to another aspect of the invention, a vehicle includes: a sensor; one or more processors; and one or more non-transitory computer-readable media including instructions that, when executed by the one or more processors, cause the vehicle to: identify sensor data from the sensor of the vehicle, wherein the sensor data includes information relating to the attitudes of at least two other vehicles on a road; determine the geometry of a portion of the road based at least in part on the information relating to the attitudes of the at least two other vehicles; compare the geometry of the portion of the road with map data to identify a match between the portion of the road and a portion of the map data; and determine the attitude of the vehicle relative to the map data based at least in part on the match. Attached Figure Description

[0006] Figure 1 An example of an AV with autonomous capabilities is shown.

[0007] Figure 2 The computer system is shown.

[0008] Figure 3 An example architecture for AV is shown.

[0009] Figure 4 Examples of inputs and outputs that the perception module can use are shown.

[0010] Figure 5 A block diagram showing the relationship between the inputs and outputs of the planned circuit.

[0011] Figure 6 Examples of lane detection based on other vehicles are shown according to various embodiments.

[0012] Figure 7 Examples of map data according to various embodiments are shown.

[0013] Figure 8 Examples of positioning based on lane detection and map data according to various embodiments are shown.

[0014] Figure 9 Examples of positioning techniques according to various embodiments are shown.

[0015] Figure 10 Alternative examples of positioning techniques according to various embodiments are shown. Detailed Implementation

[0016] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the invention. However, it will be apparent that the invention can be practiced without these specific details. In other instances, well-known constructions and apparatuses are shown in block diagram form to avoid unnecessarily obscuring the invention.

[0017] In the accompanying drawings, for ease of description, a specific arrangement or order of schematic elements (such as those representing devices, modules, instruction blocks, and data elements) is shown. However, those skilled in the art will understand that the specific order or arrangement of the schematic elements in the drawings is not intended to imply a requirement for a particular processing order or sequence, or a separation of processing procedures. Furthermore, the inclusion of schematic elements in the drawings is not intended to imply that such elements are required in all embodiments, nor is it intended to imply that features represented by such elements cannot be included in some embodiments or cannot be combined with other elements in some embodiments.

[0018] Furthermore, in the accompanying drawings, connecting elements, such as solid or dashed lines or arrows, are used to illustrate connections, relationships, or associations between two or more other schematic elements. The absence of any such connecting element does not imply that connections, relationships, or associations cannot exist. In other words, connections, relationships, or associations between some elements are not shown in the drawings so as not to obscure the content of this disclosure. Additionally, for ease of illustration, a single connecting element is used to represent multiple connections, relationships, or associations between elements. For example, if a connecting element represents communication of signals, data, or instructions, those skilled in the art will understand that such an element represents one or more signal paths (e.g., a bus) that may be necessary to influence the communication.

[0019] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0020] The features described below can each be used independently of each other or in any combination with other features. However, any individual feature may not solve any of the problems discussed above, or may only solve one of the problems discussed above. Some of the problems discussed above may not be adequately solved by any of the features described herein. Although headings are provided, information relating to specific headings but not found in the sections bearing those headings can be found elsewhere in this specification. Embodiments are described herein based on the following summary:

[0021] 1. General Overview

[0022] 2. System Overview

[0023] 3. AV Architecture

[0024] 4. AV Input

[0025] 5. Path planning

[0026] 6. Lane detection and positioning

[0027] General Overview

[0028] To determine the location of a vehicle (such as an AV), the vehicle uses onboard sensors to collect information about other vehicles on the road. For example, the vehicle uses onboard cameras, LiDAR, RADAR, or other sensors to determine the attitude of other vehicles on the road. The vehicle uses the sensed information about other vehicles to estimate the geometry of a portion of the road, such as lanes. The vehicle compares the determined geometry of this portion of the road with map data to ensure a match. Based on this match, the vehicle determines its location, orientation, or other characteristics used to locate other vehicles on the road.

[0029] Some advantages of these technologies include improved vehicle localization and navigation. For example, the technologies allow vehicles to locate themselves without relying on landmarks, thereby improving localization in areas with few landmarks (e.g., roads) and in situations where landmarks (or sensors used to detect them) are obscured. It will also be understood that, as used herein, “localization” refers to identifying the vehicle’s attitude (e.g., location and orientation) relative to identified map data.

[0030] Thus, the positioning technique described in this paper complements other positioning techniques, and vehicles can be configured to select the optimal positioning technique based on characteristics of the vehicle's environment or the state of its sensors. By improving positioning, the technique described in this paper allows for safer and more robust navigation.

[0031] System Overview

[0032] Figure 1 An example of an AV 100 with autonomous capabilities is shown.

[0033] As used herein, the term “autonomy” refers to a function, feature, or facility that enables a vehicle to operate partially or fully without real-time human intervention, including but not limited to full AV, high AV, and conditional AV.

[0034] As used in this article, an AV is a vehicle with autonomous capabilities.

[0035] As used in this article, "vehicle" includes any mode of transport for goods or people. Examples include cars, buses, trains, airplanes, drones, trucks, ships, vessels, submersibles, and spacecraft. Driverless cars are an example of vehicles.

[0036] As used herein, a “track” refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In embodiments, the first spatiotemporal location is referred to as the initial location or starting point, and the second spatiotemporal location is referred to as the destination, final location, target, target location, or target position. In some examples, a track consists of one or more segments (e.g., segments of a road), and each segment consists of one or more blocks (e.g., a lane or part of an intersection). In embodiments, spatiotemporal locations correspond to real-world locations. For example, a spatiotemporal location is a pick-up or drop-off point for people or goods to board or alight.

[0037] As used herein, “(one or more) sensors” includes one or more hardware components for detecting information relating to the environment surrounding the sensor. Some hardware components may include sensing components (e.g., image sensors, biometric sensors), transmission and / or receiving components (e.g., laser or radio frequency wave transmitters and receivers), electronic components (such as analog-to-digital converters), data storage devices (such as random access memory (RAM) and / or non-volatile memory), software or firmware components, and data processing components (such as application-specific integrated circuits), microprocessors, and / or microcontrollers.

[0038] As used herein, a “scene description” is a data structure (e.g., a list) or data stream that includes one or more classified or labeled objects detected by one or more sensors on an AV vehicle, or one or more classified or labeled objects provided by a source outside the AV.

[0039] As used in this article, a "road" is a physical area that can be traversed by vehicles and can correspond to a named passageway (e.g., a city street, an interstate highway, etc.) or an unnamed passageway (e.g., a driveway within a house or office building, a section of a parking lot, a section of an vacant parking lot, a waste disposal area in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, SUVs, etc.) can traverse a variety of physical areas that are not particularly suitable for vehicle travel, a "road" can be any physical area that is not formally defined as a passageway by any municipality or other government or administrative agency.

[0040] As used herein, a "lane" is the portion of a road that can be traversed by vehicles. Sometimes lanes are identified based on lane markings. For example, a lane may correspond to most or all of the space between lane markings, or only a portion of the space between lane markings (e.g., less than 50%). For instance, a road with widely spaced lane markings may accommodate two or more vehicles, allowing one vehicle to overtake another without crossing the lane markings; therefore, this could be interpreted as a lane being narrower than the space between lane markings, or as having two lanes. Lanes can also be interpreted in the absence of lane markings. For example, a lane may be defined based on physical features of the environment (e.g., rocks and trees along a main road in a rural area, or natural obstacles that should be avoided, for example, in underdeveloped areas). Lanes can also be interpreted independently of lane markings or physical features. For example, a lane may be interpreted based on any unobstructed path in an area that would otherwise lack features that would be interpreted as lane boundaries. In the example scenario, an AV could interpret a lane as a lane traversing an unobstructed portion of a field or open space. In another example scenario, an AV can interpret lanes on a wide road (e.g., wide enough for two or more lanes) that does not have lane markings. In this scenario, an AV can communicate lane-related information to other AVs, allowing them to coordinate route planning using the same lane information.

[0041] The term “over-the-air (OTA) client” includes any AV, or any electronic device embedded in, coupled to, or communicating with an AV (e.g., computer, controller, IoT device, electronic control unit (ECU)).

[0042] The term "OTA update" means any update, alteration, deletion, or addition to software, firmware, data, or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies, including but not limited to: cellular mobile communications (e.g., 2G, 3G, 4G, 5G), radio local area networks (e.g., WiFi), and / or satellite Internet.

[0043] The term "edge node" refers to one or more edge devices coupled to a network that provide a portal for communicating with AV and can communicate with other edge nodes and cloud-based computing platforms to schedule OTA updates and deliver OTA updates to OTA clients.

[0044] The term "edge device" refers to a device that implements an edge node and provides a physical wireless access point (AP) to the core network of an enterprise or service provider (such as Verizon or AT&T). Examples of edge devices include, but are not limited to: computers, controllers, transmitters, routers, routing switches, integrated access devices (IADs), multiplexers, metropolitan area network (MAN) and wide area network (WAN) access devices.

[0045] "One or more" includes functions performed by a single element, functions performed by multiple elements, such as in a distributed manner, several functions performed by a single element, several functions performed by several elements, or any combination of the foregoing.

[0046] It will also be understood that, although in some cases the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact.

[0047] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments described and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that “and / or” as used herein refers to and includes any and all possible combinations of one or more of the relevant list items. It will also be understood that when the terms “comprising,” “including,” “possessing,” and / or “having” are used in this specification, they specifically indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0048] As used herein, depending on the context, the term "if" may optionally be understood as meaning "when" or "at that time" or "in response to being determined" or "in response to being detected." Similarly, depending on the context, the phrase "if determined" or "if [the stated condition or event] has been detected" may optionally be understood as meaning "when determined" or "in response to being determined" or "when [the stated condition or event] is detected" or "in response to being detected."

[0049] As used herein, an AV system refers to an AV and an array of hardware, software, stored data, and real-time generated data that support AV operation. In embodiments, the AV system is incorporated within an AV. In embodiments, the AV system is distributed across several locations. For example, some of the software for the AV system is implemented in a cloud computing environment.

[0050] Generally, this document describes technologies applicable to any vehicle with one or more autonomous capabilities, including fully automated vehicle (AV), highly automated vehicle (AV), and conditionally automated vehicle (AV), such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads, the entire contents of which are incorporated herein by reference for further details on vehicle autonomy levels). The technologies described in this document are also applicable to partially automated vehicle (AV) and driver-assisted vehicle (MAV) vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads). In embodiments, one or more Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle systems are capable of automatically performing certain vehicle operations (e.g., steering, braking, and map usage) under certain operating conditions based on the processing of sensor inputs. The technologies described in this document can benefit vehicles of any level, ranging from fully automated vehicle (AV) to human-operated vehicles.

[0051] AVs have advantages over vehicles that require human drivers. One advantage is safety. For example, in 2016, the U.S. experienced 6 million car accidents, 2.4 million injuries, 40,000 deaths, and 13 million vehicle collisions, with an estimated social cost of over $910 billion. From 1965 to 2015, the number of traffic fatalities per 100 million miles driven in the U.S. decreased from about 6 to about 1, partly due to additional safety features deployed in vehicles. For example, an extra half-second of warning associated with an impending collision is believed to mitigate 60% of front and rear collisions. However, passive safety features (such as seat belts and airbags) may have reached their limits in improving these figures. Therefore, active safety measures, such as automated vehicle controls, are a possible next step in improving these statistics. Since human drivers are considered to be responsible for serious pre-collision events in 95% of collisions, autonomous driving systems could potentially achieve better safety outcomes by: identifying and avoiding emergencies more reliably than humans; making better decisions, obeying traffic regulations better than humans, and predicting future events better than humans; and controlling vehicles more reliably than humans.

[0052] refer to Figure 1The AV system 120 enables the AV 100 to operate along a trajectory 198, traversing the environment 190 to the destination 199 (sometimes referred to as the final location), while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists and other obstacles) and complying with road rules (e.g., operating rules or driving preferences).

[0053] In an embodiment, the AV system 120 includes means 101 for receiving and operating operation commands from and on a computer processor 146. The term "operation command" is used to refer to executable instructions (or a set of instructions) that cause a vehicle to perform actions (e.g., driving maneuvers). Operation commands may, without limitation, include instructions for causing the vehicle to begin moving forward, stop moving forward, begin moving backward, stop moving backward, accelerate, decelerate, make a left turn, and make a right turn. In an embodiment, the computer processor 146 is referenced below. Figure 2 The processor 204 described is similar. Examples of the device 101 include a steering controller 102, a brake 103, a gear, an accelerator pedal or other acceleration control mechanism, a windshield wiper, a side door lock, a window controller, and a turn indicator.

[0054] In an embodiment, the AV system 120 includes sensors 121 for measuring or inferring attributes of the state or condition of the AV 100, such as the AV's position, linear velocity and angular velocity, linear acceleration and angular acceleration, and heading (e.g., the orientation of the front of the AV 100). Examples of sensors 121 are GPS, inertial measurement units (IMUs) that measure both linear acceleration and angular rate of the vehicle, wheel rate sensors for measuring or estimating wheel slip ratio, wheel braking pressure or braking torque sensors, engine torque or wheel torque sensors, and steering angle and angular rate sensors.

[0055] In an embodiment, sensor 121 also includes sensors for sensing or measuring properties of the AV's environment. Examples include a monocular or stereo camera 122 with visible, infrared, or thermal (or both) spectra, a LiDAR 123, a RADAR, an ultrasonic sensor, a time-of-flight (TOF) depth sensor, a rate sensor, a temperature sensor, a humidity sensor, and a precipitation sensor.

[0056] In one embodiment, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with a computer processor 146 or data collected by the sensor 121. In another embodiment, the data storage unit 142 is associated with the following... Figure 2The described ROM 208 or storage device 210 is similar. In this embodiment, memory 144 is similar to main memory 206 described below. In this embodiment, data storage unit 142 and memory 144 store historical, real-time, and / or predictive information about environment 190. In this embodiment, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In this embodiment, data related to environment 190 is transmitted from remote database 134 to AV 100 via a communication channel.

[0057] In an embodiment, AV system 120 includes communication devices 140 for transmitting measured or inferred attributes of the state and conditions of other vehicles, such as position, linear velocity and angular velocity, linear acceleration and angular acceleration, and linear heading and angular heading, to AV 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices, as well as devices for wireless communication via point-to-point or ad hoc networks, or both. In an embodiment, communication device 140 communicates across the electromagnetic spectrum (including radio and optical communications) or other media (e.g., air and acoustic media). Combinations of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communication (and in some embodiments, one or more other types of communication) are sometimes referred to as vehicle-to-all-things (V2X) communication. V2X communication typically conforms to one or more communication standards for communication with and between AVs.

[0058] In one embodiment, the communication device 140 includes a communication interface. This may be a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near-field, infrared, or radio interface. The communication interface transmits data from a remote database 134 to the AV system 120. In one embodiment, the remote database 134 is embedded in a cloud computing environment. The communication device 140 transmits data collected from the sensor 121 or other data related to the operation of the AV 100 to the remote database 134. In another embodiment, the communication device 140 transmits information related to teleoperation to the AV 100. In some embodiments, the AV 100 communicates with other remote (e.g., "cloud") servers 136.

[0059] In this embodiment, the remote database 134 also stores and transmits digital data (e.g., data such as road and street locations). This data is stored in memory 144 on the AV 100 or transmitted from the remote database 134 to the AV 100 via a communication channel.

[0060] In one embodiment, the remote database 134 stores and transmits historical information (e.g., rate and acceleration distribution) related to the driving attributes of vehicles that previously traveled along trajectory 198 at similar times of day. In one implementation, such data can be stored in memory 144 on the AV 100 or transmitted from the remote database 134 to the AV 100 via a communication channel.

[0061] The computer processor 146 located on AV 100 generates control actions in an algorithmic manner based on both real-time sensor data and prior information, allowing AV system 120 to perform its autonomous driving capabilities.

[0062] In one embodiment, the AV system 120 includes a computer peripheral device 132 coupled to a computer processor 146 for providing information and alerts to a user of the AV 100 (e.g., an occupant or a remote user) and receiving input from that user. In another embodiment, the peripheral device 132 is similar to the one described in the following reference. Figure 2 The discussed display 212, input device 214, and cursor controller 216 are coupled wirelessly or wiredly. Any two or more interface devices can be integrated into a single device.

[0063] In one embodiment, the AV system 120 receives and enforces a privacy level for an occupant, such as one specified by the occupant or stored in a profile associated with the occupant. The occupant's privacy level determines how access is permitted to specific occupant-related information (e.g., occupant comfort data, biometric data, etc.) stored in the occupant profile and / or stored on cloud server 136 and associated with the occupant profile. In one embodiment, the privacy level specifies specific occupant-related information that is deleted once the ride is complete. In another embodiment, the privacy level specifies specific occupant-related information and identifies one or more entities authorized to access that information. Examples of the specified entities authorized to access the information may include other AV systems, third-party AV systems, or any entity that could potentially access the information.

[0064] An occupant's privacy level can be specified at one or more granular levels. In one embodiment, the privacy level identifies specific information to be stored or shared. In another embodiment, the privacy level applies to all information associated with the occupant, allowing the occupant to specify that her personal information should not be stored or shared. The designation of entities authorized to access specific information can also be specified at various granular levels. The various sets of entities authorized to access specific information may include, for example, other AVs, cloud server 136, specific third-party AV systems, etc.

[0065] In an embodiment, AV system 120 or cloud server 136 determines whether AV 100 or another entity can access certain information associated with an occupant. For example, a third-party AV system attempting to access occupant input related to a specific time and place must, for example, obtain authorization from AV system 120 or cloud server 136 to access occupant-related information. For example, AV system 120 uses a specified privacy level for the occupant to determine whether location- and time-related occupant input can be presented to a third-party AV system, AV 100, or another AV. This allows the occupant's privacy level to specify which other entities are allowed to receive data related to the occupant's actions or other data associated with the occupant.

[0066] Figure 2 Example: Computer system 200. In implementation, computer system 200 is a dedicated computing device. The dedicated computing device is hardwired to perform these technologies, or includes a digital electronic device such as one or more ASICs or field-programmable gate arrays (FPGAs) persistently programmed to perform the aforementioned technologies, or is capable of including one or more general-purpose hardware processors programmed to perform these technologies according to program instructions in firmware, memory, other memory, or a combination thereof. Such a dedicated computing device can also combine custom hardwired logic, ASICs, or FPGAs with custom programming to accomplish these technologies. In various embodiments, the dedicated computing device is a desktop computer system, a portable computer system, a handheld device, a network device, or any other device that includes hardwired and / or program logic to implement these technologies.

[0067] In an embodiment, computer system 200 includes a bus 202 or other communication mechanism for conveying information, and a processor 204 coupled to the bus 202 to process information. Processor 204 is, for example, a general-purpose microprocessor. Computer system 200 also includes main memory 206, such as RAM or other dynamic storage, coupled to the bus 202 to store information and instructions executed by processor 204. In one implementation, main memory 206 is used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 204. When these instructions are stored in a non-transitory storage medium accessible to processor 204, computer system 200 becomes a dedicated machine customized to perform the operations specified in the instructions.

[0068] In an embodiment, the computer system 200 further includes a read-only memory (ROM) 208 or other static storage device coupled to the bus 202 for storing static information and instructions of the processor 204. A storage device 210, such as a disk, optical disk, solid-state drive, or three-dimensional cross-point memory, is provided and coupled to the bus 202 to store information and instructions.

[0069] In this embodiment, computer system 200 is coupled via bus 202 to display 212, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, or organic light-emitting diode (OLED) display for displaying information to a computer user. Input device 214, including alphanumeric keys and other keys, is coupled to bus 202 for transmitting information and command selections to processor 204. Another type of user input device is a cursor controller 216, such as a mouse, trackball, touchscreen, or cursor arrow keys, for transmitting directional information and command selections to processor 204 and for controlling cursor movement on display 212. Such input devices typically have two degrees of freedom on two axes (a first axis (e.g., x-axis) and a second axis (e.g., y-axis)), allowing the device to specify a position in a plane.

[0070] According to one embodiment, the techniques described herein are executed by computer system 200 in response to processor 204 executing one or more sequences of one or more instructions contained in main memory 206. These instructions are read into main memory 206 from another storage medium, such as storage device 210. Executing the sequence of instructions contained in main memory 206 causes processor 204 to perform the process steps described herein. In alternative embodiments, hardwired circuitry is used instead of or in combination with software instructions.

[0071] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media include non-volatile and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, solid-state drives, or three-dimensional cross-point memory such as storage device 210. Volatile media include dynamic memory, such as main memory 206. Common forms of storage media include, for example, floppy disks, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with perforations, RAM, PROMs and EPROMs, FLASH-EPROMs, NV-RAMs, or any other memory chips or memory cartridges.

[0072] Storage media differ from transmission media, but can be used in conjunction with them. Transmission media participate in the information transmission between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, which include wires with a bus 202. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0073] In embodiments, various forms of media involve carrying one or more sequences of one or more instructions to processor 204 for execution. For example, these instructions may initially be executed on a disk or solid-state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and transmits them over a telephone line using a modem. A local modem of computer system 200 receives data over the telephone line and converts the data into an infrared signal using an infrared transmitter. An infrared detector receives the data carried in the infrared signal, and appropriate circuitry places the data on bus 202. Bus 202 carries the data to main memory 206, from which processor 204 retrieves and executes the instructions. Instructions received in main memory 206 may optionally be stored on storage device 210 before or after execution by processor 204.

[0074] Computer system 200 also includes a communication interface 218 coupled to bus 202. Communication interface 218 provides bidirectional data communication coupled to network link 220 connected to local network 222. For example, communication interface 218 is an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem used to provide data communication connectivity with a corresponding type of telephone line. As another example, communication interface 218 is a Local Area Network (LAN) card used to provide data communication connectivity with a compatible LAN. In some implementations, a wireless link is also implemented. In any such implementation, communication interface 218 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0075] Network link 220 typically provides data communication to other data devices via one or more networks. For example, network link 220 provides connectivity to host computer 224 or to a cloud data center or device operated by Internet Service Provider (ISP) 226 via local network 222. ISP 226, in turn, provides data communication services via a worldwide packet data communication network now commonly referred to as the “Internet” 228. Both local network 222 and Internet 228 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks, as well as signals on network link 220 and through communication interface 218, are example forms of transmission media carrying digital data entering and leaving computer system 200. In embodiments, network link 220 includes a cloud or a portion of a cloud.

[0076] Computer system 200 sends messages and receives data including program code through one or more networks, network links 220, and communication interfaces 218. In an embodiment, computer system 200 receives code for processing. The received code is executed by processor 204 upon receipt and / or stored in storage device 210, or in other non-volatile storage devices for later execution.

[0077] AV architecture

[0078] Figure 3 Showing for AV (e.g., Figure 1 The example architecture 300 of the AV 100 shown is illustrated. Architecture 300 includes sensing circuitry 302 (sometimes called a sensing module), planning circuitry 304 (sometimes called a planning module), control circuitry 306 (sometimes called a control module), positioning circuitry 308 (sometimes called a positioning module), and database circuitry 310 (sometimes called a database module). Each circuit plays a role in the operation of the AV 100. Commonly, circuits 302, 304, 306, 308, and 310 can be... Figure 1 This is part of the AV system 120 shown. In some embodiments, even if the version element is described as "circuit," any circuit in circuits 302, 304, 306, 308, and 310 is a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., one or more microprocessors, microcontrollers, application-specific integrated circuits (ASICs), hardware memory devices, other types of integrated circuits, other types of computer hardware, or any or all combinations of these hardware). Circuits 302, 304, 306, 308, and 310 are each sometimes referred to as processing circuitry or processing modules (e.g., computer hardware, computer software, or a combination of both). Any or all combinations of circuits 302, 304, 306, 308, and 310 are also examples of processing circuitry or processing modules.

[0079] In use, the planning circuit 304 receives data representing the destination 312 and determines data representing the trajectory 314 (sometimes called the route) that the AV 100 can travel to reach (e.g., arrive at) the destination 312. In order for the planning circuit 304 to determine the data representing the trajectory 314, the planning circuit 304 receives data from the sensing circuit 302, the positioning circuit 308, and the database circuit 310.

[0080] The sensing circuit 302 is used, for example, as follows Figure 1 One or more sensors 121 are shown to identify nearby physical objects. The objects are classified (e.g., grouped into types such as pedestrians, bicycles, cars, traffic signs, etc.), and a scene description including the classified objects 316 is provided to the planning circuit 304.

[0081] The planning circuit 304 also receives data representing the AV location 318 from the positioning circuit 308. The positioning circuit 308 uses data from sensor 121 and data from database circuit 310 (e.g., geographic data) to calculate the location and orientation (e.g., the AV's attitude) of the AV. For example, the positioning circuit 308 uses data from GNSS (Global Navigation Satellite System) sensors and geographic data to calculate the longitude and latitude of the AV. In embodiments, the data used by the positioning circuit 308 includes high-precision maps with lane geometry properties, maps describing road network connectivity properties, maps describing lane physical properties (such as traffic speed, traffic volume, number of vehicle and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or combinations thereof), and maps describing the spatial locations of road features (such as intersections, traffic signs, or various types of other traffic signals). In embodiments, the high-precision map is constructed by adding data to a low-precision map via automatic or manual annotation.

[0082] Control circuit 306 receives data representing trajectory 314 and data representing AV position 318, and operates AV control functions 320a-320c (e.g., steering, throttle, braking, ignition) in a manner that will cause AV 100 to travel along trajectory 314 to reach destination 312. For example, if trajectory 314 includes a left turn, control circuit 306 will operate control functions 320a-320c in such a way that the steering angle of the steering function will cause AV 100 to turn left, and the throttle and brake will cause AV 100 to pause before turning and wait for passing pedestrians or vehicles.

[0083] AV input

[0084] Figure 4 The sensing circuit 302 is shown. Figure 3 The inputs used are 402a-402d (e.g., Figure 1 Examples of sensor 121 and outputs 404a-404d (e.g., sensor data) are shown. One input 402a is a LiDAR system (e.g., Figure 1 The LiDAR system shown is 123. LiDAR is a technique that uses light (e.g., a beam of light such as infrared light) to obtain data related to physical objects in its line of sight. The LiDAR system produces LiDAR data as output 404a. For example, LiDAR data is a collection of 3D or 2D points (also called point clouds) used to construct a representation of environment 190.

[0085] Another input 402b is a RADAR system. RADAR is a technique that uses radio waves to acquire data related to nearby physical objects. RADAR can acquire data related to objects that are not within the line of sight of a LiDAR system. The RADAR system generates RADAR data as output 404b. For example, RADAR data is one or more radio frequency electromagnetic signals used to construct a representation of environment 190.

[0086] Another input 402c is a camera system. The camera system uses one or more cameras (e.g., a digital camera using a light sensor such as a charge-coupled device [CCD]) to acquire information about nearby physical objects. The camera system produces camera data as output 404c. Camera data is typically in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). In some examples, the camera system has multiple independent cameras, for example, for stereoscopic imaging (stereoscopic vision), which enables the camera system to perceive depth. Although the objects perceived by the camera system are described here as “nearby,” this is relative to the AV (view of objects). In some embodiments, the camera system is configured to “see” distant objects (e.g., objects as far as 1 kilometer or more in front of the AV). Therefore, in some embodiments, the camera system has features such as sensors and lenses optimized for perceiving distant objects.

[0087] Another input 402d is the Traffic Light Detection (TLD) system. The TLD system uses one or more cameras to acquire information related to traffic lights, street signs, and other physical objects that provide visual navigation information. The TLD system produces TLD data as output 404d. TLD data is often in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). The difference between a TLD system and a system that includes cameras is that the TLD system uses cameras with a wide field of view (e.g., using a wide-angle lens or fisheye lens) to acquire information related to as many physical objects as possible that provide visual navigation information, allowing the AV 100 to access all relevant navigation information provided by these objects. For example, the TLD system has a field of view of approximately 120 degrees or greater.

[0088] In some embodiments, sensor fusion technology is used to combine outputs 404a-404d. Thus, individual outputs 404a-404d are provided to other systems of AV 100 (e.g., provided to systems such as...). Figure 3The planned circuit 304 shown may be used to provide combined outputs to other systems, or may take the form of a single or multiple combined outputs of the same type (e.g., using the same combination technique or combining the same outputs or both) or single or multiple combined outputs of different types (e.g., using different individual combination techniques or combining different individual outputs or both). In some embodiments, an early fusion technique is used. The early fusion technique is characterized by combining the outputs before applying one or more data processing steps to the combined outputs. In some embodiments, a late fusion technique is used. The late fusion technique is characterized by combining the outputs after applying one or more data processing steps to the individual outputs.

[0089] Path planning

[0090] Figure 5 Show (for example, as) Figure 3 The diagram 500 illustrates the relationship between the inputs and outputs of the planning circuit 304. Generally, the output of the planning circuit 304 is a route 502 from a starting point 504 (e.g., a source location or initial location) to an ending point 506 (e.g., a destination or final location). Route 502 is typically defined by one or more road segments. For example, a road segment refers to a distance to be traveled on at least a portion of a street, road, highway, driveway, or other physical area suitable for vehicle travel. In some examples, such as if AV 100 is an off-road capable vehicle such as a four-wheel drive (4WD) or all-wheel drive (AWD) car, SUV, or pickup truck, route 502 includes “off-road” segments such as unpaved paths or open fields.

[0091] In addition to route 502, the planning circuit also outputs lane-level route planning data 508. Lane-level route planning data 508 is used to navigate through segments of route 502 at specific times based on conditions. For example, if route 502 comprises a multi-lane highway, lane-level route planning data 508 includes trajectory planning data 510, which AV 100 can use to select a lane from the multiple lanes based on factors such as whether an exit is nearby, whether other vehicles are present in one or more lanes, or other factors that change over a period of minutes or less. Similarly, in some implementations, lane-level route planning data 508 includes a speed constraint 512 specific to a segment of route 502. For example, if the segment includes pedestrians or unexpected traffic, speed constraint 512 limits AV 100 to a slower speed than expected, such as a speed based on the speed limit data for that segment.

[0092] In an embodiment, the input to the planning circuit 304 includes (e.g., from...) Figure 3The database circuit 310 shown contains database data 514 and current location data 516 (for example, Figure 3 The AV position shown is 318), (for example, for use with Figure 3 The destination data 518 and object data 520 shown for destination 312 (e.g., as shown) Figure 3 The sensing circuit 302 shown perceives classified objects 316. In some embodiments, database data 514 includes rules used during planning. Rules are specified using a formal language (e.g., Boolean logic). In any given situation encountered by AV 100, at least some of these rules will apply to that situation. A rule applies to a given situation if it has conditions satisfied based on information available to AV 100 (e.g., information related to the surrounding environment). Rules can have priorities. For example, a rule "move to the leftmost lane if the road is a highway" can have a lower priority than "move to the rightmost lane if the exit is within one mile."

[0093] Lane detection and positioning

[0094] Such as about Figure 3 and Figure 1 As shown, the positioning circuit 308 calculates the location of the vehicle (e.g., AV) using data from sensor 121 and data from database circuit 310 (e.g., geographic data) to determine the location of the vehicle. Figure 1 The location of AV 100 is determined by the sensor 121. However, in some cases, one or more fields of view of the sensor 121 may be completely or partially obscured by objects such as another vehicle, road features such as dividers, or other environmental features such as vegetation. In such cases, the sensor 121 cannot receive data that is sufficient, either quantitatively or qualitatively, for the AV system 120 to locate AV 100. For example, if AV 100 is on a highway, the sensor 121 will only detect obstacles or lanes adjacent to the highway. In many cases, this structure is uniform, so the sensor does not acquire unique structures for locating AV 100. As another example, AV 100 may be surrounded by other vehicles, which would impair the sensor's ability to detect structures near the location of the vehicles.

[0095] The embodiments described herein provide techniques for positioning based on vehicles in the vicinity of AV 100. Figures 6 to 8An example technique is illustrated graphically, which identifies lanes based on the attitude of other vehicles in the vicinity of AV 100. The lanes are then compared to map data to identify a match. Based on this match, localization is performed. As previously stated, and as used herein, the term "attitude" refers to both the position and orientation of a vehicle such as AV 100 or vehicles in the vicinity of AV 100.

[0096] Figure 6 Examples of lane detection based on other vehicles according to various embodiments are shown. Specifically, multiple vehicles are shown at Figure 605. The vehicles include vehicle 615, which is similar to or shares one or more features with AV 100. The vehicles at Figure 605 also include multiple other vehicles 610 in the vicinity of vehicle 615. As shown in Figure 605, vehicles 610 and 615 have a lane detection capability along... Figure 6 The length measured along the Y-axis. Vehicles 610 and 615 also have this feature. Figure 6 The width measured along the X-axis.

[0097] Those skilled in the art will readily understand that Figure 605 is a simplified illustration of a total of twelve vehicles with uniform measurements arranged in a clear grid pattern. This simplification is shown for the convenience of the description of this application. It will be appreciated that real-world embodiments will include variations such as more or fewer vehicles, vehicles offset from each other, vehicles with different widths or lengths, or other modifications. Figure 6 , Figure 7 and Figure 8 Other aspects or descriptions may be further considered for similar variations.

[0098] The vehicle 610 is equipped with at least one sensor (such as...) Figure 1 (e.g., one of the sensors in sensor 121). For example, vehicle 615 uses sensors such as LiDAR system 402a, RADAR system 402b, camera system 402c, etc., to identify. Figure 6 At least one of the vehicle vehicles 610 has an attitude-related aspect.

[0099] Generally, it will be recognized that most vehicles on a road have a length greater than their width, and that vehicles travel in a direction parallel to their length. Therefore, as shown in Figure 625, lane 620 is estimated based on vehicle 610. Specifically, vehicle 615 (or more specifically, sensing circuitry such as sensing circuitry 302 of vehicle 615) identifies the attitude of vehicle 610. As described above, the “attitude” of vehicle 610 involves both its location and orientation.

[0100] Based on the location and orientation of vehicle 610, vehicle 615 can interpret the location of lane 620. For example, based on the orientation of one of the vehicles 610, vehicle 615 will be able to identify the direction of travel of vehicle 610. Specifically, vehicle 615 (or a planning circuit of the vehicle, such as planning circuit 304) will identify that vehicle 610 is traveling in its oriented direction. Vehicle 615 will also use the premise that vehicle 610 complies with applicable traffic regulations and is located within the existing lane of vehicle 610 and the lane in which vehicle 615 is traveling as a prerequisite for this analysis. Thus, based on the attitude of vehicle 610, vehicle 615 can estimate the existence of lane 620. In embodiments, the estimation of lane 620 will be based on techniques such as piecewise linear regression, etc.

[0101] In this embodiment, lane 620 is estimated based on deep learning used to initially detect lane 620, for example, by mapping the original image onto a lane mask. Lane parameter estimation is then performed, where the lane curve can be matched to the pixel-level detection. Finally, the lane curve is “smoothed” using various tracking techniques. However, it will be appreciated that this is an advanced example, and other techniques exist additionally or alternatively in other embodiments.

[0102] In another embodiment, vehicle 615 will determine the presence of lane 620 based on the travel direction of vehicle 615 without needing to analyze the travel direction of vehicle 610. As an example of this embodiment, since sensing circuit 302 or planning circuit 304 is not configured to determine the orientation of vehicle 610, this orientation will be uncertain. As an alternative example, sensing circuit 304 (more specifically, a sensor communicatively coupled to sensing circuit 304) will receive enough data to identify the presence of vehicle 610, but will not be able to distinguish whether the identified portion of vehicle 610 is the front end, rear end, or side view of vehicle 610, etc. In this embodiment, vehicle 615 will identify its travel direction and orientation, and then determine an assumed travel direction of vehicle 610. For example, vehicle 615 will assume that vehicle 610 is traveling in approximately the same direction as vehicle 610. Based on this extrapolation method, the vehicle 615 assumes that the vehicle 610 complies with applicable traffic regulations related to the direction of travel, and identifies the lane 620 based on this assumption.

[0103] It should be understood that these embodiments are described as exemplary embodiments of how vehicle 615 is configured to recognize the presence of lane 620. Other embodiments will include additional or alternative technologies or combinations of the technologies described above. An example of an additional technology is determining the attitude of vehicles of different shapes or sizes. For example, in one embodiment, sensing module 302 is configured to recognize the attitude of vehicles of different shapes or sizes, such as cars, compact cars, trucks, vans, commercial freight transport vehicles, etc.

[0104] We will understand, although Figure 6 An estimation of lane 620 based on the presence of eleven vehicles 610 is described, but in another embodiment, lane 620 will be estimated based on only two vehicles. Specifically, vehicle 615 will be estimated based on the attitude of vehicle 610 in the leftmost lane 620 and the rightmost lane 620 (as shown in...). Figure 6 The vehicle 615 estimates lane 620 based on the orientation of the vehicles listed in Figure 625. By identifying the orientation of these vehicles, the leftmost lane 620 and the rightmost lane 620 are identified, and then the existence of the center lane 620 is inferred.

[0105] In other embodiments, the estimation of lane 620 at Figure 625 is based on one or more additional factors. For example, in one embodiment, the estimation of lane 620 at Figure 625 is based on factors such as lane lines, other road markings, obstacles or barriers on the side of the road, deceleration markings, etc.

[0106] It will also be understood that although lane 620 is depicted as generally straight, this illustration is a simplified discussion for the purposes of discussing the subject matter of this application. In other embodiments, lane 620 will have variations such as curvature, ramps, merging lanes, or exiting lanes.

[0107] In another embodiment, the estimation of lane 620 at illustrated 625 will be based on a metric related to the estimation quality of the attitude of one or more vehicles 610. For example, one example of an estimation quality metric is a deterministic factor applied to the estimation of the attitude of each vehicle 610. As used herein, a “deterministic factor” refers to one or more numerical factors, values, or functions applied to information relating to one or more of the vehicles 610, providing an output compared to a threshold. For example, a deterministic factor might be based on the type of information relating to a given vehicle 610 that vehicle 615 can identify, the distance between one of the vehicles 610 and vehicle 615, the variance of the position or orientation of vehicle 610 relative to the position or orientation of the other vehicles 610, the variance of the position or orientation of vehicle 610 relative to vehicle 615, etc.

[0108] In embodiments, deterministic factors are reduced or otherwise influenced by the attitude, shape, or size of the vehicle 610, which is not easily identifiable by the vehicle 615, as analyzed by the vehicle 615 itself or in the context of other vehicles within the vehicle 610. For example, if the vehicle 610 is changing lanes, its attitude or location will deviate from that of other vehicles 610 driving within the lane. As another example, larger vehicles, such as semi-trailers or other large vehicles, will present a different profile to the sensors and therefore behave differently from other vehicles within the vehicle 610. In these cases, deterministic factors associated with a vehicle changing lanes or behaving differently from other vehicles are reduced, causing data associated with that vehicle to be processed or discarded differently.

[0109] If the deterministic factor associated with one of the vehicles 610 is below (or equal to or lower than) a threshold, that vehicle 610 is excluded from the estimation of lane 620 and weighted to have less impact on the estimation of lane 620, etc. In some embodiments, the threshold is a pre-identified threshold (e.g., one standard deviation), while in other embodiments, the threshold is dynamic (e.g., the three vehicles 610 with the lowest deterministic factors).

[0110] Based on the identified lane 620, and as shown in Figure 630, the sensing circuitry 302 of vehicle 615, or more specifically vehicle 615, will then identify its location and orientation (e.g., its attitude) within the identified lane 620. In another embodiment, vehicle 615 is not located in the central lane 620, but rather in one of the leftmost or rightmost lanes 620. The attitude of vehicle 615 will be identified based on, for example, the attitudes of other vehicles 610 in the vicinity of vehicle 615.

[0111] Then, information related to the attitude of the vehicle 615, such as that identified at point 630, is provided to the positioning circuitry of the vehicle 615 (e.g., Figure 3 (e.g., positioning circuit 308). Then, the positioning circuit compares the identified attitude of the vehicle 615 and information related to the lane 620 with map data to identify the location of the vehicle 615.

[0112] Figure 7 A simplified top-down example of map data 700 according to various embodiments is shown. Specifically, map data 700 depicts information such as the shape or layout of multiple lanes 705. In embodiments, this information is obtained, for example, from positioning circuitry such as... Figure 3 Map data 700 is retrieved from a database such as database 310. In another embodiment, map data 700 is retrieved from a database or repository outside the vehicle 615 (i.e., from the cloud).

[0113] In one embodiment, the identification of map data by the vehicle 615 is based on the most recently known location of the vehicle 615. For example, if the vehicle 615 is previously known to be at a certain location, the map data 700 will include the area located in the vicinity of that most recently known location. In one embodiment, the identification of map data is based on the most recently known attitude of the vehicle 615. For example, if the vehicle has an attitude, that attitude will indicate the trajectory of the vehicle, such that the map data 700 is based on the indicated trajectory. In another embodiment, the map data 700 is additionally or alternatively based on one or more factors, such as the speed of the vehicle 615, the previously identified trajectory of the vehicle 615, predictive analysis of the driving habits of the driver of the vehicle 615, the known destination of the vehicle 615, or some other factor. It will be understood that these factors are intended as examples, and in other embodiments, additionally or alternatively, the map data 700 will be based on other factors not explicitly stated herein.

[0114] Figure 8 The following are illustrated based on various embodiments. Figure 6 Lane detection and Figure 7An example of positioning 800 using map data. Specifically, the positioning circuitry of vehicle 615 attempts to... Figure 6 The lane 620 identified in the diagram 630 is... Figure 7 The identified map data 700 is matched. In one embodiment, the positioning circuitry of the vehicle 615 matches the identified lane 620 with the map data 700 using a convolution algorithm. In another embodiment, the matching is performed based on regression analysis. In other embodiments, other techniques or algorithms are additionally or alternatively used.

[0115] like Figure 8 As shown, Figure 6 The identified lane 620 corresponds to a portion 810 of lane 705. Based on this correspondence between lane 620 and lane 705 at this portion 810, the vehicle 615 (specifically, the positioning circuitry of the vehicle 615) can identify the vehicle's location relative to map data 700. Furthermore, based on the orientation of the vehicle 615 identified in Figure 630, the vehicle 615 can identify its orientation relative to map data 700. This allows the vehicle to be positioned even with few or no road signs, or when such road signs are obscured (e.g., identification of the vehicle's orientation relative to known map data), which the vehicle 615 will typically use for positioning.

[0116] Figure 9 Examples of positioning technologies according to various embodiments are shown. These technologies include those described above. Figure 6 , Figure 7 and Figure 8 The elements described are similar to those of a vehicle, and are performed by elements such as sensing circuit 302, positioning circuit 308, etc. More generally, in this embodiment, the technique is performed by processor 204.

[0117] The technology includes identifying surrounding vehicles, such as vehicle 610, at location 905. As described above, the surrounding vehicles are identified by sensors such as LiDAR system 402a, RADAR system 402b, camera 402c, etc.

[0118] As mentioned above, for example Figure 6 As illustrated in Figure 625, the technique further includes estimating a lane, such as lane 620, at 910. Specifically, the lane is estimated based on the attitude of the surrounding vehicle 610, the attitude of the vehicle 615, or both. The lane is estimated based on estimation techniques such as regression or some other techniques.

[0119] The technology also includes: identifying estimated map data such as map data 700 at point 915. (See also: regarding...) Figure 7As shown, in one embodiment, map data is identified based on previously known locations of the vehicle. In another embodiment, map data is identified based on one or more factors such as the vehicle's known heading or trajectory, the driver's known driving habits, the vehicle's known destination, etc. In one embodiment, map data is identified by circuitry or logic on the vehicle, while in another embodiment, map data is identified at least partially by a neural network located at least partially outside the vehicle (e.g., in the cloud) or other logic.

[0120] The technology also includes matching the lane identified from feature 910 at point 920 with the map data identified at point 915. (As mentioned above...) Figure 8 In this embodiment, the matching is performed based on analysis such as regression analysis, convolutional analysis, or some other techniques.

[0121] The technology also includes identifying the attitude of the main vehicle relative to map data at point 925. (As mentioned above...) Figure 8 The attitude of the main vehicle is identified at 925 based on the matching of map data from element 915 with the lane estimated at 910. The attitude of the main vehicle identified at 925 is further based on information such as the attitude of the identified main vehicle relative to the lane estimated from element 910.

[0122] In one embodiment, the technique then proceeds from element 925 to operate the main vehicle based on attitude at 940. For example, the vehicle's attitude or positioning process is typically used by circuitry such as planning circuitry 304 or control circuitry 306 (or both) to identify the trajectory or specific controls to be used by the vehicle for travel.

[0123] In another embodiment, the positioning process described with respect to elements 905 / 910 / 915 / 920 / 925 is one of multiple positioning processes that a vehicle can perform. In one embodiment, it is desirable to verify the accuracy of the vehicle attitude identified at 925. For example, in an embodiment, the vehicle can identify one or more metrics related to the determinism of lane estimation at 920, map data identified at 915, and matches identified at 920. Therefore, optionally, the technique includes elements 930, 935, and 945.

[0124] Specifically, in this embodiment, the technique includes identifying whether the attitude accuracy is acceptable at 945. The identification at 945 is based on one or more factors, such as deterministic metrics associated with elements 915, 920, or 925. In another embodiment, the identification at 945 is additionally or alternatively based on one or more other deterministic related metrics. The identification at 945 involves comparing one or more deterministic metrics with pre-identified thresholds (e.g., whether the determinism is within one standard deviation). In another embodiment, the identification at 945 is based on dynamic thresholds, which are based on factors such as the number of cars identified at 905, the type of sensor used to identify vehicles at 905, the number of lanes, or the technology used to identify lanes at 910. In these techniques, if, for example, one or more deterministic metrics are higher than (or equal to or greater than) one or more of the thresholds, the attitude accuracy at 945 is identified as acceptable. Other criteria are used in other embodiments.

[0125] If the attitude accuracy at 945 is deemed acceptable, the technique proceeds to the operation of the main vehicle at 940 as described above. However, if the attitude accuracy at 945 is not deemed acceptable, an auxiliary localization process is performed at 930. In one embodiment, the auxiliary localization process includes re-performing elements 905, 910, 915, 920, and 925. In another embodiment, the auxiliary localization process is a localization process such as LiDAR scan matching, visual feature matching, deep learning-based localization, etc.

[0126] Then, at 935, the vehicle's attitude is identified based on the auxiliary positioning process at 930. In one embodiment, the vehicle's attitude at 935 is identified solely based on the positioning process at 930. In another embodiment, the vehicle's attitude at 935 is identified based on a combination of the attitude identified at 925 and the positioning process at 930. For example, in one embodiment, the vehicle's attitude at 935 is identified based on the average of information related to the attitude identified at 925 and the positioning process performed at 930 to generate an average vehicle attitude. In another embodiment, the vehicle's attitude at 935 is identified based on a function such as the median of information related to elements 925 and 935, the mean of information related to elements 925 and 930, etc. Then, at 940, the vehicle is operated based on one or both of the attitudes identified at 925 or 935.

[0127] To be understood Figure 9The above-described technique is intended as an example technique in which the assisted positioning process at 930 results in unacceptable attitude accuracy at 945, and other embodiments will vary. For example, in another embodiment, it is desirable to always perform the assisted positioning process at 930 such that the vehicle's attitude is based on a combination of the attitude identified at 925 and the assisted positioning process at 930.

[0128] Figure 10 Alternative examples of positioning techniques according to various embodiments are shown. Typically, Figure 10 Intended as a response to Figure 9 The technology is complementary, and includes with Figure 9 One or more elements similar to the element. With Figure 9 similar, Figure 10 The technology is performed by components of the vehicle, such as sensing circuit 302 and positioning circuit 308. More generally, in this embodiment, the technology is performed by processor 204.

[0129] The technology includes: identifying sensor data from sensors of a first vehicle at point 1005, wherein the sensor data includes information related to the attitude of at least two other vehicles on the road relative to the first vehicle. Identification at point 1005 is similar to, for example, identification at point 905, or as per [the relevant information]. Figure 6 As depicted in Figure 605, the vehicle 610 is the first vehicle, for example... Figure 6 The transport vehicle 615.

[0130] The technology also includes determining, at least in part, the geometry of a portion of the road at location 1010 based on information relating to the attitudes of at least two other vehicles. For example, this determination involves identifying, such as... Figure 6 The estimated lanes at positions 620, 910, etc.

[0131] The technology also includes: at point 1015, comparing the geometry of a portion of the road with map data (e.g., at...). Figure 7 The map data 700, identified at point 915 or feature 915, is compared to identify that part of the road with a portion of the map data (e.g., Figure 8 The comparison is between parts 810). Figure 8 Or as described in element 920.

[0132] The technology also includes determining the attitude of the first vehicle relative to the map data at least in part based on the matching at point 1020. The attitude determination is related to the above regarding... Figure 9 The determination of the posture described in element 925 is similar to, or as above. Figure 8 The posture shown is determined as described.

[0133] To be understood Figure 9 and Figure 10 The technique described is intended as an example of one embodiment, and other embodiments include one or more variations of the technique described. For example, in another embodiment, certain elements are performed simultaneously, or in an order different from that described. Other embodiments include more or fewer elements than depicted. Other variations exist in other embodiments.

[0134] In addition, it will be noted that, although Figure 6 , Figure 7 and Figure 8 Description and Figure 9 and Figure 10 The technical description herein pertains to lanes, but the term "lane" is used herein as an example embodiment. The concepts herein can be extended to other road geometries in other embodiments. As an example, the comparison of information relating to a vehicle (e.g., vehicle 615) and other vehicles (e.g., vehicle 610) is also applicable to the identification of unique road geometries such as embankments. In this example, the angle of vehicle 610 (or compared to another vehicle in vehicle 610, vehicle 615, or compared to the horizon, etc.) is used to identify the presence of such an embankment. Once the embankment is identified, map data relating to the most recent known location of vehicle 615 is compared to identify the location, orientation, or attitude of vehicle 615 based on the embankment.

[0135] As another example, positioning is based on the presence of roundabouts or curved lanes. Specifically, if the location, orientation, or attitude of vehicle 610 indicates the presence of a curved lane or roundabout, that curved lane or roundabout is compared with map data associated with vehicle 615 to identify the vehicle's location, orientation, or attitude within the map data. Other embodiments use additional or alternative features or road data (e.g., hills, merging lanes, or exit lanes).

[0136] In the preceding description, embodiments of the invention have been described with reference to numerous specific details, which vary from implementation to implementation. Therefore, the specification and drawings should be considered illustrative rather than restrictive. The sole and exclusive indication of the scope of the invention, and what the applicant expects to be the scope of the invention, is the literal and equivalent scope of the claims published from this application in the specific form of the claims, including any subsequent amendments. Any definitions of terms expressly set forth herein for inclusion in such claims should be taken as meaning as such terms are used in the claims. Furthermore, when the term “comprising” is used in the preceding specification or appended claims, what follows that phrase may be an additional step or entity, or a sub-step / sub-entity of a previously stated step or entity.

Claims

1. A computer-implemented method, comprising: Using at least one processor, sensor data from sensors of a first vehicle is identified, wherein the sensor data includes information related to the attitude of at least two other vehicles on the road; Using the at least one processor, determine deterministic factors related to information relating to the attitude of the at least two other vehicles on the road; Using the at least one processor, determine whether the deterministic factor is equal to or higher than a threshold; Based on the determination that the deterministic factor is equal to or higher than the threshold: Using the at least one processor, the geometry of a portion of the road is determined based at least on information relating to the attitude of the at least two other vehicles; Using the at least one processor, the geometry of that portion of the road is compared with map data to identify a match between that portion of the road and a portion of the map data; and The attitude of the first vehicle relative to the map data is determined at least based on the matching.

2. The method according to claim 1, wherein, The sensor is a vision sensor or a distance sensor.

3. The method according to claim 1, wherein, The sensor data includes information relating to the location of the respective of the at least two other vehicles relative to the first vehicle.

4. The method according to claim 1, wherein, The sensor data includes information relating to the orientation of the respective vehicle among the at least two other vehicles relative to the first vehicle.

5. The method according to claim 1, wherein, This section of the road is the roadway.

6. The method according to claim 1, wherein, Determining the attitude of the first vehicle includes determining the position and orientation of the first vehicle relative to the map data, at least based on the matching.

7. One or more non-transitory computer-readable media, comprising instructions that, when executed by one or more processors, cause a vehicle to perform the following operations: Sensor data from sensors of the first vehicle were identified, wherein, The sensor data includes information related to the attitude of at least two other vehicles on the road; Deterministic factors related to information relating to the attitude of the at least two other vehicles on the road; Determine whether the deterministic factor is equal to or higher than the threshold; Based on the determination that the deterministic factor is equal to or higher than the threshold: The geometry of a portion of the road is determined based at least on information relating to the attitude of the at least two other vehicles. The geometry of that section of the road is compared with map data to identify a match between that section of the road and a portion of the map data; as well as The attitude of the first vehicle relative to the map data is determined at least based on the matching.

8. One or more non-transitory computer-readable media according to claim 7, wherein, The instructions are also used to determine the accuracy of the attitude of the first vehicle relative to the map data, at least based on the quality of the match between that portion of the road and that portion of the map data.

9. One or more non-transitory computer-readable media according to claim 7, wherein, The instructions are also used to determine the accuracy of the attitude of the first vehicle relative to the map data, based at least on the number of the at least two other vehicles.

10. One or more non-transitory computer-readable media according to claim 7, wherein, The first vehicle's attitude is a first attitude, and the instruction is further used for: Determine the accuracy of the first attitude of the first vehicle relative to the map data; and In response to the determination that the accuracy of the first attitude does not meet a predefined threshold, a positioning process is performed to determine the second attitude of the first vehicle relative to the map data.

11. One or more non-transitory computer-readable media according to claim 10, wherein, The instruction is also used for: The first attitude and the second attitude are averaged to determine the average attitude of the first vehicle relative to the map data.

12. A vehicle, comprising: sensor; One or more processors; as well as One or more non-transitory computer-readable media, comprising instructions that, when executed by the one or more processors, cause the vehicle to perform the following operations: Identify sensor data from the sensors of the vehicle, wherein the sensor data includes information related to the attitude of at least two other vehicles on the road; Deterministic factors related to information relating to the attitude of the at least two other vehicles on the road; Determine whether the deterministic factor is equal to or higher than the threshold; Based on the determination that the deterministic factor is equal to or higher than the threshold: The geometry of a portion of the road is determined based at least on information relating to the attitude of the at least two other vehicles. The geometry of that section of the road is compared with map data to identify a match between that section of the road and a portion of the map data; and The attitude of the vehicle relative to the map data is determined at least based on the matching.

13. The vehicle according to claim 12, wherein, The instructions are also used to identify the sensor data from the sensors in response to determining that the vehicle is traveling on a road.

14. The vehicle according to claim 12, wherein, The instruction is also used to identify sensor data from the sensor in response to determining that the sensor is occluded.

15. The vehicle according to claim 12, wherein, The instruction is also configured to identify the sensor data from the sensor in response to determining that the accuracy of the vehicle's attitude relative to the map data generated by the positioning process does not meet a predefined threshold.

16. The vehicle according to claim 12, wherein, The instruction is also used for: Determine the previous location of the vehicle; and The geometry of that section of the road is compared with map data of the vehicle's previous location to identify a match between that section of the road and that section of the map data.

17. The vehicle according to claim 12, wherein, The comparison of the geometry of that section of the road with the map data includes convolving the determined geometry of that section of the road with the map data.

18. The vehicle according to claim 12, wherein, The instructions are also used to cause the vehicle to be navigated at least based on the determined attitude.

19. The vehicle according to claim 12, wherein, The instruction is also used for: Determine the orientation of the vehicle relative to the geometry of that section of the road; as well as Determining the attitude of the vehicle relative to the map data based at least on the matching includes mapping the attitude of the vehicle relative to the geometry of that portion of the road to the map data.

20. The vehicle according to claim 12, wherein, The sensor is a vision sensor or a distance sensor.

21. A computer program product comprising instructions that, when executed by one or more processors, implement the method of any one of claims 1 to 6.

Images