Including and using security and confidence information associated with object in autonomous driving map

By using the confidence information in the map database in the autonomous driving system and adjusting the autonomous driving mode and data weight, the shortcomings of the autonomous driving system in utilizing the map data accuracy and confidence information are solved, and decision-making accuracy and safety are improved .

CN118922686APending Publication Date: 2024-11-08QUALCOMM INC
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202380030057.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-06
Filing Date
2023-01-20
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When the autonomous driving system performs autonomous driving functions, it is difficult to effectively utilize the accuracy and confidence information of map data, resulting in limited decision-making accuracy and safety.

Method used

Access the map database through the processor of the vehicle, obtain map data of nearby objects or features and their related confidence information, and use this information when performing autonomous or semi-autonomous driving actions to adjust the autonomous driving mode and data weight to improve decision-making accuracy.

Benefits of technology

It improves the decision-making accuracy and safety of the autonomous driving system in functions such as path planning and object avoidance, ensures that the driving mode is consistent with the confidence information of the map data, and enhances the safety and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118922686A_ABST
    Figure CN118922686A_ABST
Patent Text Reader

Abstract

Various embodiments include methods and systems (100) for an autonomous driving system that uses map data when performing an autonomous driving function. Various embodiments may include accessing map data regarding objects or features in the vicinity of a vehicle (102), accessing confidence information associated with the map data, and using the confidence information when performing an autonomous or semi-autonomous driving action by a processor (205). The confidence information may be stored in the map database (228) or in a separate data structure accessible by the processor (205). A method of generating map security and confidence information may include receiving information about a map object or feature, including a confidence metric in the information; generating security and confidence information about the object or feature using the received confidence metric; and storing the security and confidence information for access to the vehicle (102) autonomous and semi-autonomous driving systems.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Related Applications

[0002] This application claims the benefit of priority to U.S. non-provisional application No. 17 / 658,126, filed on April 6, 2022; the entire contents of which are incorporated herein by reference. Background Art

[0003] Advanced driver assistance systems (ADAS) and autonomous driving systems (ADS) may use digital maps as part of various operations, including route planning, navigation, collision and obstacle avoidance, and managing interactions with the driver. However, while an autonomous vehicle may receive information, perform path planning, and make maneuvering decisions based on sensor and map data, an ADS may only be informed of the accuracy, precision, and other confidence information regarding the vehicle's sensor data, and therefore may not be able to consider the accuracy, precision, and similar confidence information regarding the map data. Summary of the invention

[0004] Various aspects include methods for including and using safety and / or confidence information about object and feature map data in autonomous and semi-autonomous driving operations. Various aspects may include a method performed by a processor of an autonomous driving system of a vehicle for using map data when performing autonomous driving functions, the method comprising: accessing map data about an object or feature in the vicinity of the vehicle from a map database accessible by the processor; accessing, by the processor, confidence information associated with the map data about the object or feature in the vicinity of the vehicle; and using, by the processor, the confidence information when performing autonomous or semi-autonomous driving actions.

[0005] In some aspects, the confidence information may include one or more of the following: an Automotive Safety Integrity Level (ASIL) autonomy level near the object or feature, an indication related to the accuracy of the map data regarding the object or feature, an indication related to the reliability of the map data regarding the object or feature, a statistical score indicating the precision of the map data regarding the object or feature, or the age or freshness of the map data regarding the object or feature.

[0006] In some aspects, accessing the confidence information associated with the map data regarding the object or feature in the vicinity of the vehicle may include obtaining, by the processor, the confidence information from the map database, wherein the information in the map database is obtained from one or more of a system memory, a remote computing device, or another vehicle. In some aspects, accessing the confidence information associated with the map data regarding the object or feature in the vicinity of the vehicle includes obtaining the confidence information from a data structure other than the map database accessible by the processor based on the location of the object or feature.

[0007] In some aspects, using the confidence information when performing an autonomous or semi-autonomous driving action by the vehicle may include: applying, by the processor, a weight to the accessed map data about the object or feature based on the confidence information; and using, by the processor, the weighted map data about the object or feature when performing a path planning, object avoidance, or steering autonomous driving action. In some aspects, using the confidence information when performing an autonomous or semi-autonomous driving action by the vehicle may include: changing an autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle.

[0008] In some aspects, changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle may include: changing the autonomous driving mode of the vehicle implemented by the processor to a driving mode compatible with the confidence information about the object or feature near the vehicle. Some aspects may also include: in response to determining that the confidence information about the object or feature near the vehicle does not support a fully autonomous driving mode, notifying the driver that he needs to participate in driving the vehicle; and after notifying the driver, changing the autonomous driving mode of the vehicle implemented by the processor. In some aspects, the confidence information about the object or feature may include confidence information about objects and features within a defined area, and changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle may include: while the vehicle is in the defined area, changing the autonomous driving mode of the vehicle implemented by the processor to an autonomous driving mode consistent with the confidence information.

[0009] Some aspects may also include: obtaining, by the processor, sensor data about the object or feature near the vehicle from a vehicle sensor; determining, by the processor, whether the obtained sensor data about the object or feature near the vehicle differs from the map data about the object or feature obtained from the map database by a threshold amount; and uploading, by the processor, to a remote computing device, the obtained sensor data about the object or feature near the vehicle together with confidence information based on one or more of a type of sensor used to detect or classify the object or feature, a perceived quality of the object or feature achieved by the sensor, or an accuracy or precision of the sensor data, in response to determining that the obtained sensor data differs from the map data about the object or feature obtained from the map database by at least the threshold amount.

[0010] Additional aspects include a vehicle processing system including a memory and a processor configured to perform operations of any of the methods outlined above. Additional aspects may include a vehicle processing system having various components for performing functions corresponding to any of the methods outlined above. Additional aspects may include a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause a processor of the vehicle processing system to perform various operations corresponding to any of the methods outlined above.

[0011] Additional aspects include methods performed by a computing device for including safety and confidence information in map data used by autonomous and semi-autonomous driving systems in a vehicle. Various aspects may include: receiving, by the computing device, information about an object or feature from a source for inclusion in a map database, the information including a confidence measure in the information about the object or feature; using the received confidence measure in the information about the object or feature to generate safety and confidence information about the object or feature suitable for use by the vehicle autonomous and semi-autonomous driving system in an autonomous or semi-autonomous driving operation, wherein the safety and confidence information may include one or more of the following: an ASIL autonomous driving level near the object or feature; an indication related to the accuracy of the map data about the object or feature; a statistical score indicating the precision of the map data about the object or feature; an indication related to the reliability of the map data about the object or feature; or the age or freshness of the map data about the object or feature; and storing the safety and confidence information about the object or feature in a manner that enables access by the vehicle autonomous and semi-autonomous driving system. In some aspects, storing the safety and confidence information may include updating information about the object or feature in the map database based at least in part on the received confidence metric in the received information about the object or feature confidence.

[0012] In some aspects, information about an object or feature is received for inclusion in a map database, the information including a confidence measure in the information about the object or feature, and receiving the information may include receiving information from one or more vehicles including: a location of the object or feature; characteristics of the object or feature; and a confidence measure in the information about the location or characteristics of the object or feature.

[0013] In some aspects, storing the safety and confidence information about the object or feature may include including the safety and confidence information as part of the location and other information about the object or feature in the map database provided to the vehicle for use in autonomous or semi-autonomous driving operations. In some aspects, storing the safety and confidence information about the object or feature may include: storing the safety and confidence information in a database separate from the map database associated with the location information of the object or feature; and providing the database to the vehicle for use in autonomous or semi-autonomous driving operations.

[0014] In some aspects, receiving information about an object or feature for inclusion in a map database may include: receiving information about the object or feature from multiple sources together with a confidence measure in the information about the object or feature. Such aspects may also determine a set of information about the object or feature and combined safety and confidence information of the determined set of information about the object or feature based on the information received from the multiple sources; and storing the safety and confidence information about the object or feature in a manner that enables access by the vehicle's autonomous and semi-autonomous driving systems for use in autonomous or semi-autonomous driving operations may include storing the combined safety and confidence information of the determined set of information about the object or feature in a manner that enables access by the vehicle's autonomous and semi-autonomous driving systems for use in autonomous or semi-autonomous driving operations.

[0015] Additional aspects include a computing device, such as a server, comprising a memory and a processor configured to perform the operations of any of the methods outlined above. Additional aspects may include a computing device having components for performing functions corresponding to any of the methods outlined above. Additional aspects may include a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause a processor of the computing device to perform various operations corresponding to any of the methods outlined above. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description and detailed description given, serve to explain the features herein.

[0017] Figure 1A is a system block diagram illustrating an example communication system suitable for implementing various embodiments.

[0018] Figure 1B is a system block diagram illustrating an example decomposed base station architecture suitable for implementing any of the various embodiments.

[0019] Figure 2A is a component diagram of an example vehicle system suitable for implementing various embodiments.

[0020] Figure 2B is a component block diagram illustrating the computing layer of an example vehicle ADS processing system in accordance with various embodiments.

[0021] Figure 3A is a block diagram illustrating components of a system-on-chip for use in a vehicle ADS processing system according to various embodiments.

[0022] Figure 3Bis a component block diagram illustrating a system configured to perform operations for using safety and / or confidence information associated with object and feature map data according to various embodiments.

[0023] Figure 4A and Figure 4B is a street profile diagram illustrating objects and features that may be included in a map database, as well as safety and confidence information regarding their use, according to various embodiments.

[0024] Figure 4C A diagram of data fields for data elements of a map database including security and confidence information suitable for implementing some embodiments.

[0025] Figure 4D and Figure 4E A data field diagram of data elements of a map database and data elements of security and confidence information suitable for implementing some embodiments.

[0026] Figure 5A is a process flow diagram of an example method 500a for using safety and / or confidence information associated with object and feature map data, according to various embodiments.

[0027] FIG. 5B to FIG. 5H is a process flow diagram of example operations 500b through 500h according to some embodiments, which example operations may be used as shown in FIG. Figure 5A The portion of the method described in for using safety and / or confidence information associated with object and feature map data is performed.

[0028] Fig. 6A is a process flow diagram of an example method performed by a computing device for generating a database of safety and / or confidence information based on information received from an ADS-equipped vehicle in accordance with various embodiments.

[0029] FIG. 6B to FIG. 6E is a process flow diagram of example operations 600b through 600e according to some embodiments, which example operations may be used as Fig. 6A The portion of the method 600a shown in is performed for generating a database of safety and / or confidence information based on information received from an ADS-equipped vehicle.

[0030] Figure 7 is a component block diagram of a computing device suitable for use with the various embodiments. DETAILED DESCRIPTION

[0031] Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and implementations are for illustrative purposes and are not intended to limit the scope of the claims.

[0032] Various embodiments include methods and processors for a vehicle autonomous driving system (ADS) that uses map data including safety and / or confidence information when performing autonomous driving functions. Various embodiments may include a vehicle ADS processing system accessing a map database to obtain map data about objects or features near the vehicle, and also accessing information about the autonomous driving safety level and / or information about the confidence that should be attributed to the corresponding map data about a given object or feature (referred to herein as "confidence information"). Such safety and / or confidence information can be included in or linked to map data about objects or features near the vehicle in a manner that enables the vehicle ADS processing system to obtain the information in conjunction with accessing or using the corresponding map data. Various embodiments may also include the use of safety and / or confidence information by a processor when an autonomous or semi-autonomous driving action is performed by the processor.

[0033] In some embodiments, the safety and / or confidence information may include an automotive safety integrity level (ASIL) autonomous driving level near an object or feature. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include an accuracy factor, metric, or indication associated with the object or feature map data. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include a reliability factor or metric. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include a statistical score indicating the accuracy of the map data about the object or feature. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include an indication associated with the reliability of the object or feature map data. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include the age or freshness of the object or feature map data. For example, when near an object or feature, the vehicle ADS processing system may use the safety information associated with the object or feature map data to determine and implement an appropriate autonomous driving mode. As another example, the vehicle ADS processing system may use confidence information to determine weights to apply to object or feature map data and use the weighted object or feature map data while performing path planning, object avoidance, or steering autonomous driving actions. As another example, safety and / or confidence information may include a statistical score or measurement, such as an F-score or F-measure, which has a value between 1 (best) and 0 (worst) and provides a measure of the accuracy of a measurement calculated from the precision of the measurement or sensor. An example of an F-score that can be used is called an F1 score, which is the harmonic mean of the precision. The F1 score is also known as the Sorensen-Dice coefficient or the Dice similarity coefficient.

[0034] In some embodiments, security and / or confidence information associated with object or feature map data may be included in a map database so that the information may be obtained by a processor in the same or related operation as obtaining the object or feature information. For example, the security and / or confidence information may be stored in one or more data fields along with location and descriptive information about objects and features. In some embodiments, the security and / or confidence information associated with object or feature map data may be stored in and obtained from a data structure other than the map database that can be accessed by the processor, such as a supply or download (or downloadable) data table indexed to the location or identifier of the objects and features.

[0035] The map database and / or safety and / or confidence information database may be stored in system memory and / or obtained from a remote data store, such as a roadside unit, a central database server, and / or other vehicles. The map information stored in the database hosted in the memory may come from a remote side unit (e.g., a smart RSU) or from another vehicle. In such embodiments, the confidence assigned to objects and features in the map data may depend on the source of the map data. For example, the confidence level assigned to or associated with objects and features in a map generated from a single other vehicle received via V2X communication may be less than the confidence level assigned to or associated with objects and features in a map generated by map crowdsourcing.

[0036] In some embodiments, the vehicle ADS processing system may identify when vehicle sensor data about an object or feature near the vehicle differs from the map data by a threshold amount and, in such a case, upload the sensor data to a remote computing device. Such uploaded sensor data about the object or feature may include map coordinates together with information about a measurement or estimate of the accuracy or precision of the sensor data. Some embodiments also include a remote computing device that may receive the object or feature sensor data and use the received measurements of confidence information about the object or feature confidence to generate safety and / or confidence information about the object or feature in a format suitable for use by the vehicle ADS when performing autonomous or semi-autonomous driving operations (e.g., within a map database or within a separate database).

[0037] Various embodiments include storing or providing information about a safety ASIL (or other measured) autonomy level (collectively referred to herein as "safety information") and / or information about a confidence level (e.g., accuracy, precision, reliability, age, etc.) in object or feature map data. The safety information and confidence information may be related in some instances, as it may be suitable to indicate that fully autonomous driving is unsafe near an object or feature for which there is a low confidence in the map data. However, the safety information may be independent of the confidence information, such as when the map object or feature is associated with typical road, traffic, or pedestrian conditions in which fully autonomous driving is risky (i.e., where there is a high confidence in the map data). In addition, there may be a low confidence in the map data for some objects or features without affecting the safety autonomy level, such as the location of an object or feature next to but not in the road. In various embodiments, the safety information and information about the confidence level of the object or feature map data may be stored and accessed in the same or similar manner. For these reasons and for ease of reference, safety information and confidence information are referred to herein as "safety and / or confidence information" or collectively as "confidence information." Therefore, references to "confidence information" in the specification and some claims are not intended to exclude information limited to safe autonomous driving levels.

[0038] As used herein, the term "vehicle" generally refers to any of a car, truck, bus, train, boat, and any other type of mobile ADS-capable system that can access map data to perform autonomous or semi-autonomous functions.

[0039] The term "system on chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that includes multiple resources and / or processors integrated on a single substrate. A single SOC may contain circuits for digital, analog, mixed signal, and radio frequency functions. A single SOC may also include any number of general and / or special processors (digital signal processors, modem processors, video processors, etc.), storage blocks (e.g., ROM, RAM, flash memory, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). The SOC may also include software for controlling the integrated resources and processors and for controlling peripheral devices.

[0040] The term "system-in-package" (SIP) may be used herein to refer to a single module or package that contains multiple resources, computing units, cores and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a unified substrate. A SIP may also include multiple independent SOCs that are connected together by high-speed communication circuits and packaged in close proximity, such as on a single motherboard or in a single wireless device. The proximity of the SOCs facilitates high-speed communication and the sharing of memory and resources.

[0041] Technologies and technical standards are being developed in many regions of the world to support evolving and future highway systems and vehicles, including setting standards for enabling safe autonomous and semi-autonomous vehicle operations. Such technologies include standardizing vehicle-based communication systems and functions, and developing standards for vehicle autonomous driving systems (ADS).

[0042] One of the standards being developed for autonomous and semi-autonomous vehicles includes the International Organization for Standardization (ISO) standard 26262 on functional safety of road vehicles. ASIL is a risk classification defined by the ISO 26262 standard, which defines functional safety as "the absence of unreasonable risk of hazards caused by the faulty behavior of electrical or electronic systems". ISO 26262 defines the Automotive Safety Integrity Level (ASIL), which is a risk classification associated with an appropriate level of performance, accuracy and reliability imposed on the vehicle ADS systems and data to ensure an acceptable level of functional safety in different autonomous driving modes. ASILs establish safety requirements for automotive components to comply with the standard based on the likelihood and acceptability of hazards. Four ASILs are identified in ISO 26262-A, B, C and D. ASIL-A represents the lowest level and ASIL-D represents the highest level of automotive hazards. Systems such as airbags, anti-lock brakes and power steering require ASIL D rating, which is the highest level of stringency applied to safety assurance because the risk associated with their failure is the highest. At the other end of the safety spectrum, components such as taillights only require ASIL A rating. Headlights and brake lights are typically ASIL-B, while cruise control is typically ASIL-C.

[0043] ASIL is also referred to as a level. Including the level without functional safety equipment ("L0"), there are five ASILs. In L0, the driver is fully responsible for the safe operation of the vehicle and no driver assistance is provided. In L1, the driver can represent steering or acceleration / braking, but the system only performs one driving task. In L2, the driver must constantly monitor the system, but the ADS performs several driving tasks (e.g., steering, cruise control with safe distance control, and autonomous braking). In L3, the driver can divert attention from the road in certain circumstances, and the ADS can autonomously control the vehicle on a limited route (e.g., in highway driving). In L4, the driver can transfer complete control to the system, but can control at any time because the system is able to perform all driving tasks. Finally, no driver is required in L5 because the system can autonomously control the vehicle under all conditions.

[0044] The ASIL level defines not only the type of driving, but also the confidence, accuracy and reliability level required for the vehicle to operate at a given ASIL autonomy level. Therefore, when the safety and / or confidence information associated with the map data of the nearby object or feature is less than the current ASIL operating level (e.g., L4 or L5) of the vehicle or the autonomous / semi-autonomous driving mode required, the vehicle ADS processor should change the operating mode to an ASIL autonomous driving level consistent with the object / feature safety and / or safety and confidence information (e.g., L3). For example, if the vehicle operates autonomously (e.g., in L4 or L5) and approaches an object and / or feature with safety and / or safety and confidence information that only supports semi-autonomous or driver-assisted operating modes (e.g., L3 or L2), the vehicle processor should notify the driver that he / she must pay attention to the road or control the vehicle. However, there is currently no agreed solution for notifying the vehicle ADS about the ASIL level associated with or suitable for driving near a specific object or feature identified in the map data used by the ADS.

[0045] One of the technologies and standards supporting autonomous and semi-autonomous driving is the communication technology and network for intelligent highway systems (ITS). Examples include standards being developed for North America in the Institute of Electrical and Electronics Engineers (IEEE) and the Society of Automotive Engineers (SAE), or standards being developed for Europe in the European Telecommunications Standards Institute (ETSI) and the European Committee for Standardization (CEN). For example, the IEEE 802.11p standard is the basis for dedicated short-range communications (DSRC) and ITS-G5 communication standards. IEEE1609 is a higher-layer standard based on IEEE 802.11p. The cellular vehicle-to-everything (C-V2X) standard is a competitive standard developed under the support of the 3rd Generation Partnership Project. These standards are based on the basis of wireless communications based on vehicles, and can be used to support intelligent highways, autonomous and semi-autonomous vehicles, and improve the overall efficiency and safety of highway transportation systems. ITS communications can be supported by the next generation 5GNR communication system. These and other V2X wireless technologies can be used in various embodiments to download map data and safety and / or confidence information, and upload observations of vehicle sensors according to various embodiments for updating map data.

[0046] The C-V2X protocol defines two transmission modes, which together provide 360° non-line-of-sight awareness and a higher level of predictability for enhanced road safety and autonomous driving. The first transmission mode includes direct C-V2X, which includes vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) and vehicle-to-pedestrian (V2P), and provides enhanced communication range and reliability in the dedicated intelligent transportation system (ITS) 5.9 gigahertz (GHz) spectrum independent of the cellular network. The second transmission mode includes vehicle-to-network communication (V2N) in mobile broadband systems and technologies (such as third-generation wireless mobile communication technologies (3G) (e.g., Global System for Mobile Communications (GSM) Evolution (EDGE) system, Code Division Multiple Access (CDMA) 2000 system, etc.), fourth-generation wireless mobile communication technologies (4G) (e.g., Long Term Evolution (LTE) system, Advanced LTE system, Mobile Global Interoperability for Microwave Access (Mobile WiMAX) system, etc.), fifth-generation new radio mobile communication technologies (5G NR system, etc.), etc.).

[0047] Autonomous vehicles can use map data in combination with vehicle sensors and other information sources to perform autonomous and semi-autonomous functions, such as autonomous braking and speed management, path planning, steering, obstacle avoidance, etc. The vehicle ADS processing system usually receives information about the external environment (e.g., landmarks, road signs, traffic signs, etc.) and other vehicles from multiple on-board sensors (e.g., cameras, radars, lidars, global positioning system (GPS) receivers, etc.), and the processor can use this information for navigation and object avoidance. The vehicle processor can also use digital map data, which includes the location and information about streets, roads and short-distance objects, road markings, traffic control signals, and other information that is useful for autonomously or semi-autonomously operating vehicles safely. Examples of such digital maps include SD maps, information-rich high-definition (HD) maps, dynamic maps, and autonomous driving maps. The vehicle processor can also receive location information from a positioning system or a communication network.

[0048] Sensor data from various vehicle sensors used to perform autonomous and semi-autonomous functions will exhibit different levels of accuracy and precision. Depending on the nature and design of each sensor (e.g., sensor operating wavelength, aperture size, position on the vehicle and field of view, etc.), the sensor has inherent limitations in accuracy and precision. Additionally, environmental conditions such as precipitation, smoke, dust, lighting, sun angle, etc. can affect the accuracy and precision of sensor data. Therefore, when performing autonomous and semi-autonomous functions, the vehicle processor can take into account the inherent and potential inaccuracies in the sensor data, such as by merging data from multiple sensors to determine the position of the vehicle relative to features and objects in the environment. In this case, the processor can consider the confidence level associated with each sensor or sensor data set, and rely more heavily on sensor data with a high confidence level than sensor data with a lower confidence level. For example, when merging data from multiple sensors for determining position, making steering or braking decisions, and path planning, the processor can apply weights to various sensor data based on the confidence metric associated with each sensor to achieve a weighted or confidence-adjusted position of various objects and features in the environment.

[0049] Similar to sensor data, information about objects and features included in a map used by a vehicle processor equipped with an ADS may have varying levels of accuracy or precision depending on the source of such information. For example, location information providing road boundaries (e.g., the centerline of a lane, lane width, curb location, etc.) may be determined by surveying and therefore recorded in a map with high accuracy and confidence. In contrast, location information about landmarks and temporary obstacles (e.g., construction sites, movable obstacles, potholes, etc.) may be collected by vehicle sensors, which may have varying degrees of accuracy and precision depending on the characteristics of the sensor, the conditions under which the location information is collected, the viewing angle when measuring the object or feature, and the like. In addition, some sources of location information may be more credible than others. However, conventional maps used by vehicle ADS processing systems for autonomous and semi-autonomous driving functionality may provide little or no information about the reliability or accuracy of object and feature location data. Thus, while a vehicle ADS processing system may be configured to consider confidence metrics of vehicle sensor data, conventional maps do not provide equivalent safety and confidence information about map objects and features that a processor may consider in positioning, driving decisions, route planning, etc.

[0050] Various embodiments overcome such limitations in digital map data used by a vehicle ADS processing system for autonomous and semi-autonomous driving functions by including safety and confidence information (e.g., confidence metrics) associated with objects and features in a digital map. Various embodiments include methods for performing autonomous and semi-autonomous driving functions using safety and confidence information (such as confidence metrics) associated with objects and features in a digital map. Some embodiments include methods for including safety and confidence information (such as confidence metrics) associated with objects and features in a digital map suitable for use by a vehicle equipped with a vehicle ADS processing system configured to perform the operations of the various embodiments.

[0051] In some embodiments, the vehicle processor may be configured to perform operations including accessing map data about objects or features near the vehicle from a map database accessible by the processor and obtaining or accessing safety and / or confidence information associated with the object or feature map data near the vehicle. In some embodiments, the safety and / or confidence information may be included in the map database, such as a portion of a map data record. In some embodiments, the safety and / or confidence information may be stored in a database separate from the map database, such as having an index or common data element that enables the vehicle processor to find safety and / or confidence information corresponding to specific object and feature map data. The vehicle processor may then use the safety and / or confidence information when the processor performs autonomous or semi-autonomous driving actions. In various embodiments, when performing autonomous or semi-autonomous driving actions, the vehicle ADS processing system may adjust the level of autonomous driving performed by the system based on safety and confidence information (e.g., switching to a lower level of autonomous driving consistent with the safety information), taking confidence information in object or feature map information into consideration as part of sensor fusion and navigation, route planning, object avoidance, interruption or suspension of autonomous driving functions, functionality, features or actions, etc.

[0052] In some embodiments, the vehicle processor of the autonomous driving system may apply weights to the accessed map data about the object or feature based on the confidence information, and the weighted map data about the object or feature is used by the processor while performing path planning, object avoidance, steering, and / or other autonomous driving actions. In some embodiments, the vehicle processor of the autonomous driving system may perform operations including changing the autonomous driving mode of the vehicle implemented by the vehicle processor based on safety and / or confidence information about objects or features near the vehicle. In some embodiments or situations, the vehicle processor of the autonomous driving system may interrupt or suspend autonomous driving functions, functionalities, features, or actions.

[0053] In some embodiments, a vehicle processor of an autonomous driving system may obtain sensor data about objects and features near the vehicle from vehicle sensors, determine whether the sensor data indicates a new object or feature, or a difference between the sensor data and map data about the object or feature, and upload information about the location of the new or changed object or feature to a remote computing device, including information about the confidence level (e.g., the accuracy, precision, or reliability of the underlying sensor data) in the uploaded location information. In this way, an ADS-equipped vehicle may support the creation of a map database that includes confidence information.

[0054] In some embodiments, a computing device (such as a server) may be configured to receive a report from an ADS-equipped vehicle that identifies new or different locations of objects and features included in one or more maps used by the vehicle's ADS processing system for autonomous and semi-autonomous driving functions, including safety and confidence information (e.g., confidence metrics) associated with each identified location. In such embodiments, the computing device may be configured to perform operations on the received information to determine appropriate confidence information for the added map data, and store the confidence information in a database provided to or accessible by the ADS-equipped vehicle.

[0055] Figure 1A 1 is a system block diagram illustrating an example communication system 100 suitable for implementing various embodiments. The communication system 100 includes a 5G New Radio (NR) network, an ITS V2X wireless network, and / or any other suitable network, such as a Long Term Evolution (LTE) network. References to 5G networks and 5G network elements in the following description are for illustrative purposes and are not intended to be limiting.

[0056] The communication system 100 may include a heterogeneous network architecture including a core network 140, a plurality of base stations 110, and various mobile devices including a vehicle 102 equipped with an ADS 104 including wireless communication capabilities. The base stations 110 may communicate with the core network 140 over a wired network 126. The communication system 100 may also include a roadside unit 112 that supports V2X communication with the vehicle 102 via a V2X wireless communication link 124.

[0057] Base station 110 is a network element that communicates with a wireless device (e.g., vehicle 102) via a NodeB, an LTE evolved NodeB (eNodeB or eNB), an access point (AP), a radio head, a transmit receive point (TRP), a new radio base station (NR BS), a 5G NodeB (NB), a next generation NodeB (gNodeB or gNB), etc., and may also be referred to as a NodeB, an LTE evolved NodeB (eNodeB or eNB), an access point (AP), a radio head, a transmit receive point (TRP), a new radio base station (NR BS), a 5G NodeB (NB), a next generation NodeB (gNodeB or gNB), etc. Each base station 110 may provide communication coverage for a particular geographic area or "cell". In 3GPP, the term "cell" may refer to the coverage area of ​​a base station, or a base station subsystem serving the coverage area, or a combination thereof, depending on the context in which the term is used. Core network 140 may be any type of core network, such as an LTE core network (e.g., an evolved packet core (EPC) network), a 5G core network, a reference Figure 1BDecomposed networks, etc.

[0058] According to various embodiments, the roadside unit may be coupled to a remote computing device 132 via a wired network 128, which may store and map data and confidence information for communication to the vehicle 102. According to some embodiments, the roadside unit 112 may communicate with an ITS- and ADS-equipped vehicle 102 via a V2X wireless communication link 124 to download information useful for ADS autonomous and semi-autonomous driving functions, including downloading map databases and other databases including safety and / or confidence information databases. According to some embodiments, the V2X wireless communication link 124 may also be used to upload information about objects and features obtained by vehicle sensors and associated confidence metrics to the remote computing device 132 for use in generating map data.

[0059] According to some embodiments, cellular wireless communications, such as 5G wireless communications supported by base station 110, may also be used to download information useful for ADS autonomous and semi-autonomous driving functions, including downloading map databases and other databases including safety and / or confidence information databases, and for uploading information about objects and features and associated confidence metrics obtained by vehicle sensors to a remote computing device 132 for use in generating map data. To support such communications, the remote computing device 132 hosting the map and confidence information databases may be coupled to a core network via a communication link 127 such as the Internet, and the map data and confidence information may be communicated to the base station 110 via a wired communication link 126 (e.g., Ethernet, optical fiber, etc.) for download to the vehicle 102 via a cellular wireless communication link 122 such as a 5G wireless communication link.

[0060] The cellular wireless communication link 122 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. The wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that may be used in the wireless communication links include: 3GPP LTE, 3G, 4G, 5G (e.g., NR), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Worldwide Interoperability for Microwave Access (WiMAX), Time Division Multiple Access (TDMA), and other mobile phone communication technology cellular RATs. Other examples of RATs that may be used in one or more of the various wireless communication links within the communication system 100 include medium-range protocols (such as Wi-Fi, LTE-U, LTE-Direct, LAA, MuLTEfire) and relatively short-range RATs (such as ZigBee, Bluetooth, and Bluetooth Low Energy (LE)).

[0061] Figure 1Bis a system block diagram illustrating an example decomposed base station 160 architecture that may be part of a V2X and / or 5G network suitable for communicating map data to a vehicle and communicating updated object / or feature locations in accordance with any of various embodiments. Figure 1A and Figure 1B , the decomposed base station 160 architecture may include one or more central units (CU) 162, which may communicate directly with the core network 180 via a backhaul link, or indirectly with the core network 180 through one or more decomposed base station units (such as a near real-time (Near-RT) RAN intelligent controller (RIC) 164 via an E2 link, or a non-real-time (Non-RT) RIC 168 associated with a service management and orchestration (SMO) framework 166, or both). The CU 162 may communicate with one or more distributed units (DU) 170 via a corresponding midhaul link such as an F1 interface. The DU 170 may communicate with one or more radio units (RU) 172 via a corresponding fronthaul link. The RU 172 may communicate with a corresponding UE 120 via one or more radio frequency (RF) access links. In some implementations, a user equipment (UE) such as the vehicle ADS system 104 may be served by multiple RUs 172 simultaneously.

[0062] Each of the units (i.e., CU 162, DU 170, RU 172), as well as near-RT RIC 164, non-RT RIC 168, and SMO framework 166 may include one or more interfaces, or be coupled to one or more interfaces configured to receive or send signals, data, or information (collectively referred to as signals) via a wired or wireless transmission medium. Each of the units or an associated processor or controller that provides instructions to the communication interface of these units may be configured to communicate with one or more of the other units via a transmission medium. For example, these units may include a wired interface that is configured to receive or send signals to one or more of the other units via a wired transmission medium. Additionally, these units may include a wireless interface that may include a receiver, a transmitter, or a transceiver (such as a radio frequency (RF) transceiver) that is configured to receive or send signals, or both, to one or more of the other units via a wireless transmission medium.

[0063] In some aspects, CU 162 may host one or more higher layer control functions. Such control functions may include radio resource control (RRC), packet data convergence protocol (PDCP), service data adaptation protocol (SDAP), etc. Each control function may be implemented with an interface configured to communicate signals with other control functions hosted by CU 162. CU 162 may be configured to handle user plane functionality (i.e., central unit-user plane (CU-UP)), control plane functionality (i.e., central unit-control plane (CU-CP)), or a combination thereof. In some specific implementations, CU 162 may be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP unit may communicate bidirectionally with the CU-CP unit via an interface (such as an E1 interface). As needed, CU 162 may be implemented to communicate with DU 170 for network control and signaling.

[0064] DU 170 may correspond to a logical unit that includes one or more base station functions for controlling the operation of one or more RUs 172. In some aspects, DU 170 may host one or more of a radio link control (RLC) layer, a medium access control (MAC) layer, and one or more high physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) at least in part according to a functional split such as those defined by the 3rd Generation Partnership Project (3GPP). In some aspects, DU 170 may further host one or more low PHY layers. Each layer (or module) may be implemented using an interface that is configured to communicate signals with other layers (and modules) hosted by DU 170 or with control functions hosted by CU 162.

[0065] The lower layer functionality may be implemented by one or more RUs 172. In some deployments, the RUs 172 controlled by the DUs 170 may correspond to logical nodes that host RF processing functions or low PHY layer functions (such as performing fast Fourier transforms (FFTs), inverse FFTs (iFFTs), digital beamforming, physical random access channel (PRACH) extraction and filtering, etc.), or both, based at least in part on functional splits (such as lower layer functional splits). In such an architecture, the RUs 172 may be implemented to handle over-the-air (OTA) communications with one or more UEs 120. In some specific implementations, real-time and non-real-time aspects of control and user plane communications with the RUs 172 may be controlled by the corresponding DUs 170. In some scenarios, this configuration may enable the DUs 170 and CUs 162 to be implemented in a cloud-based radio access network (RAN) architecture (such as a vRAN architecture).

[0066] The SMO framework 166 may be configured to support RAN deployment and provisioning of non-virtualized network elements and virtualized network elements. For non-virtualized network elements, the SMO framework 166 may be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which may be managed via an operation and maintenance interface (such as an O1 interface). For virtualized network elements, the SMO framework 166 may be configured to interact with a cloud computing platform (such as an open cloud (O-Cloud) 176) to perform network element lifecycle management (such as instantiating virtualized network elements) via a cloud computing platform interface (such as an O2 interface). Such virtualized network elements may include, but are not limited to, CU 162, DU 170, RU 172, and near-RT RIC 164. In some specific implementations, the SMO framework 166 may communicate with hardware aspects of the 4G RAN (such as an open eNB (O-eNB) 174) via the O1 interface. Additionally, in some specific implementations, the SMO framework 166 may communicate directly with one or more RUs 172 via the O1 interface. The SMO framework 166 may also include a non-RT RIC 168 configured to support the functionality of the SMO framework 166 .

[0067] The non-RT RIC 168 may be configured to include logic functions that enable non-real-time control and optimization of RAN elements and resources, artificial intelligence / machine learning (AI / ML) workflows including model training and updating, or policy-based guidance of applications / features in the near-RT RIC 164. The non-RT RIC 168 may be coupled to or in communication with the near-RT RIC 164 (such as via an A1 interface). The near-RT RIC 164 may be configured to include logic functions that enable near-real-time control and optimization of RAN elements and resources via data collection and actions through an interface (such as via an E2 interface) that connects one or more CUs 162, one or more DUs 170, or both, and the O-eNB with the near-RT RIC 164.

[0068] In some implementations, in order to generate an AI / ML model to be deployed in the near-RT RIC 164, the non-RT RIC 168 may receive parameters or external enrichment information from an external server. Such information may be utilized by the near-RT RIC 164 and may be received from a non-network data source or from a network function at the SMO framework 166 or the non-RT RIC 168. In some examples, the non-RT RIC 168 or the near-RT RIC 164 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 168 may monitor long-term trends and patterns of performance and employ AI / ML models to perform corrective actions through the SMO framework 166 (such as via reconfiguration of O1) or via creation of RAN management policies (such as A1 policies).

[0069] Figure 2A is a component diagram of an example vehicle ADS system 200 suitable for implementing various embodiments. Figures 1A to 2A , the system 200 may include a vehicle 102 including a vehicle ADS 104. The vehicle processing system 104 may communicate with various systems and devices such as an onboard network 210, an infotainment system 212, various sensors 214, various actuators 216, and a radio module 218 coupled to an antenna 219. The vehicle processing system 104 may also communicate with a roadside unit 112, a cellular communication network base station 110, and other external devices.

[0070] The vehicle ADS processing system 204 may include a processor 205, a memory 206, an input module 207, an output module 208, and a radio module 218. The processor 205 may be coupled to the memory 206 (i.e., a non-transitory storage medium) and may be configured with processor-executable instructions stored in the memory 206 to perform operations according to the methods of the various embodiments described herein. In addition, the processor 205 may be coupled to the output module 208 (which may control the vehicle-mounted display) and to the input module 207 to receive information from vehicle sensors and driver input.

[0071] The vehicle ADS processing system 204 may include a V2X antenna 219 coupled to a radio module 218, which is configured to communicate with one or more ITS participants (e.g., stations), roadside units 112, and base stations 110 or another suitable network access point. The V2X antenna 219 and the radio module 218 may be configured to receive dynamic traffic flow characteristics information via vehicle-to-everything (V2X) communications. In various embodiments, the vehicle ADS processing system 204 may receive information from multiple information sources, such as an onboard network 210, an infotainment system 212, various sensors 214, various actuators 216, and a radio module 218. The vehicle ADS processing system 204 may be configured to use map data in addition to sensor data to perform autonomous or semi-autonomous driving functions, as further described below.

[0072] Examples of vehicle networks 210 include controller area networks (CAN), local interconnect networks (LIN), networks using the FlexRay protocol, media oriented systems transport (MOST) networks, and automotive Ethernet networks. Examples of vehicle sensors 214 include location determination systems (such as global navigation satellite systems (GNSS) systems), cameras, radars, lidars, ultrasonic sensors, infrared sensors, and other suitable sensor devices and systems. Examples of vehicle actuators 216 include various physical control systems such as for steering, braking, engine operation, lights, direction signals, etc.

[0073] Figure 2B 2 is a component block diagram illustrating components of an example vehicle ADS processing system stack 220. The vehicle management system 220 may include various subsystems, communication elements, computing elements, computing devices, or units that may be utilized within the vehicle 102. Figures 1A to 2A The various computing elements, computing devices or units within the ADS processing system stack 220 may be implemented within a system (i.e., a subsystem) of computing devices that communicate data and commands to each other via the vehicle network 210 (e.g., by Figure 2B In some implementations, various computing elements, computing devices, or units within the vehicle ADS processing system 104 may be implemented within a single computing device, such as separate threads, processes, algorithms, or computing elements. Figure 2BEach subsystem / computing element illustrated in the 2000 is also generally referred to herein as a "layer" within the computing "stack" that makes up the vehicle ADS processing system 220. However, the use of the terms layer and stack when describing various embodiments is not intended to imply or require that the corresponding functionality be implemented within a single vehicle computing device, although this is a potential specific implementation embodiment. Instead, the use of the term "layer" is intended to include subsystems with independent processors, computing elements (e.g., threads, algorithms, subroutines, etc.) running in one or more computing devices, and combinations of subsystems and computing elements.

[0074] The vehicle ADS processing system stack 220 may include a radar and / or lidar perception layer 222, a camera perception layer 224, a positioning engine layer 226, a map database 228 including safety and / or confidence information (or a linked database storing such information), a map fusion and arbitration layer 230, a route planning layer 232, an ASIL operating mode assessment layer 234, a sensor fusion and road world model (RWM) management layer 236, a motion planning and control layer 238, and a behavior planning and prediction layer 240. Layers 222 to 240 are merely examples of some of the layers in one example configuration of the vehicle ADS processing system stack 220. In other configurations, other layers may be included, such as additional layers for other perception sensors (e.g., a lidar perception layer, etc.), additional layers for planning and / or control, additional layers for modeling, etc., and / or some of the layers 222 to 240 may be excluded from the vehicle ADS processing system stack 220. Figure 2B As illustrated by the arrows in , each of the layers 222 to 240 can exchange data, calculation results, and commands. In addition, the vehicle ADS processing system stack 220 can receive and process data from sensors (e.g., radar, lidar, camera, inertial measurement unit (IMU), etc.), navigation information sources (e.g., GPS receiver, IMU, etc.), vehicle networks (e.g., controller area network (CAN) bus), and databases in memory (e.g., digital map data). The vehicle ADS processing system stack 220 can output vehicle control commands or signals to the ADS vehicle control unit 220, which is a system, subsystem, or computing device that interfaces directly with the vehicle steering, throttle, and brake controls. Figure 2A The configuration of the vehicle ADS processing system stack 220 and the ADS vehicle control unit 242 illustrated in FIG. 2 is merely an example configuration, and other configurations of the vehicle management system and other vehicle components may be used. As an example, Figure 2B The configuration of the vehicle ADS processing system stack 220 and the ADS vehicle control unit 242 illustrated in FIG. 2 may be used in a vehicle configured for autonomous or semi-autonomous operation, while a different configuration may be used in a non-autonomous vehicle.

[0075] The radar and / or lidar perception layer 222 may receive data from one or more detection and ranging sensors, such as radar (e.g., 132) and / or lidar (e.g., 138), and process the data to identify and determine the location of other vehicles and objects in the vicinity of the vehicle 100. The radar perception layer 222 may include the use of neural network processing and artificial intelligence methods to identify objects and vehicles, and pass such information to the sensor fusion and RWM management layer 236.

[0076] The camera perception layer 224 may receive data from one or more cameras, such as cameras, and process the data to identify and determine the location of other vehicles and objects in the vicinity of the vehicle 100. The camera perception layer 224 may include the use of neural network processing and artificial intelligence methods to identify objects and vehicles, and pass such information to the sensor fusion and RWM management layer 236.

[0077] The positioning engine layer 226 may receive data from the radar and / or lidar sensing layer 222, the camera sensing layer 224, and various navigation information sources, and process the data and information to determine the location of the vehicle 100. The various navigation information sources may include, but are not limited to, GPS receivers, IMUs, and / or other sources and sensors connected via a CAN bus. The positioning engine layer 226 may also utilize input from one or more cameras, such as cameras and / or any other available sensors capable of identifying and determining the direction and distance to objects near the vehicle, such as radar, lidar, etc.

[0078] The vehicle ADS processing system 220 may include or be coupled to a vehicle wireless communication subsystem 218. The wireless communication subsystem 218 may be configured to communicate with a highway communication system, such as via a V2X communication link (e.g., 124) and / or communicate with a remote information source (e.g., computing device 132) via a cellular wireless communication link (e.g., 122), such as via a 5G cellular network.

[0079] The map fusion and arbitration layer 230 may access the map database 228 to obtain location information and safety and / or confidence information about nearby objects and features, and receive positioning / navigation information output from the positioning engine layer 226, and process the data to further determine the location of the vehicle 102 in the map, such as the location within a lane, the location within a street map, etc. The sensor data may be stored in a memory (e.g., memory 312).

[0080] In determining the location of the vehicle 102 within the map, the positioning engine layer 226 considers confidence information about the location of objects and features within the map and confidence (e.g., accuracy and / or precision information) in sensor data used in the positioning engine layer 226, such as confidence information associated with radar, lidar, and / or camera sensor data. The locations of objects and features within the map data may have varying confidence levels provided by safety and / or confidence information within a map database or a linked database, so the map fusion and arbitration layer 230 may consider such information and the confidence in the sensor data when developing arbitrated map location information. For example, the map fusion and arbitration layer 230 may convert latitude and longitude information from a GPS into a location within a surface map of roads contained in a map database and compare such location with information received from radar, lidar, and / or camera sensors that may identify and locate objects and features associated with roads in the map data.

[0081] Similar to the location information in some map objects and features and sensor accuracy and precision, GPS positioning includes some errors, so the map fusion and arbitration layer 230 can be used to determine the best guess position of the vehicle within the road based on arbitration between GPS coordinates, sensor data, and map data regarding objects and features in and near the road. For example, although the GPS coordinates may place the vehicle near the middle of a two-lane road in the sensor data, the map fusion and arbitration layer 230 may determine that the vehicle is most likely aligned with a lane of travel that is consistent with the direction of travel based on the direction of travel. The map fusion and arbitration layer 230 can pass the arbitrated map position information to the sensor fusion and RWM management layer 236.

[0082] The route planning layer 232 may utilize sensor data and input from an operator or dispatcher to plan a route to be followed by the vehicle 102 to a particular destination. The route planning layer 232 may pass map-based location information to the sensor fusion and RWM management layer 236. However, other layers, such as the sensor fusion and RWM management layer 236, do not require the use of a priori maps. For example, other stacks may operate and / or control a vehicle based solely on sensory data without a provided map, building concepts of lanes, boundaries, and local maps as the sensory data is received.

[0083] In an embodiment including an ASIL mode assessment layer 234, the processing layer may use safety and / or confidence information about nearby objects and features identified in the map database 228 to select an appropriate ADS driving mode. In some embodiments, the ASIL mode assessment layer 234 may determine whether the current autonomous or semi-autonomous driving mode is consistent or appropriate with the nearby objects and features in the driving environment in view of the safety and / or confidence information about the nearby objects and features in the driving environment. For example, the ASIL mode assessment layer 234 may compare the ASIL safety level information associated or linked to the nearby objects and features in the map database, and initiate an action to change the driving mode to an ASIL level that is compatible or consistent with the ASIL safety level information of the nearby objects and features.

[0084] The sensor fusion and RWM management layer 236 may receive data and outputs generated by the radar and / or lidar perception layer 222, the camera perception layer 224, the map fusion and arbitration layer 230, the route planning layer 232, and the ASIL mode assessment layer 234, and use some or all of such inputs to estimate or refine the position and state of the vehicle 102 relative to the road, other vehicles on the road, and other objects in the vicinity of the vehicle 100. For example, the sensor fusion and RWM management layer 236 may combine image data from the camera perception layer 224 with arbitration map position information from the map fusion and arbitration layer 230 to refine the determined position of the vehicle within the traffic lane. As another example, the sensor fusion and RWM management layer 236 may combine object recognition and image data from the camera perception layer 224 with object detection and ranging data from the radar and / or lidar perception layer 222 to determine and refine the relative position of other vehicles and objects in the vicinity of the vehicle. As another example, the sensor fusion and RWM management layer 236 may receive information about the location and direction of travel of other vehicles from V2X communications (such as via a CAN bus) and combine this information with information from the radar and / or lidar perception layer 222 and the camera perception layer 224 to refine the location and motion of other vehicles. The sensor fusion and RWM management layer 236 may output the refined location and state information of the vehicle 100 and the refined location and state information of other vehicles and objects in the vicinity of the vehicle to the motion planning and control layer 238 and / or the behavior planning and prediction layer 240.

[0085] As a further example, the sensor fusion and RWM management layer 236 can use dynamic traffic control instructions to direct the vehicle 102 to change speed, lane, direction of travel, or other navigation elements, and combine this information with other received information to determine refined position and state information. The sensor fusion and RWM management layer 236 can output the refined position and state information of the vehicle 102 and the refined position and state information of other vehicles and objects in the vicinity of the vehicle 100 to the motion planning and control layer 238, the behavior planning and prediction layer 240, and / or devices remote from the vehicle 102, such as data servers, other vehicles, etc., via wireless communication (such as through a C-V2X connection, other wireless connections, etc.).

[0086] As yet a further example, the sensor fusion and RWM management layer 236 may monitor perception data from various sensors (such as perception data from the radar and / or lidar perception layer 222, the camera perception layer 224, other perception layers, etc.) and / or data from one or more sensors themselves to analyze conditions in the vehicle sensor data. The sensor fusion and RWM management layer 236 may be configured to detect conditions in the sensor data, such as sensor measurements being at, above, or below a threshold, certain types of sensor measurements occurring, etc., and may output the sensor data as part of refined position and state information of the vehicle 102 that is provided to the behavior planning and prediction layer 240 and / or devices remote from the vehicle 100 (such as data servers, other vehicles, etc.) via wireless communications (such as through a C-V2X connection, other wireless connections, etc.).

[0087] The behavior planning and prediction layer 240 of the autonomous vehicle system stack 220 can use the refined position and state information of the vehicle 102 and the position and state information of other vehicles and objects output from the sensor fusion and RWM management layer 236 to predict the future behavior of other vehicles and / or objects. For example, the behavior planning and prediction layer 240 can use such information to predict the future relative positions of other vehicles based on the own vehicle position and speed and the positions and speeds of other vehicles in the vicinity of the vehicle. Such predictions can take into account information from map data and route planning to anticipate changes in relative vehicle positioning along the road as the host vehicle and other vehicles. The behavior planning and prediction layer 240 can output other vehicle and object behavior and position predictions to the motion planning and control layer 238. Additionally, the behavior planning and prediction layer 240 can use the object behavior in conjunction with the position predictions to plan and generate control signals for controlling the motion of the vehicle 102. For example, based on the route planning information, the refined position in the road information, and the relative position and movement of other vehicles, the behavior planning and prediction layer 240 may determine that the vehicle 102 needs to change lanes and accelerate, such as to maintain or achieve a minimum spacing with other vehicles and / or to prepare for a turn or exit. As a result, the behavior planning and prediction layer 240 may calculate or otherwise determine the steering angle of the wheels and the change to the throttle setting, which will be commanded to the motion planning and control layer 238 and the ADS vehicle control unit 242 along with various such parameters necessary to implement such lane changes and accelerations. One such parameter may be a calculated steering wheel command angle.

[0088] The motion planning and control layer 238 may receive data and information output from the sensor fusion and RWM management layer 236, safety and / or confidence information from the map database 232 (or separate and linked databases), and other vehicle and object behavior and position predictions from the behavior planning and prediction layer 240, and use this information to plan and generate control signals for controlling the motion of the vehicle 102, and verify that such control signals meet the safety requirements of the vehicle 100. For example, based on the route planning information, the refined position in the road information, and the relative position and motion of other vehicles, the motion planning and control layer 238 may verify and pass various control commands or instructions to the ADS vehicle control unit 242.

[0089] The ADS vehicle control unit 242 may receive commands or instructions from the motion planning and control layer 238 and convert such information into mechanical control signals for controlling the wheel angles, brakes, and throttle of the vehicle 100. For example, the ADS vehicle control unit 242 may respond to a calculated steering wheel command angle by transmitting a corresponding control signal to a steering wheel controller.

[0090] In various embodiments, the wireless communication subsystem 218 can communicate with other V2X system participants via wireless communication links to send sensor data, positioning data, vehicle data, and data collected by onboard sensors about the vehicle's surroundings. Other V2X system participants can use such information to update stored sensor data for relay to other V2X system participants.

[0091] In various embodiments, the vehicle ADS processing system stack 220 may include functionality to perform safety checks or oversight of various commands, plans, or other decisions of various layers that may affect the safety of the vehicle and its occupants. Such safety checks or oversight functionality may be implemented within a dedicated layer or distributed between various layers and included as part of the functionality. In some embodiments, various safety parameters may be stored in a memory, and the safety check or oversight functionality may compare the determined values ​​(e.g., relative spacing to nearby vehicles, distance to the centerline of the road, etc.) with the corresponding safety parameters, and issue a warning or command if the safety parameters are violated or about to be violated. For example, a safety or supervisory function in the behavior planning and prediction layer 240 (or in a separate layer) may determine a current or future separation distance between another vehicle (as defined by the sensor fusion and RWM management layer 236) and the vehicle (e.g., based on a world model refined by the sensor fusion and RWM management layer 236), compare the separation distance to a safe separation distance parameter stored in memory, and issue a command to the motion planning and control layer 238 to accelerate, decelerate, or turn if the current or predicted separation distance violates the safe separation distance parameter. As another example, a safety or supervisory function in the motion planning and control layer 238 (or in a separate layer) may compare a determined or commanded steering wheel command angle to a safe wheel angle limit or parameter, and issue an override command and / or alarm in response to the commanded angle exceeding the safe wheel angle limit.

[0092] Some safety parameters stored in memory may be static (i.e., do not change over time), such as maximum vehicle speed. Other safety parameters stored in memory may be dynamic in that they are continuously or periodically determined or updated based on vehicle state information and / or environmental conditions. Non-limiting examples of safety parameters include: maximum safe speed, maximum brake pressure, maximum acceleration, and safe wheel angle limits, all of which may vary depending on road and weather conditions.

[0093] Figure 3A An example SOC architecture of a processing device system on a chip (SOC) 300 suitable for implementing various embodiments in a vehicle is illustrated. Figures 1A to 3A, the processing device SOC 300 may include several heterogeneous processors, such as a digital signal processor (DSP) 303, a modem processor 304, an image and object recognition processor 306, a mobile display processor 307, an application processor 308, and a resource and power management (RPM) processor 317. The processing device SOC 300 may also include one or more coprocessors 310 (e.g., a vector coprocessor) connected to one or more of the heterogeneous processors 303, 304, 306, 307, 308, 317. Each of these processors may include one or more cores and an independent / internal clock. Each processor / core may perform operations independently of other processors / cores. For example, the processing device SOC 300 may include a processor that executes a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor that executes a second type of operating system (e.g., Microsoft Windows). In some embodiments, the application processor 308 may be the main processor, central processing unit (CPU), microprocessor unit (MPU), arithmetic logic unit (ALU), etc. of the SOC 300. Graphics processor 306 may be a graphics processing unit (GPU).

[0094] The processing device SOC 300 may include analog and custom circuits 314 for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations, such as processing encoded audio and video signals for rendering in a web browser. The processing device SOC 300 may also include system components and resources 316, such as voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components for supporting processors and software clients (e.g., web browsers) running on the computing device.

[0095] The processing device SOC 300 may also include a dedicated circuit for camera actuation and management (CAM) 305, which includes, provides, controls and / or manages the operation of one or more cameras (e.g., main camera, web camera, 3D camera, etc.), video display data from camera firmware, image processing, video pre-processing, video front end (VFE), embedded JPEG, high-definition video codec, etc. The CAM 305 may be an independent processing unit and / or include an independent or internal clock.

[0096] In some embodiments, the image and object recognition processor 306 may be configured with processor executable instructions and / or specialized hardware configured to perform image processing and object recognition analysis involved in various embodiments. For example, the image and object recognition processor 306 may be configured to perform operations of processing images received from the camera via the CAM 305 to recognize and / or identify other vehicles, and otherwise perform the functions of the camera perception layer 224 as described. In some embodiments, the processor 306 may be configured to process radar or lidar data and perform the functions of the radar and / or lidar perception layer 222 as described.

[0097] System components and resources 316, analog and custom circuits 314, and / or CAM 305 may include circuits to interface with peripheral devices such as cameras, radars, lidar electronic displays, wireless communication devices, external memory chips, etc. Processors 303, 304, 306, 307, 308 may be interconnected to one or more memory elements 312, system components and resources 316, analog and custom circuits 314, CAM 305, and RPM processor 317 via interconnect / bus module 324, which may include a reconfigurable logic gate array and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). Communications may be provided by advanced interconnects, such as a high-performance network-on-chip (NoC).

[0098] The processing device SOC 300 may further include an input / output module (not illustrated) for communicating with resources external to the SOC, such as a clock 318 and a voltage regulator 320. Resources external to the SOC (e.g., clock 318, voltage regulator 320) may be shared by two or more internal SOC processors / cores (e.g., DSP 303, modem processor 304, graphics processor 306, application processor 308, etc.).

[0099] In some embodiments, the processing device SOC 300 may be included in a control unit (e.g., 140) for use in a vehicle (e.g., 100). The control unit may include a communication link for communicating with a telephone network (e.g., 180), the Internet, and / or a network server (e.g., 184), as described.

[0100] The processing device SOC 300 may also include additional hardware and / or software components suitable for collecting sensor data from sensors, including motion sensors (e.g., accelerometers and gyroscopes of an IMU), user interface elements (e.g., input buttons, touch screen displays, etc.), microphone arrays, sensors for monitoring physical conditions (e.g., position, direction, motion, orientation, vibration, pressure, etc.), cameras, compasses, GPS receivers, communication circuits (e.g., WLAN, WiFi, etc.) and other well-known components of modern electronic devices.

[0101] Figure 3B is a component block diagram illustrating elements of a vehicle ADS 330 configured according to various embodiments. Figures 1A to 3B The vehicle ADS 330 may include a vehicle ADS processing system 204 of a vehicle (eg, 102 ), which may be configured to communicate with a roadside unit 112 and / or a cellular network base station 110 .

[0102] The vehicle ADS processing system 204 may include one or more processors 205, memory 206, radio module 218, and other components. The vehicle ADS processing system 204 may include multiple hardware, software, and / or firmware components that operate together to provide the functionality attributed herein to the processor 205.

[0103] The memory 206 may include a non-transitory storage medium that electronically stores information. The electronic storage medium of the memory 206 may include a system storage device provided integrally with the vehicle ADS processing system 204 (i.e., substantially non-removable) and / or one or both of a removable storage device that is removably connected to the vehicle ADS processing system 204 via, for example, a port (e.g., a universal serial bus (USB) port, a firewire port, etc.) or a drive (e.g., a disk drive, etc.). In various embodiments, the memory 206 may include a charge-based storage medium (e.g., EEPROM, RAM, etc.), a solid-state storage medium (e.g., a flash drive, etc.), an optically readable storage medium (e.g., an optical disk, etc.), a magnetically readable storage medium (e.g., a tape, a magnetic hard drive, a floppy disk drive, etc.), and / or other electronically readable storage media. The memory 206 may include one or more virtual storage resources (e.g., a cloud storage device, a virtual private network, and / or other virtual storage resources). The memory 206 may store software algorithms, information determined by the processor 205, information received from one or more other vehicles 220, information received from the roadside unit 112, information received from the base station 110, and / or other information that enables the vehicle ADS processing system 204 to function as described herein.

[0104] The processor 205 may include one of a plurality of local processors that may be configured to provide information processing capabilities in the vehicle ADS processing system 204. Thus, the processor 205 may include one or more of the following: a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although the processor 205 is Figure 3B205 is shown as a single entity, but this is for illustrative purposes only. In some embodiments, processor 205 may include multiple processing units. These processing units may be physically located within the same device, or processor 205 may represent the processing functionality of multiple devices distributed within the vehicle and operating in coordination.

[0105] The vehicle ADS processing system 204 may be configured by machine-readable instructions 332, which may include one or more instruction modules. The instruction modules may include computer program modules. In various embodiments, the instruction modules may include one or more of a map data access module 334, a confidence information access module 336, one or more autonomous driving modules 338, a sensed object and feature map data upload module 340, and / or other modules.

[0106] The map data access module 334 may be configured to access a map database that may be stored in the memory 206 (or other vehicle memory) to obtain map data regarding objects and / or features in the vicinity of the vehicle.

[0107] The confidence information access module 334 may be configured to access a map database or other database indexed to map data (which may be stored in the memory 206 (or other vehicle memory)) to obtain safety and / or confidence information associated with the map data relating to objects and / or features near the vehicle. In some embodiments, the confidence information access module 334 may access a memory within the vehicle storing the confidence information. In some embodiments, the confidence information access module 334 may access a network accessible memory, such as a server storing the confidence information.

[0108] The autonomous driving system module 338 may be configured to perform various functions of autonomous and semi-autonomous driving of the vehicle ADS, including using the confidence information of the processor when the processor performs autonomous or semi-autonomous driving actions and other operations of various embodiments. In some embodiments, the autonomous driving system module 338 may use the confidence information by weighting the corresponding map data and using the weighted map data in the driving operation. In some embodiments, the autonomous driving system module 338 may take one or more actions to change the current autonomous driving mode to a driving movement consistent with the safety level and / or confidence level of the map data about nearby objects and features.

[0109] The sensed object and feature map data upload module 340 may be configured to identify objects and features detected by the vehicle sensors that should be uploaded for consideration in generating map data and uploading the information to a remote computing device along with confidence information. In some embodiments, a processor executing the sensed object and feature map data upload module 340 may obtain sensor data about one or more objects and / or features near the vehicle from the vehicle sensors, and determine whether the obtained sensor data about any object or feature near the vehicle is sufficiently different (e.g., differs by a threshold amount) from map data about the object or feature obtained from a map database to justify reporting the data to a remote computing device. When such objects and / or features are identified in the vehicle sensor data, the sensed object and feature map data upload module 340 may upload location information about the observed objects and / or features and confidence information about the information.

[0110] Processor 205 may be configured to execute modules 332 - 340 and / or other modules via software, hardware, firmware, some combination of software, hardware and / or firmware, and / or other mechanisms for configuring processing capabilities on processor 205 .

[0111] The description of the functionality provided by the different modules 332-340 is for illustrative purposes and is not intended to be limiting, as any of the modules 332-340 may provide more or less functionality than described. For example, one or more of the modules 332-340 may be eliminated, and some or all of their functionality may be provided by other modules in the modules 332-340. As another example, the processor 205 may be configured to execute one or more additional modules that may perform some or all of the functionality attributed below to one of the modules 332-340.

[0112] Figure 4A and Figure 4B The following examples illustrate different forms of objects and features that may exist in the driving environment and be represented in map data with safety and / or confidence information. Figure 4A , objects and features near the vehicle 102 may include a curb 404 defining the edge of a road, a traffic lane 406 that may be divided by a dividing line 408. Such objects may also be associated with relevant descriptions, such as the height or nature, width, and / or roadbed structure (e.g., asphalt, cement, dust, etc.) of the curb 404 and the color (e.g., white, yellow, red, etc.) or nature (e.g., solid line, dashed line, double line, etc.) of the dividing line 408. As another example, Figure 4A A road feature is illustrated in the form of a merging of lanes 412, signaled by a yield sign 412 and arrow 414 in the road.

[0113] Figure 4A Also illustrated are examples of features that can be included in map data in the form of zones or regions within a map, where certain driving conditions, such as ASIL safety and / or autonomous driving levels, may be indicated. For example, the figure shows that a vehicle 102 is within a driving area 420 including four driving lanes, such as typically in a large highway environment. In such a driving area 420, autonomous driving may be feasible because the traffic will be similar, there are no turns or cross traffic and no pedestrians enter the road. Therefore, the driving area 420 provides examples of road features that may be assigned or associated with ASIL safety or autonomous driving L4 or L5, indicating that a vehicle equipped with ADS may operate completely autonomously without driver attention. In various embodiments, an indication of an L4 or L5 autonomous driving level may be included in the map data within a map database, or may be provided in a separate database that is linked to or indexed into the map data, so that the vehicle ADS processing system may be notified when the vehicle is in such a position.

[0114] Similarly, driving zone or area 422 involves a merge of four lanes of traffic to two lanes of traffic, which may be challenging for autonomous driving systems (and human drivers) due to the unpredictable nature of the merging traffic that will occur within the area. Therefore, driving zone 422 provides an example of road features that may be assigned or associated with ASIL safety or autonomous driving L2 or L3, indicating that a vehicle equipped with an ADS should not operate fully autonomously, and that the driver should be focused on the road and on standby to control the vehicle or begin steering the vehicle (if not controlled). In some embodiments, the ADS of a vehicle 102 operating in a fully autonomous driving mode in area 420 may receive safety information from a map database or a linked database regarding area 422 as it approaches, and in response to receiving the L2 or L3 safety information, warn the driver that a change in driving mode is beginning, and transition to an appropriate semi-autonomous driving mode upon entering area 422. Once through the merge area 422, the road may again be suitable for autonomous driving, in which case the ADS may receive safety information for the area from the map database or a linked database as the vehicle 102 approaches the area 424, and notify the driver that the vehicle may transition to autonomous driving mode. Such a transition from manual or semi-manual driving to fully autonomous driving mode may require driver consent.

[0115] Figure 4AThe objects and features illustrated in are examples of permanent objects and features that are unlikely to change over time. Therefore, the age of the information about objects and features 404 to 414 in the map database may not be relevant. For such objects and features, the safety and / or confidence information provided in the map database or a linked database may not include information about the date or age of the associated map data.

[0116] However, some types of objects and features that may be included in map data may be temporary or change over a period of time, such that the date or age of the associated map data is relevant and may be included in the safety and / or confidence information. Figure 4B Several examples of objects and features that may change over time are illustrated, and their date or age information may be included in the confidence information. For example, while road curbs 404, driving lanes 406, and dividing lines 408 are likely to remain as defined in the map data for a long time, some road structures 430 that may be used as navigation points for vehicle sensors (e.g., radar, lidar, cameras) may be modified or removed over time. As another example, potholes 432 are road features that are relevant to autonomous driving but may be temporary. The location of the pothole may be reported to a computing device that generates map data by a vehicle equipped with an ADS implementing various embodiments. The location of the pothole may then be distributed to all autonomous vehicles via an updated map database. Because potholes may be repaired at some point, the date when the pothole was first reported or the age of the pothole location data may be included in the confidence information associated with the pothole map data included in the map database or linked database. Providing this information may enable a vehicle ADS operating in an autonomous driving mode to avoid confusion when approaching the location of a pothole 432 that is not observed by vehicle sensors (eg, cameras) if the age of the pothole location data exceeds a threshold.

[0117] Figure 4B Examples of temporary objects and features that may be associated with road construction projects are also illustrated. For example, a map database may include the location of traffic cones 436 or obstacles blocking a line of travel, as well as construction areas 434. Because highway construction is often temporary, the age of traffic cones and construction area location data may be included in the confidence information associated with the data included in the map database or a linked database. As an example of the value of such information, the vehicle ADS processing system may consider the age of such construction area data when performing route planning even before approaching the area, such as selecting to travel along a path that includes such construction if the age of the construction location data exceeds a threshold.

[0118] Figure 4A and 4BAlso illustrated are examples of objects and features that may be associated with different confidence levels due to the manner in which location information may be obtained. Figure 4A In the example illustrated in , road features of curb 404, driving lane 406, dividing line 408, yield sign 410, zone 412, and street marking 414 may be defined by very accurate surveying and therefore have a high level of confidence in the location information. Figure 4B In the example illustrated in , structures on the side of the road (such as antenna 430) and temporary road features (such as potholes 432, construction areas 434, and temporary obstacles 436) may be detected and the location information obtained by vehicle sensors that are less accurate and subject to various errors and limited accuracy. Therefore, the confidence information included in the map database or linked database for such objects and features 430, 432, 434, 436 may reflect a confidence level consistent with the accuracy, precision, and overall confidence of the sensors and methods used to determine the location.

[0119] FIG. 4C to FIG. 4E Three non-limiting examples of data structures that may be used in various map-related databases for implementing various embodiments are illustrated. In some embodiments, a map database (e.g., Figure 4C ) or a confidence information database (e.g., Figure 4D and Figure 4E ) may be stored in a network location accessible to a vehicle equipped with the ADS, such as to facilitate downloading the database prior to commencing a journey or during a journey. In some embodiments, a map database including confidence information (e.g., Figure 4C ) or a confidence information database (e.g., Figure 4D and 4E ) may be sent or otherwise distributed to ADS-equipped vehicles, such as via over-the-air updates. For example, a map database or a confidence information database including confidence information may be accessed and / or received by an ADS-equipped vehicle from a base station 110 via a cellular network communication link 122, and / or accessed and / or received from a roadside unit 112 via a V2X communication link 124.

[0120] In some embodiments, safety and / or confidence information (including date or age information) may be included as data fields within the data records for each object and feature included within the map database. Figure 4CA non-limiting example of such a data record 440 is illustrated, where the data record may include location data 442 for a given object or feature, such as in the form of latitude and longitude coordinates as illustrated, or other forms of coordinates (e.g., map coordinate references). The map data record may also include the name or index of the object or feature and description (e.g., color) information 444. In a conventional map database, these information elements (i.e., 442, 444) may be all of the information included in a given data record. In some embodiments, additional fields may be added to provide safety and confidence information, such as a data field for safety information (such as an ASIL driving level "DL" 446), a data field "CL" 448 for confidence information, and a data field 450 for the age or date of the data record 440. Other data fields may also be included in the data record to provide additional forms of safety and / or confidence information.

[0121] In some embodiments, safety and / or confidence information (including date or age information) may be stored and made available in a separate database containing data records linked to specific data records in the map database. Figure 4D A data structure is illustrated in which safety and / or confidence information is provided in a separate database, wherein the information stored in a data record 462 is linked to the location data 442 of a given map data record 460. In such an example, the vehicle ADS processing system may obtain safety and / or confidence information from a separate confidence information database by using the location information 442 in a given map database data record 460 as lookup information to identify the corresponding data record 462 in the separate confidence information database. Such a data record 462 may similarly include data fields for ASIL driving level "DL" 446, confidence information "CL" 448, and a data field 450 for age or date, as well as other information.

[0122] Figure 4EAnother non-limiting example of an exemplary data structure in which safety and / or confidence information is provided in a separate database, wherein the information stored in a data record 472 is linked to an index 452 included in a given map data record 470. In this exemplary embodiment, a data record 470 in a map database may include an index 452 for linking the data record to a separate confidence information database. With the data record 472 indexed in this manner, a vehicle ADS processing system accessing a given object or feature data record 470 in a map database may obtain safety and / or confidence information by using the index 452 included in the data record to find the corresponding data record 472 in a separate confidence database. Such a data record 472 may similarly include data fields for ASIL driving level "DL" 446, confidence information "CL" 448, and data fields 450 for age or date, as well as other information.

[0123] Figure 5A is a process flow diagram of an example method 500a performed by a processor of an autonomous driving system of a vehicle for using secure and / or confidential information in or associated with map data when performing autonomous driving functions in accordance with various embodiments. FIG. 5B to FIG. 5H is a process flow diagram of example operations 500b through 500h that may be performed as part of using secure and confidential information in or associated with map data when performing autonomous driving functions as described, according to some embodiments. Figures 1A to 5H , method 500a and operations 500b to 500h may be performed by a processor of a vehicle ADS processing system (e.g., 205, 300) or other vehicle processors (e.g., 104, 204, 205, 220, 300), which may be implemented in hardware elements, software elements, or a combination of hardware elements and software elements (collectively referred to as "vehicle processor").

[0124] In box 502, the vehicle processor may perform operations including accessing map data about objects or features near the vehicle from a map database accessible by the processor. In some embodiments, the vehicle processor may access the map database to obtain all data records of objects and features within a threshold distance of the current position of the vehicle. As described herein, the vehicle processor may use various sensors to maintain position information, including GPS coordinate data, dead reckoning, and visual navigation based on the relative position of objects and features near the vehicle using information stored in the accessed map database records. For example, when the vehicle is advancing on the road, the vehicle processor may continuously or periodically access the map database to obtain data records of objects and features within a threshold distance in front of the vehicle, so that such data is accessed before the vehicle reaches the object or feature to give the vehicle ADS processing system time for route planning and object avoidance processing. The component for performing the operation of box 502 may include a memory (e.g., 206) storing the map database and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) executing the map data access module 334.

[0125] In box 504, the vehicle processor may perform operations including accessing, by the processor, confidence information associated with map data regarding an object or feature near the vehicle. In some embodiments, the confidence information may be an Automotive Safety Integrity Level (ASIL) autonomous driving level near the object or feature. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include an indication related to the accuracy of the map data regarding the object or feature. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include an indication related to the reliability of the map data regarding the object or feature. Additionally or alternatively, in some embodiments, the safety and / or confidence information may include a statistical score indicating the accuracy of the map data regarding the object or feature (e.g., a statistical measure of accuracy, an F1 score, etc.). Additionally or alternatively, in some embodiments, the safety and / or confidence information may include the age or freshness of the map data regarding the object or feature. For example, as described in reference to FIG. 4C to FIG. 4E As described, the confidence information may be stored in a map database as part of the object and feature data record or may be stored in a separately linked database, and the processor may obtain the confidence information using a position or index value obtained from the map data record obtained in block 502. The means for performing the operations of block 504 may include a memory (e.g., 206) storing a map database and a processor (e.g., 205, 300) of a vehicle ADS processing system (e.g., 104, 204) executing the confidence information access module 336.

[0126] In box 506, the vehicle processor may perform operations including using confidence information by the processor when performing autonomous or semi-autonomous driving actions. In some embodiments, the vehicle ADS processing system may adjust the level of autonomous driving performed by the system in accordance with the safety and confidence information (e.g., switching to a lower level of autonomous driving consistent with the safety information), taking confidence information in the object or feature map information into account as part of sensor fusion and navigation, route planning, object avoidance, etc. In some embodiments, the vehicle processor of the autonomous driving system may interrupt autonomous or semi-autonomous driving functions, functionalities, features, or actions. The components for performing the operations of box 506 may include the vehicle network 210, and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) that executes the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0127] FIG. 5B to FIG. 5H 5 is a process flow diagram of example operations 500b through 500h that may be performed as part of using secure and confidential information in or associated with map data when performing autonomous driving functions as described, according to some embodiments. Operations 500b through 500h may be performed by a processor of a vehicle ADS processing system (e.g., 205) or other vehicle processors (e.g., 104, 204, 205, 220, 300), which may be implemented in hardware elements, software elements, or a combination of hardware elements and software elements (collectively, "vehicle processors").

[0128] Figure 5BOperation 500b that can be performed by a vehicle ADS processing system for using safety and confidentiality information in or associated with map data when performing autonomous driving functions according to some embodiments is illustrated. In box 510, the vehicle processor may perform operations including obtaining confidence information from a map database by the processor. In some embodiments, the confidence information may be included in a data element associated with each object or feature in the map database. In some embodiments, the confidence information may be included as metadata linked to a data element associated with each object or feature in the map database. Thereafter, the vehicle processor may perform operations in box 506 of method 500a as described. In various embodiments, the information in the map database is obtained from one or more of a system memory, a remote computing device, or another vehicle, and the confidence information included in the map data may depend on the source of the map information. The components for performing the operations of box 510 may include a memory (e.g., 206) storing a map database and a processor (e.g., 205, 300) of a vehicle ADS processing system (e.g., 104, 204) executing a confidence information access module 336.

[0129] Figure 5C Operations 500c are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. Figures 1A to 5C , after the operations in block 502 as described, in block 512, the vehicle processor may perform operations including obtaining confidence information based on the location of the object or feature from a data structure other than a map database accessible by the processor. For example, as described in reference Figure 4D and Figure 4E As described, the vehicle processor may use information obtained from the map database (such as the location of the object or feature or the index included in the map data record of the object or feature) to look up the confidence information in the confidence database. Thereafter, the vehicle processor may perform the operations in block 506 of method 500a as described. The means for performing the operations of block 512 may include a memory (e.g., 206) storing the map database and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) executing the confidence information access module 336.

[0130] Following the operations in block 512 , the vehicle processor may perform the operations in block 506 as described.

[0131] Figure 5DOperations 500d are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. FIG. 1A to FIG. 5D , after the operations in box 504, in box 514, the vehicle processor of the autonomous driving system may perform operations including applying weights to the accessed map data about the object or feature based on the confidence information by the vehicle processor. For example, the vehicle processor may weight low-confidence map data so that in map fusion operations (e.g., in the map fusion and arbitration layer 230) and / or road world model management (e.g., in the sensor fusion and RWM management layer 236), the map data may be applied or addressed in conjunction with the sensor data based on the accuracy, precision, and / or reliability of the data used for motion planning and control. The components for performing the operations of box 504 may include a memory (e.g., 206) storing a map database and a processor (e.g., 205, 300) of a vehicle ADS processing system (e.g., 104, 204) executing the autonomous driving system module 338.

[0132] In box 516, the vehicle processor may perform operations including using weighted map data about objects or features by the processor while performing path planning, object avoidance or steering autonomous driving actions. For example, the vehicle processor may assign a large weight (e.g., 1) to objects and feature data for which confidence information indicates a significant confidence in the accuracy, precision and / or reliability of the data. As another example, the vehicle processor may assign a weight less than one to objects and feature data for which confidence information indicates the degree of inaccuracy, imprecision and / or unreliability involved in the map data. As another example, the vehicle processor may assign a weight less than one to old and therefore possibly no longer correct objects and feature data. The components for performing the operations of box 516 may include a memory (e.g., 206) storing a map database and a processor (e.g., 205, 300) of a vehicle ADS processing system (e.g., 104, 204) executing the autonomous driving system module 338.

[0133] Figure 5E Operations 500e are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. Figures 1A to 5E, after the operations in box 504, in box 518, the vehicle processor may perform operations including changing the autonomous driving mode of the vehicle implemented by the vehicle processor based on confidence information about objects or features near the vehicle. For example, when making steering decisions, collision avoidance maneuvers, path planning, and other autonomous driving functions, the vehicle processor may evaluate map data about objects and features in front of the vehicle based on weights, for example, in a manner similar to vehicle sensor data. The components for performing the operations of box 518 may include the vehicle network 210, and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) that executes the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0134] Fig. 5F Operations 500f are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. Figures 1A to 5F , after the operations in box 504, in box 520, the vehicle processor may perform operations including changing the autonomous driving mode of the vehicle implemented by the vehicle processor to a driving mode that is compatible with the confidence information about the objects or features near the vehicle. For example, if the vehicle ADS processing system is operating in a fully autonomous mode (e.g., L4 or L5) and the safety information associated with the map data of the objects, features, or areas near the vehicle indicates that the driving environment (e.g., the nature of the road, pedestrian conditions, ongoing construction, etc.) indicates that the area is unsafe for autonomous operation, the vehicle processor may take action to transition to a driver assistance or driver takeover operating mode. For example, the vehicle processor may emit a warning sound and / or display a notification to the driver that he / she needs to pay attention to, and begin the process of switching operating modes in a safe manner. As another example, if the vehicle ADS processing system is operating in a driver control mode or a driver assistance mode (e.g., L2 or L3) and the safety information associated with the map data of the object, feature, or area near the vehicle indicates that the driving environment (e.g., the nature of the road) indicates that the area is now safe for autonomous operation, then the vehicle processor may notify the driver that it is safe to change the vehicle operating mode accordingly. The components for performing the operations of block 520 may include the vehicle network 210, and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) executing the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0135] Figure 5GOperations 500g are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. Figure 1A to Figure 5G , after the operation in box 504, in box 522, the vehicle processor may perform operations including notifying the driver of the need to participate in the driving of the vehicle in response to determining that the confidence information about the object or feature near the vehicle does not support the fully autonomous driving mode. For example, the vehicle processor may sound a warning and / or display a notification to the driver that his / her attention is required, and the vehicle will change the operating mode accordingly when receiving a confirmation from the driver or the driver holds the steering wheel. The components for performing the operation of box 522 may include the vehicle network 210, and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) that executes the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0136] In box 524, the vehicle processor may perform operations including changing the autonomous driving mode of the vehicle implemented by the ADS after notifying the driver. For example, if the vehicle ADS processing system is operating in a fully autonomous mode (e.g., L4 or L5) and the safety information associated with the map data of the objects, features, or areas near the vehicle indicates that the driving environment is unsafe for autonomous operation, the vehicle processor may transition to a driver assistance or driver takeover operating mode based on the safety information after receiving confirmation or detecting that the driver has taken control. The components for performing the operations of box 524 may include the vehicle network 210, and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) that executes the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0137] In some examples, the confidence information about the object or feature may be confidence information about objects and features within a defined area. In such examples, in block 526, the vehicle processor may perform operations including changing the autonomous driving mode of the vehicle implemented by the ADS to a driving mode compatible with the confidence information while the vehicle is within the defined area.

[0138] Figure 5H Operations 500h are illustrated that may be performed by a vehicle processor for using security and confidential information in or associated with map data when performing autonomous driving functions in accordance with some embodiments. Figures 1A to 5H, after the operations in box 504, in box 526, the vehicle processor may perform operations including obtaining, by the vehicle processor, sensor data about objects or features in the vicinity of the vehicle from the vehicle sensors. For example, the sensor fusion and RWM management layer 236 executed in the vehicle processor may compare the object or feature relative position and / or features with the information determined by the map fusion and arbitration layer to identify whether the information obtained from the vehicle sensors (e.g., radar, lidar, camera, etc.) is inconsistent with the map data sufficiently to indicate that the map data is erroneous or misses the detected feature object. The components for performing the operations of box 526 may include the vehicle sensors 214, the vehicle network 210, and a processor (e.g., 205, 300) executing a vehicle ADS processing system (e.g., 104, 204) equipped with the ADS processing system stack 220 and / or the autonomous driving system module 338.

[0139] In box 528, the vehicle processor may perform operations including determining whether the sensor data obtained about an object or feature near the vehicle differs from the map data about the object or feature obtained from the map database by a threshold amount. For example, the sensor fusion and RWM management layer 236 executed in the vehicle processor may determine whether the difference between the information obtained from the vehicle sensor and the map data has a magnitude indicating that the sensor data should be reported to the computing device that maintains the map data. Such magnitudes may be in the form of one or more thresholds of differences and / or may be in the form of a table indicating the type and magnitude of inconsistencies that warrant the sensor data to be transmitted to the computing device that maintains the map data. The components for performing the operations of box 528 may include the vehicle network 210 and a processor (e.g., 205, 300) of the vehicle ADS processing system (e.g., 104, 204) that executes the sensed object and feature map data upload module 340.

[0140] In box 530, the vehicle processor may perform operations including uploading the obtained sensor data about objects or features near the vehicle to a remote computing device in response to determining that the obtained sensor data differs from map data about the objects or features obtained from a map database by at least a threshold amount, together with confidence information based on one or more of the type of sensor used to detect or classify the object or feature, the perceived quality of the object or feature achieved by the sensor, and / or the accuracy or precision of the sensor data. In some embodiments, the vehicle processor may upload the sensor data and confidence information to the computing device via a roadside unit (e.g., 112) using a V2X network (e.g., 124). In some embodiments and / or examples, the vehicle processor may upload the sensor data and confidence information to the computing device via a cellular wireless network (e.g., 122) such as a 5G network, via a base station (110). Means for performing the operations of block 530 may include the radio module 218 and a processor (eg, 205 , 300 ) of a vehicle ADS processing system (eg, 104 , 204 ) executing the sensed object and feature map data upload module 340 .

[0141] Fig. 6A is a process flow diagram of an example method 600a that may be performed by a computing device for including confidence information within map data used by autonomous and semi-autonomous driving systems in a vehicle in accordance with various embodiments. Figures 1A to 6A , method 600a may be performed by a computing device (eg, a server).

[0142] In block 602, a computing device may perform operations including receiving, by the computing device, information about objects or features from a source for inclusion in a map database, the information including a confidence measure in the information about the objects or features. Coordinate and descriptive information associated with each object or feature may be collected from a variety of sources, including surveys of roads and road features, overhead and satellite imagery, data from survey vehicles equipped to identify and locate objects and features and report data to the computing device, and data collected from ADS-equipped vehicles that report objects and features identified and located by the vehicle's sensors (e.g., radar, lidar, camera, etc.). Means for performing the operations of block 616 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a network access port 704 .

[0143] In box 604, the computing device may perform operations including using a confidence metric in the received information about the confidence of the object or feature to generate confidence information about the object or feature suitable for use by the vehicle autonomous and semi-autonomous driving system in autonomous or semi-autonomous driving operations. In some embodiments, the confidence information may include one or more of the following: an automotive safety integrity level (ASIL) autonomous driving level near the object or feature; an indication related to the accuracy of the map data about the object or feature; an indication related to the reliability of the map data about the object or feature; a statistical score indicating the accuracy of the map data about the object or feature; or the age or freshness of the map data about the object or feature. ASIL information about map objects and features may be received from an authority or service that assigns safe autonomous driving levels based on the accuracy of the map data and / or the challenges presented by road features.

[0144] In some embodiments, confidence information about the accuracy or precision of the map data may be established based on the accuracy and reliability of the source or method used to obtain the map data. The information used to generate the map database may come from various sources, including survey vehicles, remote side units of highway systems (e.g., cameras, traffic sensors, etc.), and vehicles on the highway. In some embodiments, the confidence assigned to objects and features in the map data in box 616 may depend on the source of the map data. For example, the confidence level assigned to or associated with objects and features in a map generated from a single vehicle received via V2X communication may be less than the confidence level assigned to or associated with objects and features in a map generated by map crowdsourcing.

[0145] Means for performing the operations of block 616 may include a computing device such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 and a volatile memory 702 .

[0146] In box 606, the computing device may perform operations including storing confidence information about objects or features in a manner that enables access by the vehicle autonomous and semi-autonomous driving systems in box 606. In some embodiments, the computing device may store the confidence information in the map data, such as in the same data record and in the location and description information of the corresponding object and feature data. In some embodiments, the computing device may store the confidence information in a database (e.g., a confidence information database) separate from the map database, such as in a data record with an index or information linking or indexing the confidence information record to the corresponding object and feature data record in the map database. In some embodiments, as part of the operation in box 606, the computing device may store a map database including confidence information, or store the confidence information database in a network location accessible to the vehicle equipped with the ADS, such as to download the database before starting the journey or during the journey. In some embodiments, as part of the operation in box 606, the computing device may send or otherwise distribute the map database including confidence information or the map database and the confidence information database to the vehicle equipped with the ADS, such as via an over-the-air update. Means for performing the operations of block 606 may include a computing device such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a large-capacity non-volatile memory 703 .

[0147] FIG. 6B to FIG. 6E 600b through 600e, which may be performed as part of obtaining secure and confidential information and storing the secure and confidential information in or associated with map data, according to some embodiments. Operations 600b through 600e may be performed by a computing device 700 (e.g., a remote server), which may be implemented in hardware elements, software elements, or a combination of hardware and software elements (collectively, "server").

[0148] In some embodiments, receiving information about an object or feature for inclusion in a map database (the information including a confidence measure in the information about the object or feature) may include receiving information from one or more vehicles including: the location of the object or feature; characteristics of the object or feature; and a confidence measure in the information about the location or characteristics of the object or feature.

[0149] See also Figure 6B, after the operations in block 602, in block 608, the computing device may perform operations that also include updating information about the object or feature in the map database based at least in part on the received confidence metric in the received information about the object or feature confidence. For example, the computing device may adjust the confidence information associated with the object or feature based on the confidence information received from various ADS-equipped vehicles. Thereafter, the computing device may perform the operations in block 604 as described. Means for performing the operations of block 608 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a large-capacity non-volatile memory 703 .

[0150] refer to Figure 6C , after the operations in block 604, in block 610, the computing device may perform operations for storing confidence information about the object or feature by including the confidence information as part of the location and other information about the object or feature in a map database provided to the vehicle for use in autonomous or semi-autonomous driving operations. For example, the computing device may store the confidence information as an additional data field within a data record corresponding to the map object or feature. Means for performing the operations of block 618 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a large-capacity non-volatile memory 703 .

[0151] See also Fig.6D , after the operations in block 604 as described, in block 612, the computing device may perform operations including storing the confidence information in a database separate from the map database associated with the location information of the object or feature. For example, the computing device may store the confidence information in a data record of a database or data table along with information or an index to a data record corresponding to the map object or feature in the map database that is also in the map database, so that a vehicle equipped with the ADS can find and access the confidence information. Means for performing the operations of block 618 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a large-capacity non-volatile memory 703 .

[0152] In box 614, the computing device may perform operations including providing a database to a vehicle for use in autonomous or semi-autonomous driving operations. In some embodiments, the computing device may periodically (or intermittently after completing updates) send the map database or only updates to the database to the ADS-equipped vehicle. For example, the computing device may use a V2X network (e.g., 124) to send updates to an ADS-equipped vehicle via a roadside unit (e.g., 112) or send an updated map database to an ADS-equipped vehicle. In some embodiments and / or examples, the computing device may use a cellular wireless network (e.g., 122) (such as a 5G network) to send updates to an ADS-equipped vehicle via a base station (110) or send an updated map database to an ADS-equipped vehicle. The components for performing the operations of box 614 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a network access port 704 .

[0153] refer to Fig. 6E In some instances, new or changed objects or features in a road may be identified and reported by many ADS-equipped vehicles. In such cases, information in such many reports may be used to determine a single map data set with a higher confidence than any one confidence information provided in the vehicle reports. In the operations in box 602a, the computing device may perform operations including receiving information about objects or features from multiple sources along with confidence metrics in the information about the objects or features. For example, when a frequently traveled road changes, many ADS-equipped vehicles may report information about the changed objects or features. In some embodiments, the computing device may accumulate reports from a certain number of vehicles (i.e., wait until enough vehicles have reported the change) before generating consolidated information based on all reports (e.g., by averaging). Components for performing the operations in box 602a may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701, a volatile memory 702, and a network access port 704. Thereafter, the computing device may perform the operations in block 604 as described.

[0154] After the operations in box 604, in box 616, the computing device may perform operations including determining a set of information about an object or feature based on information received from multiple sources and merged confidence information for the determined information about the object or feature. For example, the vehicle processor may perform a statistical analysis on the information to select an object or feature data set that best represents the information received from multiple sources, such as averaging or taking a weighted average using the confidence information associated with each reported sensor data as a weighting factor. When performing this statistical analysis, the computing device may determine a merged confidence level suitable for the merged map data based on the number of information sources used to generate the merged map data and the confidence information associated with each information source. For example, if a set of information about an object or feature stored in the map data is based on information received from a large number of sources (e.g., greater than 10), and various sources indicate high confidence in the reported information, the computing device may reflect the high confidence level in the one object or feature data set in the corresponding confidence information. Conversely, if a set of information about an object or feature stored in the map data is based on information received from a small number of sources (e.g., three or fewer), and the various sources indicate low confidence in the reported information, the computing device may reflect the low confidence level in the set of object or feature data in the corresponding confidence information. Means for performing the operations of block 616 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 and a volatile memory 702 .

[0155] In block 618, the computing device may perform operations including storing the combined confidence information of the determined set of information about the object or feature in a manner that enables access by the vehicle autonomous and semi-autonomous driving system for use in the autonomous or semi-autonomous driving operation in block 618. For example, similar to the operations in block 612, the computing device may store the confidence information in a data record of a database or data table along with information or an index to a data record that is also in a map database and corresponds to the map object or feature in the map database, so that the ADS-equipped vehicle can find and access the confidence information. Means for performing the operations of block 618 may include a computing device, such as Figure 7 The server illustrated in FIG. 7 includes a processor 701 , a volatile memory 702 , and a large-capacity non-volatile memory 703 .

[0156] Figure 7 is a component block diagram of a network computing device suitable for use with various embodiments. Figures 1A to 7 , various embodiments can be implemented on various computing devices (including but not limited to reference FIG. 6A to FIG. 6E The embodiment described in Figure 7700 is illustrated in the form of a server computing device 700. The computing device 700 may include a processor 701 coupled to a volatile memory 702 and a large capacity non-volatile memory (such as a disk drive 703). The computing device 700 may also include a peripheral memory access device such as a floppy disk drive, a compact disk (CD) or a digital video disk (DVD) drive 706 coupled to the processor 701. The computing device 700 may also include a network access port 704 (or interface) coupled to the processor 701 for establishing a data connection with a network (such as the Internet and / or a local area network coupled to other system computers and servers). The computing device 700 may include one or more transceivers 707 for sending and receiving electromagnetic radiation, which may be connected to a wireless communication link. The computing device 700 may include additional access ports such as USB, Firewire, Thunderbolt, etc. for coupling to peripheral devices, external memory or other devices.

[0157] Each specific implementation example is described in the following paragraphs. Although some of the examples in the following specific implementation examples are described according to the example methods, other example implementations may include: the example method discussed in the following paragraphs implemented by a vehicle processing device, which may be an on-board unit as part of an autonomous driving system or coupled to the autonomous driving system, the autonomous driving system is configured with processor-executable instructions to perform the operations of methods 1 to 10 of the following specific implementation examples; the example method discussed in the following paragraphs implemented by a computing device, which is configured with processor-executable instructions to perform the operations of methods 11 to 16 of the following specific implementation examples; the example method discussed in the following paragraphs implemented by a processing device including a component for performing the functions of the method of the following specific implementation example; and the example method discussed in the following paragraphs implemented by a non-transitory processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause a processor of a vehicle processing device or a computing device to perform the operations of the method of the following specific implementation example.

[0158] Embodiment 1. A method for using map data when performing an autonomous driving function, executed by a processor of an autonomous driving system of a vehicle, the method comprising: accessing map data about objects or features near the vehicle from a map database accessible by the processor; accessing, by the processor, confidence information associated with the map data about the objects or features near the vehicle; and using, by the processor, the confidence information when performing autonomous or semi-autonomous driving actions.

[0159] Embodiment 2. The method of embodiment 1, wherein the confidence information includes an ASIL autonomous driving level near the object or feature.

[0160] Embodiment 3. A method according to any one of Embodiments 1 or 2, wherein the confidence information includes an indication related to the accuracy of the map data regarding the object or feature.

[0161] Embodiment 4. A method according to any one of embodiments 1 to 3, wherein the confidence information includes an indication related to the reliability of the map data regarding the object or feature.

[0162] Embodiment 5. A method according to any one of embodiments 1 to 4, wherein the confidence information includes a statistical score indicating the accuracy of the map data regarding the object or feature.

[0163] Embodiment 6. A method according to any one of embodiments 1 to 5, wherein the confidence information includes the age or freshness of the map data about the object or feature.

[0164] Embodiment 7. A method according to any one of embodiments 1 to 6, wherein accessing confidence information associated with the map data regarding the object or feature near the vehicle includes obtaining the confidence information from the map database by the processor, wherein the information in the map database is obtained from one or more of a system memory, a remote computing device, or another vehicle.

[0165] Embodiment 8. A method according to any one of Embodiments 1 to 7, wherein accessing confidence information associated with the map data regarding the object or feature near the vehicle includes: obtaining the confidence information from a data structure other than the map database that can be accessed by the processor based on the location of the object or feature.

[0166] Embodiment 9. A method according to any one of embodiments 1 to 8, wherein using the confidence information when the vehicle performs an autonomous or semi-autonomous driving action includes: the processor applying a weight to the accessed map data about the object or feature based on the confidence information; and the processor using the weighted map data about the object or feature when performing path planning, object avoidance or steering autonomous driving actions.

[0167] Embodiment 10. A method according to any one of embodiments 1 to 9, wherein using the confidence information when performing autonomous or semi-autonomous driving actions by the vehicle includes: changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the objects or features near the vehicle.

[0168] Embodiment 11. A method according to any one of Embodiments 1 to 10, wherein changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle includes: changing the autonomous driving mode of the vehicle implemented by the processor to a driving mode that is compatible with the confidence information about the object or feature near the vehicle.

[0169] Embodiment 12. A method according to any one of embodiments 1 to 11, the method further comprising: in response to determining that the confidence information about the object or feature near the vehicle does not support a fully autonomous driving mode, notifying a driver that he needs to participate in driving the vehicle; and after notifying the driver, changing the autonomous driving mode of the vehicle implemented by the processor.

[0170] Embodiment 13. A method according to any one of Embodiments 1 to 12, wherein: the confidence information about the object or feature includes confidence information about objects and features within a defined area; and changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle includes: while the vehicle is in the defined area, changing the autonomous driving mode of the vehicle implemented by the processor to an autonomous driving mode consistent with the confidence information.

[0171] Embodiment 14. A method according to any one of embodiments 1 to 13, wherein the method further includes: obtaining, by the processor, sensor data about the object or feature near the vehicle from a vehicle sensor; determining, by the processor, whether the obtained sensor data about the object or feature near the vehicle differs from the map data about the object or feature obtained from the map database by a threshold amount; and uploading, by the processor, to a remote computing device, the obtained sensor data about the object or feature near the vehicle together with confidence information based on one or more of a type of sensor used to detect or classify the object or feature, a perceived quality of the object or feature achieved by the sensor, or accuracy or precision of the sensor data, in response to determining that the obtained sensor data differs from the map data about the object or feature obtained from the map database by at least the threshold amount.

[0172] Embodiment 15. A method performed by a computing device for including safety and confidence information in map data used by autonomous and semi-autonomous driving systems in a vehicle, the method comprising: receiving, by the computing device, information about an object or feature from a source to include in a map database, the information comprising a confidence measure in the information about the object or feature; using the received confidence measure in the information about the object or feature to generate safety and confidence information about the object or feature suitable for use by the vehicle autonomous and semi-autonomous driving system in autonomous or semi-autonomous driving operations, wherein the safety and confidence information comprises one or more of the following: an ASIL autonomous driving level near the object or feature; an indication related to the accuracy of the map data about the object or feature; a statistical score indicating the precision of the map data about the object or feature; an indication related to the reliability of the map data about the object or feature; or the age or freshness of the map data about the object or feature; and storing the safety and confidence information about the object or feature in a manner that enables access by the vehicle autonomous and semi-autonomous driving system.

[0173] Embodiment 16. A method according to embodiment 15, wherein information about an object or feature is received to be included in a map database, the information including a confidence measure in the information about the object or feature, and receiving the information includes receiving information including the following items from one or more vehicles: the location of the object or feature; characteristics of the object or feature; and a confidence measure in the information about the location or characteristics of the object or feature.

[0174] Embodiment 17. A method according to any one of Embodiments 15 or 16, the method further comprising updating the information about the object or feature in the map database based at least in part on a received confidence metric in the received information about the confidence of the object or feature.

[0175] Embodiment 18. A method according to any one of Embodiments 15 to 17, wherein storing the safety and confidence information about the object or feature includes including the safety and confidence information as part of the location and other information about the object or feature in the map database provided to the vehicle for use in autonomous or semi-autonomous driving operations.

[0176] Embodiment 19. A method according to any one of Embodiments 15 to 18, wherein storing the safety and confidence information about the object or feature includes: storing the safety and confidence information in a database separate from the map database associated with the location information of the object or feature; and providing the database to a vehicle for use in autonomous or semi-autonomous driving operations.

[0177] Embodiment 20. A method according to any one of embodiments 15 to 19, wherein: receiving information about an object or feature to include in a map database includes receiving information about the object or feature from multiple sources together with a confidence measure in the information about the object or feature, the method also includes determining an information set about the object or feature and combined safety and confidence information of the determined information set about the object or feature based on the information received from the multiple sources; and storing the safety and confidence information about the object or feature in a manner that can be accessed by a vehicle's autonomous and semi-autonomous driving system for use in autonomous or semi-autonomous driving operations includes storing the combined safety and confidence information of the determined information set about the object or feature in a manner that can be accessed by a vehicle's autonomous and semi-autonomous driving system for use in autonomous or semi-autonomous driving operations.

[0178] The various embodiments illustrated and described are provided merely as examples of the various features described in the claims. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment, and may be used or combined with other embodiments shown and described. Furthermore, the claims are not intended to be limited to any one exemplary embodiment. For example, one or more operations of the method may replace or be combined with one or more operations of the method.

[0179] The foregoing method descriptions and process flow charts are provided only as illustrative examples and are not intended to require or imply that the operations of the various embodiments must be performed in the order given. As will be appreciated by those skilled in the art, the order of operations in the foregoing embodiments may be performed in any order. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of operations; these words are only used to guide the reader through the description of the method. In addition, any reference to a claim element in singular form (e.g., a reference using the article "a," "an," or "the") should not be interpreted as limiting the element to the singular.

[0180] The various exemplary logic blocks, modules, circuits, and algorithmic operations described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and operations have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. Although the technical staff may implement the described functionality in different ways for each specific application, such specific implementation decisions should not be interpreted as causing a departure from the scope of the claims.

[0181] The hardware for implementing the various exemplary logics, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Although a general-purpose processor may be a microprocessor, in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuits specific to a given function.

[0182] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions may be stored as one or more instructions or codes on a non-transient computer-readable medium or a non-transient processor-readable medium. The operation of the method or algorithm disclosed herein may be embodied in a processor-executable software module, which may reside on a non-transient computer-readable or processor-readable storage medium. A non-transient computer-readable or processor-readable storage medium may be any storage medium accessible by a computer or processor. By way of example and not limitation, such non-transient computer-readable or processor-readable media may include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Disks and optical disks as used herein include compact discs (CDs), laser discs, optical disks, digital versatile discs (DVDs), floppy disks, and blue discs, wherein disks generally reproduce data magnetically, and optical disks reproduce data optically with lasers. The above combination is also included within the scope of non-transient computer-readable and processor-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions on a non-transitory processor-readable medium and / or computer-readable medium, which may be incorporated into a computer program product.

[0183] The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but should be accorded the broadest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. A method executed by a processor of an autonomous driving system of a vehicle for using map data when performing an autonomous driving function, the method comprising: accessing map data regarding objects or features in the vicinity of the vehicle from a map database accessible by the processor; accessing, by the processor, confidence information associated with the map data regarding the object or feature in proximity to the vehicle; and The confidence information is used by the processor in performing autonomous or semi-autonomous driving maneuvers. 2 . The method of claim 1 , wherein the confidence information comprises an Automotive Safety Integrity Level (ASIL) autonomous driving level in the vicinity of the object or feature. 3 . The method of claim 1 , wherein the confidence information comprises an indication related to the accuracy of the map data regarding the object or feature. 4 . The method of claim 1 , wherein the confidence information comprises an indication related to the reliability of the map data regarding the object or feature. 5 . The method of claim 1 , wherein the confidence information comprises a statistical score indicating the accuracy of the map data regarding the object or feature. The method of claim 1 , wherein the confidence information includes an age or freshness of the map data regarding the object or feature.

7. The method of claim 1 , wherein accessing confidence information associated with the map data regarding the object or feature near the vehicle comprises: The confidence information is obtained, by the processor, from the map database, wherein information in the map database is obtained from one or more of a system memory, a remote computing device, or another vehicle.

8. The method of claim 1 , wherein accessing confidence information associated with the map data regarding the object or feature near the vehicle comprises: The confidence information is obtained based on the location of the object or feature from a data structure other than the map database accessible by the processor.

9. The method of claim 1 , wherein using the confidence information when performing an autonomous or semi-autonomous driving maneuver by the vehicle comprises: applying, by the processor, a weight to the accessed map data regarding the object or feature based on the confidence information; as well as The weighted map data regarding the objects or features is used by the processor while performing path planning, object avoidance, or steering autonomous driving maneuvers.

10. The method of claim 1, wherein using the confidence information when performing an autonomous or semi-autonomous driving maneuver by the vehicle comprises: An autonomous driving mode of the vehicle implemented by the processor is altered based on the confidence information regarding the object or feature in the vicinity of the vehicle.

11. The method of claim 10, wherein changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information regarding the object or feature near the vehicle comprises: The autonomous driving mode of the vehicle implemented by the processor is changed to a driving mode that is compatible with the confidence information regarding the object or feature in proximity to the vehicle.

12. The method according to claim 10, further comprising: In response to determining that the confidence information regarding the object or feature in the vicinity of the vehicle does not support a fully autonomous driving mode, notifying a driver that involvement in driving the vehicle is required; and The autonomous driving mode of the vehicle implemented by the processor is changed after notifying the driver.

13. The method of claim 10, wherein: The confidence information about the objects or features includes confidence information about objects and features within a defined area; and Changing the autonomous driving mode of the vehicle implemented by the processor based on the confidence information regarding the object or feature near the vehicle includes changing the autonomous driving mode of the vehicle implemented by the processor to an autonomous driving mode consistent with the confidence information while the vehicle is in the defined area.

14. The method according to claim 1, further comprising: obtaining, by the processor, sensor data from a vehicle sensor regarding the object or feature in the vicinity of the vehicle; determining, by the processor, whether the obtained sensor data regarding the object or feature in the vicinity of the vehicle differs from the map data regarding the object or feature obtained from the map database by a threshold amount; and In response to determining that the obtained sensor data differs from the map data obtained from the map database regarding the object or feature by at least the threshold amount, the processor uploads the obtained sensor data regarding the object or feature near the vehicle to a remote computing device together with confidence information based on one or more of the type of sensor used to detect or classify the object or feature, the perceived quality of the object or feature achieved by the sensor, or the accuracy or precision of the sensor data.

15. An autonomous driving system for use in a vehicle, the autonomous driving system comprising: Memory; and a processor coupled to the memory and configured to: accessing map data regarding objects or features in the vicinity of the vehicle; accessing confidence information associated with the map data regarding the object or feature in proximity to the vehicle; and The confidence information is used when performing autonomous or semi-autonomous driving maneuvers.

16. The autonomous driving system of claim 15, wherein the confidence information includes an Automotive Safety Integrity Level (ASIL) autonomous driving level in the vicinity of the object or feature.

17. The autonomous driving system of claim 15, wherein the confidence information includes an indication related to an accuracy of the map data regarding the object or feature.

18. The autonomous driving system of claim 15, wherein the confidence information includes an indication related to the reliability of the map data regarding the object or feature.

19. The autonomous driving system of claim 15, wherein the confidence information comprises a statistical score indicating the accuracy of the map data regarding the object or feature.

20. The autonomous driving system of claim 15, wherein the confidence information includes an age or freshness of the map data regarding the object or feature.

21. The autonomous driving system of claim 15, wherein the processor is further configured to access confidence information associated with the map data regarding the object or feature near the vehicle by: The confidence information is obtained from the map database, wherein information in the map database is obtained from one or more of a system memory, a remote computing device, or another vehicle.

22. The autonomous driving system of claim 15, wherein the processor is further configured to access confidence information associated with the map data regarding the object or feature near the vehicle by: The confidence information is obtained based on the location of the object or feature from an accessible data structure other than the map database.

23. The autonomous driving system of claim 15, wherein the processor is further configured to use the confidence information when performing an autonomous or semi-autonomous driving action by the vehicle by: applying a weight to the accessed map data regarding the object or feature based on the confidence information; and Weighted map data about the objects or features is used while performing path planning, object avoidance, or steering autonomous driving maneuvers.

24. The autonomous driving system of claim 15, wherein the processor is further configured to use the confidence information when performing an autonomous or semi-autonomous driving action by the vehicle by: An autonomous driving mode of the vehicle implemented by the processor is altered based on the confidence information regarding the object or feature in the vicinity of the vehicle.

25. The autonomous driving system of claim 24, wherein the processor is further configured to change the autonomous driving mode of the vehicle implemented by the processor based on the confidence information about the object or feature near the vehicle by: The autonomous driving mode of the vehicle is changed to a driving mode that is compatible with the confidence information regarding the object or feature in a vicinity of the vehicle.

26. The autonomous driving system of claim 24, wherein the processor is further configured to: In response to determining that the confidence information regarding the object or feature in the vicinity of the vehicle does not support a fully autonomous driving mode, notifying a driver that involvement in driving the vehicle is required; and The autonomous driving mode of the vehicle is changed after notifying the driver.

27. The autonomous driving system of claim 24, wherein: The confidence information about the objects or features includes confidence information about objects and features within a defined area; and The processor is further configured to change the autonomous driving mode of the vehicle based on the confidence information regarding the object or feature near the vehicle by changing the autonomous driving mode to an autonomous driving mode consistent with the confidence information while the vehicle is in the defined area.

28. The autonomous driving system of claim 15, wherein the processor is further configured to: obtaining sensor data about the object or feature in the vicinity of the vehicle from a vehicle sensor; determining whether the obtained sensor data regarding the object or feature in the vicinity of the vehicle differs from the map data regarding the object or feature obtained from the map database by a threshold amount; and In response to determining that the obtained sensor data differs from the map data obtained from the map database regarding the object or feature by at least the threshold amount, uploading the obtained sensor data regarding the object or feature near the vehicle to a remote computing device together with confidence information regarding the type of sensor used to detect or classify the object or feature, the perceived quality of the object or feature achieved by the sensor, and the accuracy or precision of the sensor data.

29. A method performed by a computing device for including safety and confidence information within map data used by autonomous and semi-autonomous driving systems in a vehicle, the method comprising: receiving, by the computing device, from a source information about an object or feature for inclusion in a map database, the information including a confidence measure in the information about the object or feature; generating safety and confidence information about the object or feature suitable for use by a vehicle autonomous and semi-autonomous driving system in an autonomous or semi-autonomous driving operation using a confidence metric in the received information about the object or feature, wherein the safety and confidence information comprises one or more of: an Automotive Safety Integrity Level (ASIL) autonomous driving level in the vicinity of the object or feature; an indication related to the accuracy of the map data about the object or feature; a statistical score indicating the precision of the map data about the object or feature; an indication related to the reliability of the map data about the object or feature; or an age or freshness of the map data about the object or feature; as well as The safety and confidence information about the object or feature is stored in a manner that enables access by vehicle autonomous and semi-autonomous driving systems.

30. A method according to claim 29, wherein information about an object or feature is received for inclusion in a map database, the information including a confidence measure in the information about the object or feature, and receiving the information includes receiving information including the following items from one or more vehicles: the location of the object or feature; characteristics of the object or feature; and the confidence measure in the information about the location or the characteristics of the object or feature.

31. The method of claim 29, further comprising updating information about the object or feature in the map database based at least in part on the received confidence metric in the received information about the object or feature confidence.

32. The method of claim 29, wherein storing the safety and confidence information about the object or feature comprises including the safety and confidence information as part of a location and other information about the object or feature in the map database provided to a vehicle for use in autonomous or semi-autonomous driving operations.

33. The method of claim 29, wherein storing the safety and confidence information about the object or feature comprises: storing the safety and confidence information in a database separate from the map database associated with the location information of the object or feature; as well as The database is provided to a vehicle for use in autonomous or semi-autonomous driving operations.

34. The method of claim 29, wherein: Receiving information about an object or feature for inclusion in a map database includes receiving information about the object or feature from a plurality of sources along with a confidence measure in the information about the object or feature; The method also includes determining a set of information about the object or feature based on the information received from the plurality of sources and combined safety and confidence information of the determined set of information about the object or feature; and Storing safety and confidence information about the object or feature in a manner that enables access by the vehicle's autonomous and semi-autonomous driving systems for use in autonomous or semi-autonomous driving operations includes storing the combined safety and confidence information of the determined set of information about the object or feature in a manner that enables access by the vehicle's autonomous and semi-autonomous driving systems for use in autonomous or semi-autonomous driving operations.

35. A computing device, the computing device comprising: a network connection configured to receive messages from and transmit messages to a vehicle configured with the vehicle autonomous and semi-autonomous driving system; and a processor coupled to the network connection and configured to: receiving information received from a source regarding an object or feature for inclusion in a map database, the information including a confidence measure in the information regarding the object or feature; generating safety and confidence information about the object or feature suitable for use by a vehicle autonomous and semi-autonomous driving system in an autonomous or semi-autonomous driving operation using a confidence metric in the received information about the object or feature, wherein the safety and confidence information comprises one or more of: an Automotive Safety Integrity Level (ASIL) autonomous driving level in the vicinity of the object or feature; an indication related to the accuracy of the map data about the object or feature; a statistical score indicating the precision of the map data about the object or feature; an indication related to the reliability of the map data about the object or feature; or an age or freshness of the map data about the object or feature; as well as The safety and confidence information about the object or feature is stored or updated in a manner that enables access by vehicle autonomous and semi-autonomous driving systems.

36. A computing device according to claim 35, wherein the processor is further configured to receive information from one or more vehicles comprising: the location of the object or feature; characteristics of the object or feature; and a confidence measure in the information regarding the location or characteristics of the object or feature.

37. The computing device of claim 35, wherein the processor is further configured to store or update the safety and confidence information about the object or feature in a manner that enables access by vehicle autonomous and semi-autonomous driving systems by: storing the safety and confidence information as part of the location and other information about the object or feature in the map database provided to the vehicle for use in autonomous or semi-autonomous driving operations; or The safety and confidence information is stored in a database separate from the map database associated with the location information of the object or feature, and the database is provided to the vehicle for use in autonomous or semi-autonomous driving operations.

38. The computing device of claim 35, the processor further configured to: receiving information about objects or features from a plurality of sources for inclusion in a map database along with confidence measures in the information about the objects or features; determining a set of information about the object or feature based on the information received from the plurality of sources and combined safety and confidence information of the determined set of information about the object or feature; and The combined safety and confidence information of the determined set of information about the object or feature is stored or updated in a manner that enables access by vehicle autonomous and semi-autonomous driving systems for use in autonomous or semi-autonomous driving operations.

Citation Information

Patent Citations

  • Inclusion And Use Of Safety and Confidence Information Associated With Objects In Autonomous Driving Maps

    US20230322259A1