Automatic map anomaly detection and updating

Abstract

The present disclosure relates to an automatic map anomaly detection and update. A deviation prompt is received from a vehicle by a server, the deviation prompt indicating an anomaly in vehicle sensor data compared to autonomous vehicle data stored by the vehicle. A cause of the anomaly is identified based on a view of the vehicle sensor data during a period before and after the anomaly is received from the vehicle. Modified autonomous vehicle data is updated for a plurality of autonomous vehicles including the vehicle based on the cause.

Description

Technical Field

[0001] Aspects of the present disclosure are generally directed to automatically detecting anomalies in an autonomous vehicle map and automatically updating the map. Background Art

[0002] In a vehicle telematics system, a telematics control unit (TCU) can be used for various remote control services such as over-the-air (OTA) software downloads, eCall, and turn-by-turn navigation. An autonomous vehicle is a vehicle that can sense the vehicle environment and navigate without human input. Autonomous vehicles may have greater data upload and download requirements than traditional telematics systems. Summary of the invention

[0003] A vehicle includes an autonomous vehicle sensor, a memory storing a 3D a priori map and a road network definition file, and a processor. The processor is configured to: in response to a successful interpretation of an anomaly detected via raw sensor data received from the sensor, send the interpretation to an autonomous vehicle server, and in response to an unsuccessful interpretation of the anomaly, send a deviation prompt to the autonomous vehicle server, wherein the deviation prompt indicates a road segment where the anomaly is detected.

[0004] A method includes: receiving, by a server from a vehicle, a deviation alert indicating an anomaly in vehicle sensor data compared to autonomous vehicle data maintained by the vehicle; identifying, by the server, a cause of the anomaly based on a view of the vehicle sensor data over a time duration that includes a time when the anomaly was received; and sending modified autonomous vehicle data updated based on the cause to a plurality of autonomous vehicles.

[0005] A system includes a server configured to: receive a deviation alert from a vehicle, the deviation alert indicating an anomaly in vehicle sensor data compared to autonomous vehicle data maintained by the vehicle; identify a cause of the anomaly based on a view of the vehicle sensor data over a duration including a time when the anomaly was received; and send modified autonomous vehicle data updated based on the cause to a plurality of autonomous vehicles including the vehicle.

[0006] According to one embodiment of the invention, in response to the vehicle being unable to successfully identify a cause of the anomaly, a view of the vehicle sensor data is received from the vehicle.

[0007] According to one embodiment of the present invention, the server is further configured to: identify anomalies in the vehicle sensor data using a neural network trained for multiple causes of vehicle sensor data anomalies.

[0008] According to one embodiment of the present invention, the server is further configured to compare the anomaly with a signature of a predefined anomaly type, wherein the predefined anomaly type includes one or more of a lane closure sign, a vehicle that has lost the ability to drive, and an emergency vehicle.

[0009] According to an embodiment of the present invention, the duration includes sensor data from fifteen seconds before the anomaly to fifteen seconds after the anomaly.

[0010] According to one embodiment of the present invention, the vehicle sensor data includes data from a lidar sensor of the vehicle and data from a camera of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 An example system is shown including an autonomous vehicle in communication with an autonomous vehicle data server;

[0012] Figure 2 An example diagram of a vehicle implementing autonomous vehicle functionality is shown;

[0013] Figure 3 An example process for detecting anomalies by a vehicle is shown;

[0014] Figure 4 An example process for performing autonomous driving by a vehicle is shown;

[0015] Figure 5 An example process for updating autonomous vehicle data by an autonomous vehicle data server based on detected anomalies is shown. DETAILED DESCRIPTION

[0016] As required, specific embodiments of the present invention are disclosed herein; however, it should be understood that the disclosed embodiments are merely examples of the present invention, which may be implemented in various forms and alternative forms. The drawings are not necessarily drawn to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to utilize the present invention in various ways.

[0017] Figure 1An example system 100 is shown that includes vehicles 102 in communication with an autonomous vehicle data server 110. As shown, system 100 includes vehicles 102A and 102B (collectively 102) configured to communicate wirelessly with service providers 106A and 106B (collectively 106) and / or wireless stations 108 via a wide area network 104. Autonomous vehicle data server 110 is also in communication with wide area network 104. Vehicles 102 may communicate with each other via Wi-Fi or other wireless communication protocols to allow vehicles 102 to utilize the connectivity capabilities of other vehicles 102. Although in Figure 1 An example system 100 is shown in FIG. 1 , but the example components shown are not intended to be limiting. In practice, the system 100 may have more or fewer components, and additional or alternative components and / or implementations may be used. As an example, the system 100 may include more or fewer vehicles 102, service providers 106, radio stations 108, and / or autonomous vehicle data servers 110.

[0018] The vehicle 102 may include various types of motor vehicles (hybrid utility vehicles (CUVs), sport utility vehicles (SUVs), trucks, recreational vehicles (RVs)), boats, airplanes, or other mobile machines for transporting people or items. In many cases, the vehicle 102 may be driven by an internal combustion engine. As another possibility, the vehicle 102 may be a hybrid electric vehicle (HEV) driven by both an internal combustion engine and one or more electric motors, such as a series hybrid electric vehicle (SHEV), a parallel hybrid electric vehicle (PHEV), or a parallel / series hybrid electric vehicle (PSHEV). Since the type and configuration of the vehicle 102 may vary, the capabilities of the vehicle 102 may vary accordingly. As some other possibilities, the vehicle 102 may have different capabilities for passenger capacity, traction capacity and capacity, and storage capacity.

[0019] As some non-limiting examples, the wide area network 104 may include one or more interconnected communication networks, such as the Internet, a cable television distribution network, a satellite link network, a local area network, a wide area network, and a telephone network. By accessing the wide area network 104, the vehicle 102 is able to send outgoing data from the vehicle 102 to a network destination address on the wide area network 104, and is able to receive incoming data to the vehicle 102 from a network destination address on the wide area network 104.

[0020] The service provider 106 may include system hardware configured to allow the cellular transceiver of the vehicle 102 to access the communication services of the wide area network 104. In an example, the service provider 106 may be a Global System for Mobile Communications (GSM) cellular service provider. In another example, the service provider 106 may be a Code Division Multiple Access (CDMA) cellular service provider.

[0021] The radio station 108 may include system hardware configured to support a local area network connection between the vehicle 102 and the wide area network 104. In an example, the radio station 108 may support a dedicated short range communication (DSRC) connection between the vehicle 102 and the radio station 108. In another example, the radio station 108 may support a Wi-Fi connection between the vehicle 102 and the radio station 108. In another example, the radio station 108 may support a 3GPP-based cellular vehicle-to-everything (C-V2X) connection between the vehicle 102 and the radio station 108. The radio station 108 may also provide a connection between the radio station 108 and the wide area network 104, thereby allowing the connected vehicle 102 to access the wide area network without using the service provider 106.

[0022] The autonomous vehicle 102 operates by utilizing vehicle sensor data and other road environment data in combination with various driving algorithms. The autonomous vehicle data server 110 may include computing hardware configured to provide autonomous data services to the vehicle 102. In an example, the autonomous vehicle data server 110 may store autonomous vehicle data useful for the autonomous vehicle 102 along the road crossing route. Such autonomous vehicle data may include a road network definition file 112, a 3D prior map 114, and a deviation prompt 116. The vehicle 102 may receive autonomous vehicle data of the environment of the upcoming vehicle 102 from the autonomous vehicle data server 110. The vehicle 102 may use the autonomous vehicle data to identify the specific location of the vehicle along the route. The autonomous vehicle 102 may also be designed to upload the sensed road environment data so that the autonomous vehicle data server 110 updates the autonomous vehicle data. Accordingly, the autonomous vehicle data server 110 may also be configured to update the autonomous vehicle data based on the information provided from the vehicle 102 to the autonomous vehicle data server 110.

[0023] The road network definition file 112 may specify accessible road segments and provide information such as waypoints, stop sign locations, lane widths, speed limits, checkpoint locations, and parking space locations. The road network definition file 112 may also include one or more free travel areas (such as parking lots) that define a perimeter where travel is generally permitted.

[0024] The 3D a priori map 114 may include 3D model information of road segments or other traversable locations indicated by the road network definition file 112. In many examples, the 3D a priori map 114 is created by traversing the road segments by an instrumented vehicle 102 having a lidar, camera (mono, stereo, etc.), and / or other sensors useful for generating a 3D model of the road. Some examples of other sensors may include, for example, lasers, radars, global positioning systems (GPS), inertial measurement units (IMUs), altimeters, and wheel encoders.

[0025] Deviation alerts 116 include data regarding road locations where anomalies were detected when compared to 3D a priori map 114. In an example, an autonomous vehicle 102 traversing a road segment may encounter an obstacle on the road that is not indicated on 3D a priori map 114. As explained in further detail below, if a vehicle 102 cannot identify an obstacle, the vehicle 102 may generate a deviation alert 116 to notify other vehicles 102 of the unknown obstacle.

[0026] Figure 2 An example diagram 200 of a vehicle 102 implementing autonomous vehicle functionality is shown. The vehicle 102 includes a telematics controller 202 configured to communicate over a wide area network 104. Such communications may be performed using a telematics modem 208 of the telematics controller 202. Each vehicle 102 also includes an autonomous vehicle controller 222 that is additionally configured to communicate over the wide area network 104 using a dedicated autonomous vehicle modem 232. Although in Figure 2 An example vehicle 102 is shown in FIG. 1 , but the example components shown are not intended to be limiting. In practice, the vehicle 102 may have more or fewer components, and additional or alternative components and / or implementations may be used.

[0027] The telematics controller 202 may be configured to support voice command and Bluetooth interactions with the driver and devices carried by the driver (e.g., nomadic device 210), receive user input via various buttons or other controls, and provide vehicle status information to the driver or other occupants of the vehicle 102. An example telematics controller 202 may be the SYNC system provided by Ford Motor Company of Dearborn, Michigan.

[0028] The telematics controller 202 may also include various types of computing devices that support the execution of the functions of the telematics controller 202 described herein. In an example, the telematics controller 202 may include one or more processors 204 configured to execute computer instructions and a storage medium 206 on which computer executable instructions and / or data may be stored. Computer-readable storage media (also referred to as processor-readable media or memory 206) include any non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by the processor 204). Typically, the processor 204 receives instructions and / or data from, for example, the memory 206 to a memory and executes the instructions using the data to perform one or more processes, including one or more of the processes described herein. Computer executable instructions may be compiled or interpreted from a computer program created using various programming languages ​​and / or technologies, including, but not limited to, one or a combination of Java, C, C++, C#, Fortran, Pascal, VisualBasic, Python, Java Script, Perl, PL / SQL, etc.

[0029] The telematics controller 202 may be configured to communicate with a mobile device 210 of a vehicle occupant. The mobile device 210 may be any of various types of portable computing devices, such as a cellular phone, a tablet computer, a smart watch, a laptop computer, a portable music player, or other devices capable of communicating with the telematics controller 202. Like the telematics controller 202, the mobile device 210 may include one or more processors configured to execute computer instructions and a storage medium on which computer executable instructions and / or data may be stored. In many examples, the telematics controller 202 may include a wireless transceiver 212 (e.g., a Bluetooth controller, a ZIGBEE transceiver, a Wi-Fi transceiver, etc.) configured to communicate with a compatible wireless transceiver of the mobile device 210. Additionally or alternatively, the telematics controller 202 may communicate with the mobile device 210 via a wired connection, such as via a USB connection between the mobile device 210 and a USB subsystem of the telematics controller 202. Additionally or alternatively, the telematics controller 202 may utilize the wireless transceiver 212 to communicate with a Wi-Fi transceiver of a radio station 108 near a road that the vehicle 102 is traversing. As another example, the telematics controller 202 may utilize the wireless transceiver 212 to communicate with other vehicles 102 traversing the road.

[0030] The telematics controller 202 may also receive input from a human-machine interface (HMI) control 214 that is configured to provide occupant interaction with the vehicle 102. For example, the telematics controller 202 may interact with one or more buttons or other HMI controls 214 (e.g., steering wheel audio buttons, push-to-talk buttons, dashboard controls, etc.) that are configured to invoke functions on the telematics controller 202. The telematics controller 202 may also drive or otherwise communicate with one or more displays 216 that are configured to provide visual output to the vehicle occupants, such as through a video controller. In some cases, the display 216 may be a touch screen that is further configured to receive user touch input via a video controller, while in other cases, the display 216 may be just a display without touch input capabilities. In an example, the display 216 may be a head unit display included in a center console area of ​​the cabin of the vehicle 102. In another example, display 216 may be a screen of an instrument cluster of vehicle 102 .

[0031] The telematics controller 202 may also be configured to communicate with other components of the vehicle 102 via one or more in-vehicle networks 218. As some examples, the in-vehicle network 218 may include one or more of a vehicle controller area network (CAN), Ethernet, and a media-oriented system transport (MOST). The in-vehicle network 218 may allow the telematics controller 202 to communicate with other systems of the vehicle 102 (such as a body control module (BCM) 220-A, an electronic brake control system (EBCM) 220-B, a steering control system (SCM) 220-C, a powertrain control system (PCM) 220-D, a safety control system (SACM) 220-E, and a global positioning system (GPS) 220-F). As shown, the controller 220 is represented as a discrete module and system. However, the controller 220 may share physical hardware, firmware, and / or software so that functions from multiple controllers 220 may be integrated into a single module 220, and the functions of various such controllers 220 may be distributed over multiple controllers 220.

[0032] The BCM 220-A may be configured to support various functions of the vehicle 102 related to controlling current loads powered by the battery of the vehicle 102. Examples of such current loads include, but are not limited to, exterior lighting, interior lighting, heated seats, heated windshields, heated taillights, and heated mirrors. In addition, the BCM 220-A may be configured to manage access functions of the vehicle 102, such as keyless entry, remote start, and access point status verification (e.g., the closed state of the hood, doors, and / or trunk of the vehicle 102).

[0033] The EBCM 220-B may be configured to control braking functions of the vehicle 102. In some examples, the EBCM 220-B may be configured to receive signal information from vehicle wheel sensors and / or a powertrain differential and manage anti-lock braking functions and anti-skid braking functions by controlling brake line valves that modulate brake pressure from a master cylinder.

[0034] The SCM 220-C may be configured to assist in vehicle steering by increasing or reacting to steering forces provided to the wheels of the vehicle 102. In some cases, the increased steering force may be provided by a hydraulic steering assist device configured to provide controlled energy to the steering mechanism, while in other cases, the increased steering force may be provided by an electric actuator system.

[0035] The PCM 220-D may be configured to perform engine control functions and transmission control functions for the vehicle 102. For engine control, the PCM 220-D may be configured to receive throttle inputs and control actuators of the vehicle engine to set the air / fuel mixture, ignition timing, idle speed, valve timing, and other engine parameters to ensure optimal engine performance and power generation. For transmission control, the PCM 220-D may be configured to receive inputs from vehicle sensors (such as wheel speed sensors, vehicle speed sensors, throttle position, transmission fluid temperature) and determine how and when to change gears in the vehicle 102 to ensure proper performance, fuel economy, and shift quality.

[0036] The SACM 220-E may be configured to provide various functions to improve the stability of the vehicle 102 and improve the control of the vehicle 102. As some examples, the SACM 220-E may be configured to monitor vehicle sensors (e.g., steering wheel angle sensor, yaw rate sensor, lateral acceleration sensor, wheel speed sensor, etc.) and control the BCM 220-A, SCM 220-C and / or PCM 220-D. As some feasible ways, the SACM 220-E may be configured to provide throttle input adjustment, steering angle adjustment, brake modulation, and all-wheel drive power split decision through the vehicle bus 218 to improve vehicle stability and controllability. It should be noted that in some cases, the commands provided by the SACM 220-E may override other commands provided by the driver or the autonomous vehicle controller 222.

[0037] The GPS 220 -F is configured to provide current location and heading information of the vehicle 102 , and various other vehicle controllers 220 are configured to cooperate with the telematics controller 202 .

[0038] Autonomous vehicle controller 222 may include and / or may communicate with various types of computing devices to facilitate execution of functions of autonomous vehicle 102. In an example, autonomous vehicle controller 222 may include one or more processors 224 configured to execute computer instructions and a storage medium 226 on which computer-executable instructions (e.g., autonomous vehicle logic 234 discussed in more detail below) and / or autonomous vehicle data (e.g., road network definition file 112, 3D prior map 114, deviation prompts 116, etc.) may be stored.

[0039] The autonomous vehicle controller 222 can receive input from various sensors. In an example, the autonomous vehicle controller 222 can communicate with a lidar sensor 228. In other examples, the autonomous vehicle controller 222 can additionally or alternatively communicate with a laser, radar, sonar, or other type of distance sensor and / or obstacle sensor. The autonomous vehicle controller 222 can communicate with one or more cameras 230 configured to capture information related to the environment surrounding the vehicle 102.

[0040] The autonomous vehicle controller 222 may also utilize the autonomous vehicle modem 232 to communicate data (e.g., autonomous vehicle data) between the vehicle 102 and an autonomous vehicle data server 110 accessible via the wide area network 104. In some examples, the autonomous vehicle modem 232 may be configured to communicate with the same service provider 106 that provides communication services to the telematics modem 208. In other examples, the autonomous vehicle modem 232 may be configured to communicate with a service provider 106 that is different from the service provider 106 that provides communication services to the telematics modem 208. In one example, the telematics modem 208 may utilize a wireless communication interface. Figure 1 The service provider 106A shown in FIG. 1 is used to access the wide area network 104, and the autonomous vehicle modem 232 can use Figure 1 WAN 104 is accessed through service provider 106B shown in FIG.

[0041] The autonomous vehicle controller 222 may utilize driving algorithms to command braking, steering, acceleration, and other functions of the vehicle 102. These algorithms may be stored to a memory 226 and may be executed by one or more processors 224 of the autonomous vehicle controller 222 to command the vehicle 102. The autonomous vehicle controller 222 may accordingly command the vehicle 102 based on inputs such as the road network definition file 112, 3D prior map 114, and deviation prompts 116 received from the autonomous vehicle data server 110, sensor inputs received from a lidar sensor 228 (or other sensors), image inputs received from one or more camera devices 230, and data received from various controllers 220 via the vehicle bus 218.

[0042] One such algorithm executed by autonomous vehicle controller 222 is path prompting logic 234. As autonomous vehicle 102 is traversing its route, path prompting logic 234 causes vehicle 102 to continually compare the current 3D view recognized by vehicle 102 via lidar sensor 228, camera 230, and / or other sensors to a previously loaded 3D a priori map 114 stored in memory 226.

[0043] If the road prompt logic 234 identifies a discrepancy, the road prompt logic 234 may cause the vehicle 102 to send an anomaly discovery view to the autonomous vehicle data server 110. In an example, the anomaly discovery view includes a snapshot of data before and after the anomaly discovery (e.g., from fifteen seconds before the anomaly to fifteen seconds after the anomaly). The anomaly discovery view data may include data from the lidar sensor 228 and the camera 230 and / or data from other autonomous vehicle sensors of the vehicle 102. The anomaly discovery view data may be sent from the vehicle 102 to the autonomous vehicle data server 110 using the autonomous vehicle modem 232 (or in other cases, additionally or alternatively using the telematics modem 208).

[0044] Autonomous vehicle data server 110 may receive the updated data and may update the version of 3D a priori map 114 stored on autonomous vehicle data server 110 with the new information from autonomous vehicle 102. Autonomous vehicle data server 110 may also send the updated 3D a priori map 114 to other autonomous vehicles 102 in the vicinity of the discovered anomaly (e.g., vehicles taking a route along the same road segment as autonomous vehicle 102 that reported the anomaly).

[0045] These files, when uploaded and downloaded, can place a significant burden on the communication resources of the vehicle 102 and the communication resources of the wide area network 104. In addition, when the autonomous vehicle 102 returns to service, the 3D a priori map 114 in many vehicles is updated by hard drive swapping in the service area or Ethernet downloading. This results in the autonomous vehicle 102 lacking the latest 3D a priori map 114 for the anomaly until the autonomous vehicle 102 can return to the service area, which may take many hours. However, the autonomous vehicle 102 lacking the latest 3D a priori map 114 may not be able to complete the dynamic driving task.

[0046] As explained in further detail with respect to process 300 , process 400 , and process 500 , system 100 avoids using costly wireless links to send updated large anomalies, and at the same time avoids autonomous vehicles 102 waiting until they return to a depot to receive updated 3D a priori maps 114 .

[0047] In response to detecting an anomaly, autonomous vehicle 102 attempts to identify the object that caused the anomaly detection (e.g., autonomous vehicle 102 interprets raw camera data of a stop sign as metadata indicating a stop sign). If an anomaly can be detected, the metadata is sent to autonomous vehicle data server 110 along with the location of the anomaly and the reason for the detection. This information is processed by autonomous vehicle data server 110 and disseminated to other autonomous vehicles 102 in the operating area to allow other autonomous vehicles 102 to better perform autonomous driving tasks.

[0048] However, if the autonomous vehicle 102 is unable to identify the cause of the anomaly, a deviation prompt 116 is sent by the autonomous vehicle 102 to the autonomous vehicle data server 110. The deviation prompt 116 may include information such as the location coordinates (e.g., GPS coordinates) of the anomaly, an indicator (such as, caution or slow down), and the coordinates of the 3D a priori map 114 where the difference with the 3D a priori map 114 was detected. Where feasible, the autonomous vehicle 102 may also send the entire sensor data set to allow the autonomous vehicle data server 110 to further process the area of ​​the 3D a priori map 114 to better determine the possible cause of the anomaly. The deviation prompt 116 of the anomaly may also be sent by the autonomous vehicle data server 110 to other autonomous vehicles 102 that are about to enter the area where the anomaly has been detected to notify these autonomous vehicles 102 that there is a problem with the area (even if the exact problem is not initially apparent through the deviation prompt 116).

[0049] Figure 3 An example process 300 is shown for detecting anomalies by vehicle 102 . In an example, process 300 may be performed by autonomous vehicle controller 222 executing roadway prompt logic 234 .

[0050] In operation 302, the autonomous vehicle controller 222 uses one or more sensors of the vehicle 102 to detect an anomaly. In an example, the autonomous vehicle controller 222 may execute the road prompting logic 234 to track the progress of the vehicle 102 along the route using the 3D prior map 114 and data from the lidar 228, the camera 230, and / or other sensors. During the traversal of the route, the road prompting logic 234 may identify one or more 3D features of the sensor data that do not match the 3D prior map 114 and the position of the vehicle 102. For example, this situation may be caused by a temporary feature (such as a vehicle 102 that has broken down in the road) or a permanent feature (such as a change in the road structure). In this case, the road prompting logic 234 may identify an anomaly because the 3D prior map 114 does not match the data of the vehicle 102 received by the sensors of the vehicle 102.

[0051] At operation 304, autonomous vehicle controller 222 interprets the detected anomaly. In an example, autonomous vehicle controller 222 may execute road prompt logic 234 to identify the cause of the anomaly. For example, road prompt logic 234 may compare the anomaly to signatures of predefined anomaly types (such as lane closure signs, disabled vehicles, emergency vehicles, etc.). In another example, autonomous vehicle controller 222 may utilize a neural network trained for various types of road sensor data to allow autonomous vehicle controller 222 to identify anomalies in the sensor data.

[0052] At operation 306, the autonomous vehicle controller 222 determines whether the interpretation of the anomaly was successful. In the example, if the anomaly matches one of the predefined signatures or the neural network matches the sensor data, the autonomous vehicle controller 222 has successfully interpreted the anomaly. If the autonomous vehicle controller 222 determines that the interpretation of the anomaly is successful, control passes to operation 308. Otherwise, control passes to operation 310.

[0053] At operation 308, autonomous vehicle controller 222 sends the interpreted metadata to autonomous vehicle data server 110. Accordingly, in the event that road prompt logic 234 identifies a cause of an anomaly, metadata describing the identified anomaly may be sent to autonomous vehicle data server 110 without sending the original sensor data to autonomous vehicle data server 110. After operation 308, process 300 ends.

[0054] In operation 310, autonomous vehicle controller 222 sends deviation alert 116 to autonomous vehicle data server 110. In an example, deviation alert 116 may specify the location coordinates (e.g., GPS coordinates) of the anomaly, an indicator (such as caution or slow down), and the coordinates of 3D a priori map 114 where the difference from 3D a priori map 114 was detected. Autonomous vehicle controller 222 may send deviation alert 116 to autonomous vehicle data server 110 using autonomous vehicle modem 232 (or in other cases, additionally or alternatively, using telematics modem 208).

[0055] At operation 312, autonomous vehicle controller 222 sends the raw sensor data to autonomous vehicle data server 110. In the example, autonomous vehicle controller 222 may send the raw sensor data to autonomous vehicle data server 110 over time. Since deviation alert 116 has already been sent to autonomous vehicle data server 110, the timing of transmitting the raw sensor data is not as important as the raw sensor data required for system 100 to identify potential anomalies. After operation 312, process 300 ends.

[0056] Figure 4 An example process 400 is shown for updating autonomous vehicle data based on detected anomalies by autonomous vehicle data server 110 . In an example, process 400 may be performed by autonomous vehicle data server 110 .

[0057] At operation 402, the autonomous vehicle data server 110 determines whether a deviation prompt 116 is received. In an example, the autonomous vehicle data server 110 may receive a deviation prompt 116 from one of the vehicles 102, as discussed above with respect to operation 310 of process 300. If the autonomous vehicle data server 110 determines that a deviation prompt 116 is received, control passes to operation 404. Otherwise, control passes to operation 406.

[0058] In operation 404, autonomous vehicle data server 110 sends deviation alerts 116 to autonomous vehicles 102. In an example, autonomous vehicle data server 110 may receive information indicating the current location of autonomous vehicle 102, and may send deviation alerts 116 to vehicles 102 within a predefined geographic proximity of the location updated by deviation alert 116 (or in the same zip code area, state, or other area as the location of deviation alert 116). In another example, autonomous vehicle data server 110 may receive information indicating the expected route of autonomous vehicle 102, and autonomous vehicle data server 110 may, based on the information, send deviation alerts 116 to vehicles 102 that will traverse the road segment updated by deviation alert 116. In yet another example, autonomous vehicle data server 110 sends deviation alerts 116 to all vehicles 102.

[0059] At operation 406, autonomous vehicle data server 110 determines whether raw sensor data is received. In an example, autonomous vehicle data server 110 may receive raw sensor data from one of vehicles 102, as discussed above with respect to operation 312 of process 300. If autonomous vehicle data server 110 determines that raw sensor data is received, control passes to operation 408. If autonomous vehicle data server 110 determines that raw sensor data is not received, control passes to operation 414.

[0060] At operation 408, autonomous vehicle data server 110 identifies the anomaly. In an example, autonomous vehicle data server 110 may have greater computer and data storage capabilities than vehicle 102, and may accordingly utilize more sophisticated techniques to identify the nature of the anomaly than may be performed by vehicle 102. Similar to what was discussed above with respect to operation 304, these techniques may include comparing the anomaly to signatures of predefined anomaly types and / or using a trained neural network to identify the anomaly.

[0061] At operation 410, autonomous vehicle data server 110 updates road network definition file 112 and / or 3D a priori map 114. In an example, autonomous vehicle data server 110 may automatically make changes to road network definition file 112 and / or 3D a priori map 114 to include the identified anomaly in the updated version of road network definition file 112 and / or 3D a priori map 114.

[0062] In operation 412, autonomous vehicle data server 110 sends road network definition file 112 and / or 3D a priori map 114 to autonomous vehicle 102. Updated autonomous vehicle data may be sent to vehicle 102 according to one or more of the methods indicated above with respect to operation 404. After operation 412, control returns to operation 402.

[0063] In operation 414, the autonomous vehicle data server 110 determines whether the explained metadata is received. In the example, as discussed above with respect to operation 308, the vehicle 102 may have sent the explanation of the anomaly to the autonomous vehicle data server 110. If the autonomous vehicle data server 110 determines that the explained metadata is received, control passes to operation 410 to update the autonomous vehicle data with the vehicle 102 that detected the anomaly. If the autonomous vehicle data server 110 determines that the explained metadata is not received, control returns to operation 402.

[0064] Figure 5 An example process 500 is shown for performing autonomous driving by vehicle 102. In an example, process 500 may be performed by autonomous vehicle controller 222.

[0065] At operation 502 , autonomous vehicle controller 222 receives autonomous vehicle data from autonomous vehicle data server 110 . In an example, autonomous vehicle data may be received by vehicle 102 as discussed above with respect to operation 404 or operation 412 of process 400 .

[0066] At operation 504, autonomous vehicle controller 222 performs autonomous vehicle driving according to autonomous vehicle data. In the example, autonomous vehicle controller 222 utilizes the received autonomous vehicle data to perform one or more autonomous driving operations. For example, if deviation prompt 116 is received from autonomous vehicle server 110 at operation 502, vehicle 102 may adjust its route to avoid the road segment with anomalies indicated by deviation prompt 116. After operation 504, process 500 ends.

[0067] The computing devices described herein, such as autonomous vehicle controller 222 and autonomous vehicle data server 110, generally include computer-executable instructions that can be executed by one or more computing devices, such as the computing devices listed above. Computer-executable instructions, such as those of roadway prompt logic 234, can be compiled or interpreted by a computer program created using a variety of programming languages ​​and / or techniques, including, but not limited to, Java. TM , C, C++, C#, Visual Basic, Java Script, Python, Perl, PL / SQL, etc., or a combination thereof. In general, a processor (e.g., a microprocessor) receives instructions from, for example, a memory, a computer-readable medium, etc., and executes these instructions to perform one or more processes, including one or more of the processes described herein. Various computer-readable media can be used to store and transmit such instructions and other data.

[0068] With respect to the processes, systems, methods, teachings, and the like described herein, it should be understood that although the steps of the processes, and the like have been described as occurring in a particular ordered sequence, the processes may also be implemented with the described steps performed in an order different than that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of the processes herein are provided for the purpose of illustrating specific embodiments and should not be construed in any way as limiting the claims.

[0069] Therefore, it should be understood that the above description is intended to be illustrative and not restrictive. When reading the above description, many embodiments and applications other than the examples provided will be apparent. The scope should not be determined with reference to the above description, but with reference to the claims and the full range of equivalents to which the claims are authorized. It is expected and anticipated that future developments will occur in the technology discussed here, and the disclosed systems and methods will be incorporated into such future embodiments. In short, it should be understood that the application is capable of modification and variation.

[0070] Unless expressly indicated to the contrary herein, all terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those skilled in the art to which the technology described herein belongs. In particular, unless a claim recites an express limitation to the contrary, the use of the singular article should be understood to recite one or more of the indicated elements.

[0071] An abstract of the present disclosure is provided to allow the reader to quickly ascertain the essence of the present technical disclosure. The abstract is submitted with the understanding that the abstract will not be used to interpret or limit the scope or meaning of the claims. In addition, it can be seen from the foregoing detailed description that various features are combined together in various embodiments for the purpose of simplifying the present disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than those expressly recited in each claim. On the contrary, as reflected in the claims, the subject matter of the invention lies in fewer features than all the features of a single disclosed embodiment. Therefore, the claims are hereby incorporated into the detailed description, with each claim itself as a separately claimed subject matter.

[0072] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the present invention. More specifically, the words used in the specification are descriptive rather than restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of the present invention. In addition, the features of the embodiments of each implementation may be combined to form further embodiments of the present invention.

Claims

1. A vehicle comprising: Autonomous vehicle sensors; Memory, storing 3D priori map and road network definition files; The processor is configured as: detecting anomalies in the raw sensor data received from the autonomous vehicle sensor by identifying that one or more 3D features of the raw sensor data do not match a 3D a priori map for the location of the vehicle; An explanation of the cause of the anomaly; In response to successfully explaining the cause of the abnormality, sending an explanation indicating the cause of the abnormality to an autonomous vehicle server; In response to unsuccessful interpretation of the cause of the anomaly, a deviation prompt is sent to the autonomous vehicle server, wherein the deviation prompt indicates the road segment where the anomaly is detected and does not indicate the cause of the anomaly.

2. The vehicle according to claim 1, wherein: The processor is further configured to, after sending the deviation alert, send the view of the raw sensor data to an autonomous vehicle server.

3. The vehicle according to claim 2, wherein: The view of the raw sensor data includes snapshots of data before and after the anomaly.

4. The vehicle according to claim 3, wherein: The data snapshot includes data from fifteen seconds before the anomaly to fifteen seconds after the anomaly.

5. The vehicle of claim 1, wherein: The autonomous vehicle sensors include a lidar sensor and a camera, and the raw sensor data includes data from the lidar sensor and data from the camera.

6. The vehicle of claim 1, wherein: The autonomous vehicle sensors include one or more of a laser, a radar, a global positioning system positioning device, an inertial measurement unit, an altimeter, and a wheel encoder.

7. The vehicle of claim 1, wherein: The processor is further configured to: receiving another deviation alert from the autonomous vehicle server based on another anomaly detected via another raw sensor data received from another sensor of another vehicle; One or more autonomous vehicle driving maneuvers are updated based on the further deviation prompt.

8. The vehicle of claim 7, wherein: The another deviation hint indicates a road segment of the road network definition file where the another anomaly is detected, and the processor is further configured to update the one or more autonomous vehicle driving maneuvers to avoid the road segment where the another anomaly is detected.

9. A method for exception handling, comprising: receiving, by a server from the vehicle, a deviation prompt in response to the vehicle failing to successfully identify a cause of an anomaly in vehicle sensor data compared to autonomous vehicle data maintained by the vehicle, the deviation prompt indicating the anomaly without indicating a cause of the anomaly, the anomaly identified by the vehicle in response to the vehicle identifying that one or more 3D features of raw sensor data received from autonomous vehicle sensors do not match a 3D a priori map for the location of the vehicle; sending the deviation prompt to one or more other vehicles around the vehicle to notify the one or more other vehicles of the abnormality; identifying, by the server, a cause of the anomaly based on a view of vehicle sensor data over a duration that includes a time at which the anomaly was received; Modified autonomous vehicle data updated according to the reason is sent to the plurality of autonomous vehicles.

10. The method of claim 9, further comprising: In response to the vehicle not successfully identifying a cause of the anomaly, a view of the vehicle sensor data is received from the vehicle.

11. The method of claim 9, further comprising: Anomalies in the vehicle sensor data are identified using a neural network trained for multiple causes of vehicle sensor data anomalies.

12. The method of claim 9, further comprising: The anomaly is compared to signatures of predefined anomaly types, wherein the predefined anomaly types include one or more of a lane closed sign, a disabled vehicle, and an emergency vehicle.

13. The method of claim 9, wherein: The duration includes from fifteen seconds before the anomaly to fifteen seconds after the anomaly.

14. The method of claim 9, wherein: The vehicle sensor data includes data from a lidar sensor of the vehicle and data from a camera of the vehicle.

Images