Sensor device fault monitoring based on dual path ethernet

Abstract

In one embodiment, an autonomous driving vehicle (ADV) simultaneously processes a sensor data set by a first computing unit and a second computing unit, where the sensor data set is generated by a sensor. The first computing unit formats the sensor data set into a message data set. The second computing unit determines whether the sensor data set indicates a control path failure. The second computing unit reports a control path fault in response to determining that the sensor data set indicates the control path fault.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to operating an autonomous vehicle. More specifically, embodiments of the present disclosure relate to sensor device fault monitoring based on dual-path Ethernet. Background Art

[0002] Vehicles operating in an automated mode (e.g., driverless) can relieve occupants, particularly the driver, of some driving-related responsibilities. When the vehicle is operating in an automated mode, it can navigate to various locations using onboard sensor equipment, allowing the vehicle to travel with minimal human intervention, and in some cases even without passengers.

[0003] The autonomous vehicle uses the sensor driver in conjunction with the sensor device to parse the raw data generated by the sensor device; publish the parsed raw data in a defined message data format; and send the message data to other autonomous driving software units for further processing. The sensor driver also reports faults related to the sensor data path, such as data timeout, incomplete data, data error, etc. Summary of the invention

[0004] In a first aspect, there is provided a computer-implemented method for operating an autonomous vehicle, the method comprising:

[0005] The sensor data set is processed simultaneously by the first computing unit and the second computing unit, wherein the sensor data set is generated by a sensor, and the processing further includes:

[0006] formatting the sensor data set into a message data set by the first computing unit;

[0007] determining, by a second computing unit, whether the sensor data set indicates a control path fault; and

[0008] A control path fault is reported by the second computing unit in response to determining that the sensor data set indicates a control path fault.

[0009] In a second aspect, a non-transitory machine-readable medium is provided, wherein instructions are stored therein, and when the instructions are executed by a processor, the processor is caused to perform the method in the first aspect.

[0010] In a third aspect, there is provided a system for operating an autonomous vehicle, comprising:

[0011] Processing equipment; and

[0012] The memory is used to store instructions, which, when executed by the processing device, cause the processing device to execute the method.

[0013] In a fourth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, causes the processor to perform the method in the first aspect.

[0014] Through the technical solution disclosed in the present invention, the autonomous driving vehicle can effectively perform fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Embodiments of the present disclosure are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0016] Figure 1 is a block diagram illustrating a networked system according to one embodiment.

[0017] Figure 2 is a block diagram illustrating an example of an autonomous driving vehicle according to one embodiment.

[0018] Figure 3 and Figure 4 is a block diagram illustrating an example of an autonomous driving system for use with an autonomous vehicle according to one embodiment.

[0019] Figure 5 is a block diagram illustrating a system architecture for autonomous driving according to one embodiment.

[0020] Figure 6 is a block diagram showing an example of an autonomous driving data processing board according to one embodiment.

[0021] Figure 7 is a block diagram illustrating an example of a sensor device fault monitoring system based on dual-path Ethernet.

[0022] Figure 8 is a flow chart illustrating a method of performing sensor device fault monitoring based on dual-path Ethernet. DETAILED DESCRIPTION

[0023] Various embodiments and aspects of the present disclosure will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and the accompanying drawings are illustrative of the present disclosure and should not be construed as limiting the present disclosure. Many specific details are described to provide a thorough understanding of the various embodiments of the present disclosure. However, in some cases, in order to provide a concise discussion of the embodiments of the present disclosure, known or conventional details are not described.

[0024] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0025] As mentioned above, the sensor driver reports errors related to the sensor data path, such as data timeout, incomplete data, data error, etc. However, the sensor driver does not monitor the link layer path information and control path information. Therefore, the autonomous driving vehicle cannot effectively perform fault detection.

[0026] This article discloses a method for providing dual-path fault monitoring for Ethernet-based sensor devices. The method adds a sensor monitoring module on a secure computing unit to monitor and report faults in the link layer and control path, while the sensor driver continues to publish data and report errors related to the data path. By adding the sensor monitoring module, the autonomous vehicle continues to use the sensor driver to process sensor information in real time, while also monitoring the link layer and control path from the sensor.

[0027] According to one embodiment, an autonomous driving vehicle (ADV) processes a sensor data set generated by a sensor simultaneously by a first computing unit and a second computing unit. The first computing unit formats the sensor data set into a message data set. The second computing unit determines whether the sensor data set indicates a control path fault and reports the control path fault accordingly.

[0028] In one embodiment, the ADV receives a sensor data set from a sensor at an Ethernet switch and transmits the sensor data set from the Ethernet switch to a first computing unit and a second computing unit via a dedicated virtual local area network. In one embodiment, an autonomous driving vehicle includes an autonomous driving system including a first computing unit, a second computing unit, and an Ethernet switch.

[0029] In one embodiment, the first computing unit determines whether the sensor data set indicates a data path failure and reports the data path failure accordingly. In one embodiment, the first computing unit reports the data path failure to the central system monitor and the second computing unit reports the control path failure to the central system monitor.

[0030] In one embodiment, the second computing unit determines whether the sensor data set indicates a link layer failure and reports the failure accordingly. In one embodiment, the second computing unit generates a set of sensor data statistics in response to evaluating the sensor data set.

[0031] Figure 1 is a block diagram showing an autonomous driving network configuration according to an embodiment of the present disclosure. Figure 1, a network configuration 100 includes an autonomous driving vehicle (ADV) 101, which can be communicatively coupled to one or more servers 103-104 via a network 102. Although one ADV is shown, multiple ADVs can be coupled to each other and / or to servers 103-104 via network 102. Network 102 can be any type of network, such as a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a cellular network, a satellite network, or a wired or wireless combination thereof. Servers 103-104 can be any type of server or server cluster, such as a web or cloud server, an application server, a back-end server, or a combination thereof. Servers 103-104 can be data analysis servers, content servers, traffic information servers, maps and points of interest (MPOI) servers, or location servers, etc.

[0032] ADV refers to a vehicle that can be configured to be in an automatic mode, in which the vehicle navigates an environment with little or no input from the driver. Such an ADV may include a sensor system having one or more sensors that are configured to detect information about the environment in which the vehicle is operating. The vehicle and its associated controller use the detected information to navigate the environment. ADV 101 can operate in a manual mode, a fully automatic mode, or a partially automatic mode.

[0033] In one embodiment, ADV 101 includes, but is not limited to, an autonomous driving system (ADS) 110, a vehicle control system 111, a wireless communication system 112, a user interface system 113, and a sensor system 115. ADV 101 may also include certain common components included in an ordinary vehicle, such as an engine, wheels, a steering wheel, a transmission, etc., which may be controlled by the vehicle control system 111 and / or ADS 110 using various communication signals and / or commands, such as acceleration signals or commands, deceleration signals or commands, steering signals or commands, and braking signals or commands, etc.

[0034] The components 110-115 may be communicatively coupled to each other via an interconnect, a bus, a network, or a combination thereof. For example, the components 110-115 may be communicatively coupled to each other via a controller area network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host computer. It is a message-based protocol that was originally designed for multi-way electrical wiring within automobiles, but is also used in many other environments.

[0035] Reference now Figure 2In one embodiment, the sensor system 115 includes, but is not limited to, one or more cameras 211, a global positioning system (GPS) unit 212, an inertial measurement unit (IMU) 213, a radar unit 214, and a light detection and ranging (LIDAR) unit 215. The GPS system 212 may include a transceiver that is operable to provide information about the location of the ADV. The IMU unit 213 may sense the position and direction changes of the ADV based on inertial acceleration. The radar unit 214 may represent a system that uses radio signals to sense objects within the local environment of the ADV. In some embodiments, in addition to sensing objects, the radar unit 214 may also sense the speed and / or orientation of the object. The LIDAR unit 215 may use lasers to sense objects in the environment in which the ADV is located. The LIDAR unit 215 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components. The camera 211 may include one or more devices to capture images of the environment surrounding the ADV. The camera 211 may be a still camera and / or a video camera. The camera may be mechanically movable, for example, by mounting the camera on a rotating and / or tilting platform.

[0036] The sensor system 115 may also include other sensors, such as a sonar sensor, an infrared sensor, a steering sensor, a throttle sensor, a brake sensor, and an audio sensor (e.g., a microphone). The audio sensor may be configured to capture sounds of the environment surrounding the ADV. The steering sensor may be configured to sense the steering angle of the steering wheel, wheels, or a combination thereof of the vehicle. The throttle sensor and the brake sensor sense the throttle position and the brake position of the vehicle, respectively. In some cases, the throttle sensor and the brake sensor may be integrated into an integrated throttle / brake sensor.

[0037] In one embodiment, the vehicle control system 111 includes but is not limited to a steering unit 201, a throttle unit 202 (also referred to as an acceleration unit), and a brake unit 203. The steering unit 201 is used to adjust the direction or orientation of the vehicle. The throttle unit 202 is used to control the speed of the motor or engine, thereby controlling the speed and acceleration of the vehicle. The brake unit 203 slows down the speed of the wheels or tires of the vehicle by providing friction, thereby decelerating the vehicle. Please note that Figure 2 The components shown in the figure may be implemented in hardware, software or a combination thereof.

[0038] Return to reference Figure 1, the wireless communication system 112 is used to allow communication between the ADV 101 and external systems (such as devices, sensors, other vehicles, etc.). For example, the wireless communication system 112 can wirelessly communicate with one or more devices directly or through a communication network (such as servers 103-104 on the network 102). The wireless communication system 112 can use any cellular communication network or wireless local area network (WLAN), such as using WiFi to communicate with another component or system. The wireless communication system 112 can communicate directly with devices (such as a passenger's mobile device, a display device, a speaker in the vehicle 101) using, for example, an infrared link, Bluetooth, etc. The user interface system 113 can be part of a peripheral device implemented in the vehicle 101, including, for example, a keyboard, a touch screen display device, a microphone, and a speaker.

[0039] Some or all functions of ADV 101 may be controlled or managed by ADS 110, particularly when operating in an autonomous driving mode. ADS 110 includes necessary hardware (e.g., processor, memory, storage) and software (e.g., operating system, planning and routing programs) to receive information from sensor system 115, control system 111, wireless communication system 112, and / or user interface system 113, process the received information, plan a route or path from a starting point to a destination point, and then drive vehicle 101 based on the planning and control information. Alternatively, ADS 110 may be integrated with vehicle control system 111.

[0040] For example, a user as a passenger may specify a starting location and a destination of a trip, for example, through a user interface. ADS 110 obtains trip-related data. For example, ADS 110 may obtain location and route data from an MPOI server, which may be part of servers 103-104. The location server provides location services, and the MPOI server provides map services and POIs of certain locations. Alternatively, such location and MPOI information may be cached locally in a persistent storage device of ADS 110.

[0041] As ADV 101 moves along the route, ADS 110 may also obtain real-time traffic information from a traffic information system or server (TIS). Note that servers 103-104 may be operated by a third-party entity. Alternatively, the functionality of servers 103-104 may be integrated with ADS 110. Based on real-time traffic information, MPOI information, and location information, as well as real-time local environmental data (e.g., obstacles, objects, nearby vehicles) detected or sensed by sensor system 115, ADS 110 may plan the best route and drive vehicle 101 according to the planned route, such as through control system 111, to reach a designated destination safely and efficiently.

[0042] Figure 3and Figure 4 is a block diagram illustrating an example of an autonomous driving system for use with an ADV according to one embodiment. The system 300 may be implemented as Figure 1 Part of ADV 101, Figure 1 It includes but is not limited to ADS 110 , control system 111 and sensor system 115 . ADS 110 includes but is not limited to positioning module 301 , perception module 302 , prediction module 303 , decision module 304 , planning module 305 , control module 306 and route module 307 .

[0043] Some or all of the modules 301-307 may be implemented in software, hardware, or a combination thereof. For example, these modules may be installed in persistent storage device 352, loaded into memory 351, and executed by one or more processors (not shown). Note that some or all of these modules may be implemented in conjunction with Figure 2 Some or all modules of the vehicle control system 111 are communicatively coupled or integrated. Some of the modules 301-307 can be integrated together as an integrated module.

[0044] The positioning module 301 determines the current location of the ADV 300 (e.g., using the GPS unit 212) and manages any data related to the user's itinerary or route. The positioning module 301 (also referred to as the map and route module) manages any data related to the user's itinerary or route. The user can log in and specify the starting location and destination of the trip, such as through a user interface. The positioning module 301 communicates with other components of the ADV 300, such as the map and route data 311, to obtain itinerary related data. For example, the positioning module 301 can obtain location and route data from a location server and a map and POI (MPOI) server. The location server provides location services, and the MPOI server provides map services and POIs at certain locations, which can be cached as part of the map and route data 311. When the ADV 300 moves along the route, the positioning module 301 can also obtain real-time traffic information from a traffic information system or server.

[0045] Based on the sensor data provided by the sensor system 115 and the positioning information obtained by the positioning module 301, the perception module 302 determines the perception of the surrounding environment. The perception information can represent the perception of the environment around the vehicle driven by the average driver. The perception can include lane configuration, traffic light signals, the relative position of another vehicle, pedestrians, buildings, crosswalks or other traffic-related signs (such as stop signs, yield signs), etc., for example in the form of objects. The lane configuration includes information describing one or more lanes, such as the shape of the lane (e.g., straight or curved), the width of the lane, how many lanes there are in the road, one-way lanes or two-way lanes, merging lanes or diverging lanes, exit lanes, etc.

[0046] The perception module 302 may include a computer vision system or functionality of a computer vision system to process and analyze images captured by one or more cameras to identify objects and / or features in the environment of the ADV. Objects may include traffic signals, road boundaries, other vehicles, pedestrians, and / or obstacles, etc. The computer vision system may use object recognition algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system may map the environment, track objects, and estimate the speed of objects, etc. The perception module 302 may also detect objects based on other sensor data provided by other sensors such as radar and / or LIDAR.

[0047] For each object, the prediction module 303 predicts how the object will behave in this situation. The prediction is performed based on the driving environment data sensed at the current moment, taking into account the map / route information set (311) and the traffic rules 312. For example, if the object is a vehicle in the opposite direction and the current driving environment includes an intersection, the prediction module 303 will predict whether the vehicle is likely to go straight or turn. If the perception data indicates that there is no traffic light at the intersection, the prediction module 303 may predict that the vehicle may have to come to a complete stop before entering the intersection. If the perception data indicates that the vehicle is currently in a left-turn-only lane or a right-turn-only lane, the prediction module 303 can predict that the vehicle is more likely to turn left or right, respectively.

[0048] For each object, decision module 304 makes a decision about how to handle the object. For example, for a particular object (e.g., another vehicle in an intersecting path) and metadata describing the object (e.g., speed, direction, turn angle), decision module 304 decides how to encounter the object (e.g., overtake, yield, stop, pass). Decision module 304 can make such decisions based on a set of rules (e.g., traffic rules or driving rules 312), which can be stored in persistent storage device 352.

[0049] The route module 307 is configured to provide one or more routes or paths from the starting point to the destination point. For a given trip from the starting location to the destination location (e.g., a trip received from a user), the route module 307 obtains route and map information 311 and determines all possible routes or paths from the starting location to the destination location. The route module 307 can generate a reference line in the form of a topographic map for each route determined from the starting location to the destination location. The reference line refers to an ideal route or path that is not interfered with by, for example, other vehicles, obstacles, or traffic conditions. That is, if there are no other vehicles, pedestrians, or obstacles on the road, the ADV should accurately or closely follow the reference line. The topographic map is then provided to the decision module 304 and / or the planning module 305. The decision module 304 and / or the planning module 305 examine all possible routes to select and modify one of the optimal routes based on other data provided by other modules, such as traffic conditions from the positioning module 301, the driving environment perceived by the perception module 302, and traffic conditions predicted by the prediction module 303. The actual path or route used to control the ADV may be close to or different from the reference line provided by the route module 307, depending on the specific driving environment at the moment.

[0050] Based on the decision for each perceived object, the planning module 305 uses the reference line provided by the route module 307 as a basis to plan a path or route for the ADV, as well as driving parameters (such as distance, speed and / or turning angle). That is, for a given object, the decision module 304 decides what to do with the object, and the planning module 305 decides how to do it. For example, for a given object, the decision module 304 may decide to pass through the object, and the planning module 305 may determine whether to pass from the left or right side of the object. Planning and control data are generated by the planning module 305 and include information describing how the vehicle 101 will move in the next movement cycle (e.g., the next route / path segment). For example, the planning and control data may instruct the vehicle 101 to move 10 meters at a speed of 30 miles per hour (mph) and then change to the right lane at a speed of 25 miles per hour.

[0051] Based on the planning and control data, the control module 306 controls and drives the ADV according to the route or path defined by the planning and control data by sending appropriate commands or signals to the vehicle control system 111. The planning and control data includes sufficient information to drive the vehicle along the path or route from a first point on the route or path to a second point using appropriate vehicle settings or driving parameters (e.g., throttle, brake, steering commands) at different current moments.

[0052] In one embodiment, the planning phase is performed within a plurality of planning cycles (also referred to as driving cycles), for example, within a time interval of 100 milliseconds (ms). For each planning cycle or driving cycle, one or more control commands will be issued based on the planning and control data. That is, for every 100ms, the planning module 305 plans the next route segment or path segment, for example, including the target position and the time required for the ADV to reach the target position. Alternatively, the planning module 305 can also specify a specific speed, direction and / or steering angle, etc. In one embodiment, the planning module 305 plans the route segment or path segment for the next predetermined time period (e.g., 5 seconds). For each planning cycle, the planning module 305 plans the target position of the current cycle (e.g., the next 5 seconds) based on the target position planned in the previous cycle. Then, the control module 306 generates one or more control commands (e.g., throttle, brake, steering control commands) based on the planning and control data of the current cycle.

[0053] Note that the decision module 304 and the planning module 305 can be integrated into an integrated module. The decision module 304 / planning module 305 can include a navigation system or the functionality of a navigation system to determine the driving path of the ADV. For example, the navigation system can determine a series of speed and direction headings to influence the ADV to move along a path that substantially avoids perceived obstacles, while the ADV generally moves along a road-based path to the final destination. The destination can be set based on user input via the user interface system 113. The navigation system can dynamically update the driving path while the ADV is running. The navigation system can combine data from a GPS system and one or more maps to determine the driving path of the ADV.

[0054] Figure 5 5 is a block diagram showing a system architecture for autonomous driving according to one embodiment. The system architecture 500 may be represented as follows: Figure 1 The system architecture of the autonomous driving system is shown in Figure 2. Figure 5, the system architecture 500 includes, but is not limited to, an application layer 501, a planning and control (PNC) layer 502, a perception layer 503, a driver layer 504, a firmware layer 505, and a hardware layer 506. The application layer 501 may include a user interface or configuration application that interacts with a user or passenger of the autonomous vehicle, such as functionality associated with the user interface system 113. The PNC layer 502 may include functionality of the perception and planning system 110 and the control system 111. The perception layer 503 may include functionality of at least the perception and planning system 110. The firmware layer 505 may represent functionality of at least the sensor system 115, which may be implemented in the form of a field programmable gate array (FPGA). The hardware layer 506 may represent hardware of the autonomous vehicle, such as the control system 111. Layers 501-503 may communicate with the firmware layer 505 and the hardware layer 506 via the device driver layer 504.

[0055] Figure 6 is a block diagram showing an example of an autonomous driving data processing board according to one embodiment. Figure 6 , the sensor system 115 includes a plurality of sensors 610 and a sensor unit 600 coupled to the ADS 110. The ADS 110 may include Figure 3 . The sensor unit 600 may be configured, for example, in the form of an FPGA device or an ASIC (application specific integrated circuit) device. In one embodiment, the sensor unit 600 includes, among other things, one or more sensor data processing modules 601 (referred to herein as sensor processing modules), a data transmission module 602, and a sensor control module or logic 603. The modules 601-603 may communicate with the sensor 610 via a sensor interface 604 and communicate with the ADS 110 via a host interface 605. In one embodiment, an internal or external buffer 606 may be utilized to buffer data for processing.

[0056] In one embodiment, the sensor 610 may include a GPS receiver / unit, an IMU, and a LIDAR unit. The GPS unit and the IMU may be coupled together with the sensor unit 600 on a single FPGA or ASIC and is referred to as an inertial measurement unit (INS). In one embodiment, the sensor 610 includes a first IMU as a primary IMU and a second IMU as a redundant or backup IMU, which may be independently powered by a separate power supply circuit (e.g., a voltage regulator). The sensor processing module 601 may include logic to receive data from the GPS unit and the IMU and combine the data (e.g., using a Kalman filter) to estimate the position of the automated vehicle. The sensor processing module 601 may also include logic to compensate for GPS data bias due to propagation delays in the GPS data.

[0057] In one embodiment, for the receive path or upstream direction, the sensor processing module 601 is configured to receive sensor data from the sensor through the sensor interface 604 and process the sensor data (e.g., format conversion, error checking), which may be temporarily stored in the buffer 606. The data transmission module 602 is configured to transmit the processed data to the host system 110 using a communication protocol compatible with the host interface 605. Similarly, for the transmit path or downstream direction, the data transmission module 602 is configured to receive data or commands from the host system 110. The data is then processed by the sensor processing module 601 into a format compatible with the corresponding sensor. The processed data is then transmitted to the sensor.

[0058] In one embodiment, the sensor control module or logic 603 is configured to control certain operations of the sensor 610, such as activation timing for capturing sensor data, in response to commands received from a host system (e.g., perception module 302) via a host interface 605. The ADS 110 can configure the sensor 610 to capture sensor data in a collaborative and / or synchronized manner so that the sensor data can be used to perceive the driving environment around the vehicle at any current point in time.

[0059] Sensor interface 604 may include one or more of the following: Ethernet, USB (Universal Serial Bus), LTE (Long Term Evolution) or cellular, WiFi, GPS, camera, CAN, serial (e.g., Universal Asynchronous Receiver / Transmitter or UART), SIM (Subscriber Identity Module) card, and other general purpose input / output (GPIO) interfaces. Host interface 605 may be any high-speed or high-bandwidth interface, such as a PCIe (Peripheral Component Interconnect or PCI Express) interface. Sensor 610 may include various sensors used in ADVs, such as cameras, radar equipment, GPS receivers, IMUs, ultrasonic sensors, GNSS (Global Navigation Satellite System) receivers, LTE or cellular SIM cards, vehicle sensors (e.g., throttle, brake, steering sensors), and system sensors (e.g., temperature, humidity, pressure sensors), etc.

[0060] For example, a camera may be coupled via Ethernet or a GPIO interface. A GPS sensor may be coupled via USB or a specific GPS interface. A vehicle sensor may be coupled via a CAN interface. A radar sensor or an ultrasonic sensor may be coupled via a GPIO interface. Similarly, an internal SIM module may be inserted into a SIM slot of the sensor unit 600. A serial interface such as a UART may be coupled to a console system for debugging.

[0061] The sensor 610 can be any type of sensor and is provided by various manufacturers or suppliers. The sensor processing module 601 is configured to process different types of sensors and their corresponding data formats and communication protocols. According to one embodiment, each sensor 610 is associated with a specific channel for processing sensor data and transmitting the processed sensor data between the ADS 110 and the corresponding sensor. Each channel may include a specific sensor processing module and a specific data transmission module, which have been configured or programmed to process the corresponding sensor data and protocol.

[0062] In one embodiment, the sensor processing module parses the raw data from the sensor 610 and formats the parsed raw data into messages, which are sent to the ADS110 for further processing. In one embodiment, the monitoring system actively collects sensor information in various ways through available automotive standard compatible protocols, integrates plug-and-play distributed components, and communicates with flexible software application clients to achieve an open and secure monitoring mechanism. This can serve a single chip and sensor architecture, or a heterogeneous integrated computing and sensor system. The monitoring system is based on a publisher / subscriber messaging system to allow the components involved to communicate freely with each other regarding operating status or control.

[0063] The ADDP board 620 includes a main computing unit 615, which includes hardware, software, firmware, etc. to perform the functions of the sensor unit and the ADS 110. The main computing unit 615 can be in the form of an ASIC, FPGA, CPU, GPU, processor accelerator, and other logic circuits, or a combination thereof. Figure 7 As shown, the method disclosed herein adds one or more secure computing units on, for example, the ADDP board 620 to monitor the link layer and control path of the sensor data and generate faults when errors are detected (see for more details). Figure 7 and the corresponding text).

[0064] Figure 7 is a block diagram illustrating an example of a sensor device fault monitoring system based on dual-path Ethernet. Figure 7 610 is shown coupled to a sensor interface 604, which is an Ethernet switch. Sensor 601 provides sensor data 740 to sensor interface 604. Sensor interface 604 is configured to send sensor data 740 to main computing unit 615 and secure computing unit 700. Main computing unit includes a sensor driver 720 corresponding to sensor 610. In one embodiment, sensor driver 720 is Figure 5 A portion of device driver layer 504 is shown.

[0065] The sensor driver 720 runs on the main computing unit 615, for example Figure 6 The sensor driver parses the sensor data 740 and generates message data 760 from the parsed sensor data 740. The main computing unit 615 sends the message data 760 to the data processing module 725, which may include Figure 3 One or more of the modules 301-307 shown. The main computing unit 615 also checks for data path failures and sends data failure reports 770 to the central system monitor 730 as needed.

[0066] The safety computing unit 700 receives the sensor data 740 and analyzes the link layer and control path information using the sensor monitor 715. When the safety computing unit 700 detects a link layer and / or control path failure, the safety computing unit 700 sends a link layer and control path failure report 780 to the central system monitor 730. The central system monitor 730 processes the data failure report 770 and the link layer and control path failure report 780 and takes corrective action accordingly (e.g., notifying the driver of the ADV or sending a message to a remote monitoring service).

[0067] In one embodiment, the central system monitor 730 generates a log that communicates various status parameters of the global navigation satellite system (GNSS) receiver system, including receiver status and an error word, which includes several flags that specify status and error conditions. When an error occurs (displayed in the receiver error word), the receiver idles all channels, turns off the antenna and disables the RF hardware, as these conditions are considered fatal errors. The log includes a variable number of status words to allow maximum flexibility and future expansion. In one embodiment, the GNSS sensor monitor parses the RXSTATUS packet and reports errors such as DRAM errors, invalid firmware errors, ROM status errors, ESN access errors, authorization code errors, power supply voltage errors, temperature out of range errors, MINOS status errors, PLL RF status errors, and / or NVM status errors.

[0068] Figure 8 800 is a flow chart showing a method for performing sensor device fault monitoring based on dual-path Ethernet. Process 800 may be performed by processing logic, which may include software, hardware, or a combination thereof. For example, process 800 may be performed by secure computing unit 700 running on ADDP board 620, such as Figure 7 shown.

[0069] At operation 810, processing logic processes a sensor data set simultaneously by a first computing unit and a second computing unit. The sensor data set is generated by a sensor. At operation 812, when processing the sensor data set, the first computing unit formats the sensor data set into a message data set. Raw data is generated by the sensor and is included in the sensor data set. In addition, at operation 814, when processing the sensor data set, the second computing unit determines whether the sensor data set indicates a control path fault. At operation 816, in response to determining that the sensor data set indicates a control path fault, the second computing unit reports the control path fault accordingly.

[0070] Note that some or all of the components shown and described above may be implemented in software, hardware, or a combination thereof. For example, these components may be implemented as software installed and stored in a persistent storage device, which may be loaded and executed by a processor (not shown) in a memory to perform the processes or operations described throughout this application. Alternatively, these components may be implemented as executable code, programmed or embedded in dedicated hardware, such as an integrated circuit (e.g., an application-specific IC or ASIC), a digital signal processor (DSP), or a field programmable gate array (FPGA), which may be accessed from an application via a corresponding driver and / or operating system. In addition, these components may be implemented as specific hardware logic in a processor or processor core and accessed by a software component via one or more specific instructions as part of an instruction set.

[0071] Some portions of the foregoing detailed descriptions are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. These operations require physical manipulations of physical quantities.

[0072] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless it is apparent from the foregoing discussion that otherwise specifically stated, it should be understood that throughout the description, discussions using terms such as those set forth in the following claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates and converts data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0073] Embodiments of the present disclosure also relate to an apparatus for performing the operations herein. Such a computer program is stored in a non-transitory computer-readable medium. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., a read-only memory ("ROM"), a random access memory ("RAM"), a disk storage medium, an optical storage medium, a flash memory device).

[0074] The process or method described in the preceding figures may be performed by a processing logic including hardware (e.g., circuits, dedicated logic, etc.), software (e.g., embedded on a non-transitory computer-readable medium), or a combination of both. Although the process or method is described above according to some sequential operations, it should be understood that some of the operations described may be performed in a different order. In addition, some operations may be performed in parallel rather than sequentially.

[0075] The embodiments of the present disclosure are not described with reference to any particular programming language. It should be appreciated that a variety of programming languages ​​can be used to implement the teachings of the embodiments of the present disclosure as described herein.

[0076] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A computer-implemented method for operating an autonomous vehicle, the method comprising: The sensor data set is processed simultaneously by the first computing unit and the second computing unit, wherein the sensor data set is generated by a sensor, and the processing further comprises: formatting, by the first computing unit, the sensor data set into a message data set; determining, by the second computing unit, whether the sensor data set indicates a control path fault; and The control path fault is reported by the second computing unit in response to determining that the sensor data set indicates the control path fault.

2. The method according to claim 1, further comprising: receiving the sensor data set from the sensor at an Ethernet switch; as well as The sensor data set is transmitted from the Ethernet switch to the first computing unit and the second computing unit via a dedicated virtual local area network.

3. The method according to claim 2, wherein the autonomous driving vehicle includes an autonomous driving system, and the autonomous driving system includes the first computing unit, the second computing unit and the Ethernet switch.

4. The method according to claim 1, further comprising: determining, by the first computing unit, whether the sensor data set indicates a data path failure; as well as The data path failure is reported, by the first computing unit, based on determining that the sensor data set indicates the data path failure. 5 . The method of claim 4 , wherein the first computing unit reports the data path failure to a central system monitor, and wherein the second computing unit reports the control path failure to the central system monitor.

6. The method according to claim 1, further comprising: determining, by the second computing unit, whether the sensor data set indicates a link layer failure; as well as The link layer failure is reported by the second computing unit based on determining that the sensor data set indicates the link layer failure.

7. The method according to claim 1, further comprising: A set of sensor data statistics is generated by the second computing unit in response to evaluating the sensor data set.

8. A non-transitory machine-readable medium having instructions stored therein, which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.

9. A system for operating an autonomous vehicle, comprising: Processing equipment; as well as A memory for storing instructions, which, when executed by the processing device, causes the processing device to execute the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 7.

Images