Domain isolation in automotive automated driving systems

Abstract

This disclosure provides systems, methods, and apparatus for a vehicle having an automated driving system. In a first aspect, a method of isolation in an automated driving system includes detecting an error in a first domain of the automated driving system; isolating a second domain of the automatic driving system from the first domain; maintaining operation of the second domain after isolating the second domain from the first domain; and bypassing the first domain by the second domain to send a notification to the external controller via the first communication interface. Other aspects and features are also claimed and described.

Description

Cross-references to related applications

[0001] This application claims the benefit of U.S. Patent Application No. 18 / 309,209, filed April 28, 2023, entitled “DOMAIN ISOLATION IN ANAUTOMOTIVE AUTOMATED DRIVING SYSTEM”, the entire contents of which are expressly incorporated herein by reference. Technical Field

[0002] In summary, aspects of this disclosure relate to vehicles with autonomous driving systems, and more specifically, aspects of this disclosure relate to methods and systems suitable for providing domain isolation in autonomous driving systems. Background Technology

[0003] Vehicles come in many shapes and sizes, are propelled by various propulsion technologies, and carry cargo, including humans, animals, or objects. These machines have achieved the movement of cargo over long distances, at high speeds, and with a magnitude greater than that that could be achieved through human effort. Initially, humans drove vehicles to control the speed and direction of the cargo to its destination. Human operation of vehicles has led to numerous unfortunate incidents resulting from collisions between vehicles, between vehicles and objects, between vehicles and humans, or between vehicles and animals. As research into vehicle automation has progressed, various autonomous driving systems have been developed and introduced. These include GPS-guided navigation, adaptive cruise control, lane change assist, collision avoidance systems, night vision, parking assist, blind spot detection, lane keeping assist, automatic braking, and even fully autonomous driving.

[0004] As autonomous driving systems develop, the demands on the computing power to support them also increase. Vehicles may include multiple computing systems, which can be general-purpose or tailored to specific autonomous driving functions. Furthermore, depending on the features supported by these computing systems, they can be designed to meet different safety standards. Summary of the Invention

[0005] The following outlines some aspects of this disclosure to provide a basic understanding of the techniques discussed. This overview is not a comprehensive summary of all intended features of this disclosure, and is neither intended to identify key or essential elements of all aspects of this disclosure, nor to depict the scope of any or all aspects of this disclosure. Its sole purpose is to present some concepts of one or more aspects of this disclosure in an overview form as a prelude to the more detailed description that follows.

[0006] Distractions in human operators of vehicles are a contributing factor to many vehicle collisions. Driver distraction can include changing the radio, observing events outside the vehicle, and using electronic devices. Sometimes, environmental conditions can create situations that even an attentive driver cannot recognize in time to prevent a collision. Aspects of this disclosure provide improved systems for assisting drivers in vehicles by leveraging enhanced situational awareness and / or autonomous driving capabilities.

[0007] The example embodiment provides isolation between domains of an automotive System-on-a-Chip (SOC) for an automated driving system (such as an assisted or autonomous driving system) to facilitate the continued operation of a second domain of the SOC when an error is encountered in the first domain. For example, an error can be detected in the first domain of the SOC, and the second domain of the SOC can be isolated from the first domain in response to the detected error. This isolation allows the operation of the second domain of the SOC to be maintained even when the first domain becomes inoperable due to an error. Furthermore, the second domain (which can communicate with an external controller of the automated driving system via the first domain when not isolated from the first domain) can bypass the first domain and can communicate with the external controller via a first communication interface when isolated from the first domain. The isolation of the second domain from the first domain and the continued communication between the second domain and the external controller allow the second domain to continue operating in a degraded mode, enabling the second domain to control the vehicle to, for example, safely stop and shut down the engine or recover from an error in the first domain and deactivate the isolation between the first and second domains.

[0008] In one aspect of this disclosure, a method for domain isolation in an autonomous driving system includes: detecting an error in a first domain of the autonomous driving system; isolating a second domain of the autonomous driving system from the first domain; maintaining operation of the second domain of the autonomous driving system after isolating the second domain from the first domain; and after isolating the second domain of the autonomous driving system from the first domain, having the second domain bypass the first domain to send one or more notifications to the external controller via a first communication interface between the second domain and the external controller.

[0009] In an additional aspect of this disclosure, an apparatus includes at least one processor and memory coupled to the at least one processor. The at least one processor is configured to perform operations including: detecting an error in a first domain of an autonomous driving system, wherein the autonomous driving system includes the processor; isolating a second domain of the autonomous driving system from the first domain; maintaining operation of the second domain of the autonomous driving system after isolating the second domain from the first domain; and, after isolating the second domain of the autonomous driving system from the first domain, sending one or more notifications to the external controller via a first communication interface between the second domain and the external controller, bypassing the first domain.

[0010] In an additional aspect of this disclosure, an apparatus includes: a unit for receiving image data from an image sensor; a unit for detecting errors in a first domain of an autonomous driving system, wherein the autonomous driving system includes the processor; a unit for isolating a second domain of the autonomous driving system from the first domain; a unit for maintaining operation of the second domain of the autonomous driving system after isolating the second domain from the first domain; and a unit for, after isolating the second domain of the autonomous driving system from the first domain, sending one or more notifications from the second domain to the external controller via a first communication interface between the second domain and the external controller.

[0011] In a further aspect of this disclosure, a non-transitory computer-readable medium stores instructions that, when executed by a processor, cause the processor to perform operations. The operations include: detecting an error in a first domain of an autonomous driving system, wherein the autonomous driving system includes the processor; isolating a second domain of the autonomous driving system from the first domain; maintaining operation of the second domain of the autonomous driving system after isolating the second domain from the first domain; and, after isolating the second domain of the autonomous driving system from the first domain, sending one or more notifications from the second domain to the external controller via a first communication interface between the second domain and the external controller, bypassing the first domain.

[0012] The foregoing has provided a fairly broad overview of the features and technical advantages of examples according to this disclosure in order to facilitate a better understanding of the detailed description that follows. Additional features and advantages will be described below. The disclosed concepts and specific examples can be readily used as the basis for modifying or designing other structures for performing the same purpose as this disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein (both their organization and manner of operation) and their associated advantages will be better understood from the following description when considered in conjunction with the accompanying drawings. Each drawing is provided for illustrative and descriptive purposes and not as a limitation of the claims.

[0013] In various implementations, the technologies and apparatus described can be used in wireless communication networks such as: Code Division Multiple Access (CDMA) networks, Time Division Multiple Access (TDMA) networks, Frequency Division Multiple Access (FDMA) networks, Orthogonal FDMA (OFDMA) networks, Single Carrier FDMA (SC-FDMA) ng networks, LTE networks, GSM networks, 5th Generation (5G) or New Radio (NR) networks (sometimes referred to as "5G NR" networks, systems, or devices), and other communication networks. As described herein, the terms "network" and "system" are used interchangeably.

[0014] For example, CDMA networks can implement radio technologies such as Universal Terrestrial Radio Access (UTRA) and CDMA2000. UTRA includes Wideband CDMA (W-CDMA) and Low Code Rate (LCR). CDMA2000 covers the IS-2000, IS-95, and IS-856 standards.

[0015] For example, TDMA networks can implement radio technologies such as the Global System for Mobile Communications (GSM). The 3rd Generation Partnership Project (3GPP) defines the standard for the GSM EDGE (Enhanced Data Rate for GSM Evolution) Radio Access Network (RAN) (also referred to as GERAN). GERAN, along with the network connecting base stations (e.g., Ater and Abis interfaces) and base station controllers (e.g., A interface), is the radio component of GSM / EDGE. The Radio Access Network represents the component of a GSM network through which telephone calls and packet data are routed from the Public Switched Telephone Network (PSTN) and the Internet to subscriber handsets (also referred to as user terminals or user equipment (UEs)) and from subscriber handsets to the PSTN and the Internet. A mobile phone operator's network may include one or more GERANs; in the case of UMTS / GSM networks, GERAN may be coupled with UTRAN. Additionally, an operator's network may include one or more LTE networks or one or more other networks. Different network types can use different Radio Access Technologies (RAT) RANs.

[0016] OFDMA networks can implement radio technologies such as Evolved UTRA (E-UTRA), IEEE 802.11, IEEE 802.16, IEEE 802.20, and Flash OFDM. UTRA, E-UTRA, and GSM are part of the Universal Mobile Telecommunications System (UMTS). Specifically, Long Term Evolution (LTE) is a version of UMTS using E-UTRA. UTRA, E-UTRA, GSM, UMTS, and LTE are described in documents from an organization called the 3rd Generation Partnership Project (3GPP), and cdma2000 is described in documents from an organization called the 3rd Generation Partnership Project 2 (3GPP2). 5G networks include diverse deployments, diverse spectrum, and diverse services and devices that can be achieved using a unified OFDM-based air interface.

[0017] This disclosure may refer to LTE, 4G, or 5G NR technologies in certain aspects; however, this description is not intended to be limited to any particular technology or application, and one or more aspects described with reference to one technology may be understood to be applicable to another technology. Additionally, one or more aspects of this disclosure may relate to shared access to radio spectrum between networks using different radio access technologies or radio air interfaces.

[0018] Devices, networks, and systems can be configured to communicate via one or more portions of the electromagnetic spectrum. The electromagnetic spectrum is often subdivided into various categories, bands, or channels based on frequency or wavelength. In 5G NR, two initial operating bands have been designated as frequency range names FR1 (410 MHz – 7.125 GHz) and FR2 (24.25 GHz – 52.6 GHz). Frequencies between FR1 and FR2 are often referred to as mid-band frequencies. Although a portion of FR1 is greater than 6 GHz, FR1 is often (interchangeably) referred to as the “sub-6 GHz” band in various documents and articles. Similar naming issues sometimes arise regarding FR2; although FR2 differs from the extremely high frequency (EHF) band (30 GHz – 300 GHz), it is often (interchangeably) referred to as the “millimeter wave” (mmWave) band in documents and articles, while the EHF band is designated as the “mmWave” band by the International Telecommunication Union (ITU).

[0019] In light of the foregoing, unless otherwise specifically stated, it should be understood that the term "sub-6 GHz," if used herein, can broadly refer to frequencies that are less than 6 GHz, within FR1, or may include intermediate frequency band frequencies. Furthermore, unless otherwise specifically stated, it should be understood that the term "mmWave," if used herein, can broadly refer to frequencies that may include intermediate frequency band frequencies, within FR2, or within the EHF band.

[0020] 5G NR devices, networks, and systems can be implemented using optimized OFDM-based waveform characteristics. These characteristics can include: scalable digital schemes (numerology) and transmission time intervals (TTI); a common, flexible framework to efficiently multiplex services and features using dynamic, low-latency time-division duplex (TDD) or frequency-division duplex (FDD) designs; and improved radio technologies such as massive MIMO, robust mmWave transport, advanced channel decoding, and device-centric mobility. The scalability of the digital scheme in 5G NR (with scaling of subcarrier spacing) can efficiently address the operation of different services across different spectrums and deployments. For example, in various outdoor and macro coverage deployments implemented with FDD or TDD below 3 GHz, subcarrier spacing can occur at 15 kHz over bandwidths such as 1, 5, 10, and 20 MHz. For other various outdoor and small cell coverage deployments with TDD above 3 GHz, subcarrier spacing can occur at 30 kHz over an 80 / 100 MHz bandwidth. For various other indoor broadband implementations using TDD on the unlicensed portion of the 5 GHz band, subcarrier spacing can occur at 60 kHz over a 160 MHz bandwidth. Finally, for various deployments utilizing mmWave components of TDD at 28 GHz, subcarrier spacing can occur at 120 kHz over a 500 MHz bandwidth.

[0021] For clarity, certain aspects of the apparatus and technology may be described below with reference to example 5G NR implementations or in a 5G-centric manner, and 5G terminology may be used as illustrative examples in various sections described below; however, the description is not intended to be limited to 5G applications.

[0022] Furthermore, it should be understood that, in operation, wireless communication networks adapted according to the concepts herein can operate using any combination of licensed or unlicensed spectrum, depending on load and availability. Therefore, it will be apparent to those skilled in the art that the systems, apparatuses, and methods described herein can be applied to other communication systems and applications besides the specific examples provided.

[0023] While aspects and implementations are described herein by way of example, those skilled in the art will understand that additional implementations and use cases may arise in many different arrangements and scenarios. The innovations described herein can be implemented across many different platform types, devices, systems, shapes, sizes, and package arrangements. For example, implementations or uses may arise via integrated chip implementations and other devices based on non-modular components (e.g., end-user devices, vehicles, communication devices, computing devices, industrial equipment, retail or purchasing devices, medical devices, AI-enabled devices, etc.). While some examples may or may not be specific to a particular use case or application, a wide variety of applicability to the described innovations is possible.

[0024] The scope of implementations can range from chip-level or modular components to non-modular, non-chip-level implementations, and further to aggregated, distributed, or original equipment manufacturer (OEM) devices or systems incorporating one or more of the described aspects. In some practical settings, devices incorporating the described aspects and features may also include additional components and features for the implementation and enforcement of the claimed and described aspects. The innovations described herein are intended to be practiced in a variety of implementations, including both large and small devices with different sizes, shapes, and constructions, chip-level components, multi-component systems (e.g., radio frequency (RF) chains, communication interfaces, processors), distributed arrangements, end-user equipment, etc.

[0025] In the following description, numerous specific details, such as examples of specific components, circuits, and processes, are set forth to provide a thorough understanding of this disclosure. As used herein, the term "coupled" means a direct connection or a connection via one or more intermediate components or circuits. Furthermore, specific terminology is set forth in the following description and for purposes of explanation to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that practicing the teachings disclosed herein may not require these specific details. In other instances, well-known circuits and devices are illustrated in block diagram form to avoid obscuring the teachings of this disclosure.

[0026] Some of the subsequent detailed descriptions are given based on procedures, logic blocks, processes, and other symbolic representations of operations on data bits in computer memory. In this disclosure, procedures, logic blocks, processes, etc., are considered as an ordered sequence of steps or instructions that produce a desired result. Steps are those that require physical manipulation of physical quantities. Typically, although not essential, these quantities are in the form of electronic or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computer system.

[0027] In the figures, a single box may be described as performing one or more functions. The one or more functions performed by that box may be performed in a single component or across multiple components, and / or may be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability between hardware and software, various illustrative components, boxes, modules, circuits, and steps are generally described below according to their functions. Whether such a function is implemented in hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functions in varying ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure. Furthermore, example devices may include components other than those shown, including well-known components such as processors, memory, etc.

[0028] Unless otherwise expressly stated (as will be apparent from the discussion below), it should be understood that throughout this application, the discussion using terms such as “access,” “receive,” “send,” “use,” “select,” “determine,” “normalize,” “multiply,” “average,” “monitor,” “compare,” “apply,” “update,” “measure,” “derive,” “solve,” “generate,” etc., refers to the actions and processes of a computer system or similar electronic computing device that manipulates data representing physical (electronic) quantities in the registers and memories of the computer system and converts them into other data that are similarly represented as physical quantities in the registers, memories, or other such information storage, transmission, or display devices of the computer system.

[0029] The terms "device" and "apparatus" are not limited to a single physical object or a specific number of physical objects (such as a smartphone, a camera controller, a processing system, etc.). As used herein, a device can be any electronic device having one or more components that can implement at least some parts of this disclosure. Although the term "device" is used in the following description and examples to describe various aspects of this disclosure, the term "device" is not limited to a specific configuration, type, or number of objects. As used herein, an apparatus can include a device for performing the described operations or a portion thereof.

[0030] As used herein (including in the claims), the term "or" when used in a list of two or more items means that any one of the listed items can be used alone, or any combination of two or more of the listed items can be used. For example, if a composition is described as containing components A, B, or C, then the composition can contain: only A; only B; only C; a combination of A and B; a combination of A and C; a combination of B and C; or a combination of A, B, and C.

[0031] Furthermore, as used herein (including in the claims), the “or” used in a list of items ending with “at least one of” indicates a separate list, such that a list such as “at least one of A, B or C” means A or B or C or AB or AC or BC or ABC (i.e., A and B and C) or any one of these items in any combination thereof.

[0032] Furthermore, as used herein, the term "substantially" is defined as being largely but not necessarily entirely specified (and includes being specified; for example, substantially 90 degrees includes 90 degrees, and substantially parallel includes parallel), as understood by one of ordinary skill in the art. In any disclosed implementation, the term "substantially" may be replaced by "within the specified (percentage)", where the percentage includes 0.1%, 1%, 5%, or 10%.

[0033] Furthermore, as used herein, unless otherwise stated, relative terms may be understood as a quantity relative to a reference. For example, terms such as “above” or “below” or “greater than” or “less than” may be understood as a higher, lower, larger, or smaller threshold than a reference value. Attached Figure Description

[0034] A further understanding of the nature and advantages of this disclosure can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals. Furthermore, various components of the same type may be distinguished by a dash following the reference numeral and a second reference numeral used to differentiate among similar components. If only the first reference numeral is used in the specification, the description applies to any of the similar components having the same first reference numeral, without regard to the second reference numeral.

[0035] Figure 1 This is a perspective view of a motor vehicle with an autonomous driving system according to an embodiment of the present disclosure.

[0036] Figure 2 A block diagram of an example autonomous driving SoC for a vehicle, based on one or more aspects of this disclosure, is shown.

[0037] Figure 3 It is a block diagram illustrating details of an example wireless communication system according to one or more aspects.

[0038] Figure 4 This is a block diagram illustrating details of an example autonomous system, including isolation between domains of an example autonomous driving SOC, according to one or more aspects of this disclosure.

[0039] Figure 5This is an example flowchart illustrating details of the operational state of an example autonomous driving SOC, including isolation between domains, according to one or more aspects of this disclosure.

[0040] Figure 6 This is an example state diagram illustrating the state flow of an example autonomous driving SOC, including isolation between domains, according to one or more aspects of this disclosure.

[0041] Figure 7 This is an example block diagram illustrating details of an example autonomous driving system, including isolation between domains of an example autonomous driving SOC, according to one or more aspects of this disclosure.

[0042] Figure 8 This is a flowchart illustrating an example method for isolating a second domain of an example autonomous driving SOC from a first domain of the autonomous driving SOC, according to one or more aspects of this disclosure.

[0043] Figure 9 This is a flowchart illustrating an example method for isolating a second domain of an autonomous driving SOC from a first domain of the autonomous driving SOC, according to one or more aspects of this disclosure.

[0044] Figure 10 This is a flowchart illustrating an example method for isolating a second domain of an autonomous driving SOC from a first domain of the autonomous driving SOC, according to one or more aspects of this disclosure.

[0045] Similar reference numerals and naming conventions are used in the various figures to indicate similar elements. Detailed Implementation

[0046] The detailed description set forth below with reference to the accompanying drawings is intended to describe various configurations and is not intended to limit the scope of this disclosure. Rather, the detailed description includes specific details for providing a thorough understanding of the subject matter of the invention. It will be apparent to those skilled in the art that these specific details are not necessary in every case, and in some cases, well-known structures and components are shown in block diagram form for clarity.

[0047] This disclosure provides systems, apparatus, methods, and computer-readable media that support domain isolation in an automotive System-on-Chip (SOC). Example embodiments provide isolation (such as bidirectional isolation) between domains in an automotive SOC to facilitate degraded operation of a second domain when an error is encountered in the first domain. For example, an automotive SOC for providing autonomous driving functions (such as assisted or autonomous driving capabilities) can be divided into different domains and a primary domain configured to meet different safety criteria (such as Safety Island Entry (SAIL)). Bidirectional isolation between domains of the automotive SOC allows a domain that has not encountered an error to continue operating even if an uncorrectable error is encountered in another domain. To facilitate such operation, a second domain (such as a primary domain) can bypass a first domain (such as SAIL) to communicate with an external controller (such as a microcontroller or electronic control unit (ECU)), and operational continuity of the second domain is maintained when the first domain is deactivated, because when both domains are active, the second domain can communicate with the external controller only through the first domain.

[0048] Specific implementations of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages or benefits. In some aspects, this disclosure provides techniques for domain isolation that may be particularly beneficial in intelligent vehicle applications. For example, the operational continuity described herein can allow for time recovery of a vehicle-assisted SOC from an error limited to one domain, or, if the error is uncorrectable, allow a vehicle with assisted or autonomous driving capabilities to perform minimum risk maneuvering (MRM) procedures and / or other smooth shutdown procedures. For example, if the SAIL domain of an automated driving SOC (such as an autonomous or assisted driving SOC) fails, continuing operation of the SOC's primary domain in degraded mode without assistance from the SAIL domain can allow the vehicle to safely pull over and shut off.

[0049] Figure 1 This is a perspective view of a motor vehicle with a driver assistance system according to an embodiment of this disclosure. Vehicle 100 may include multiple sensors, such as a forward-facing camera 112 mounted inside the passenger compartment and viewed through the windshield 102. The vehicle may also include a passenger compartment-facing camera 114 mounted inside the passenger compartment and viewed from the passenger side of vehicle 100 (particularly the driver of vehicle 100). Although a set of mounting locations for cameras 112 and 114 is shown for vehicle 100, other mounting locations may be used for cameras 112 and 114. For example, one or more cameras may be mounted on one of the driver or passenger B-pillars 126 or one of the driver or passenger C-pillars 128, such as near the top of pillars 126 or 128. As another example, one or more cameras may be mounted at the front of vehicle 100, such as behind the radiator grille 130 or integrated with the bumper 132. As another example, one or more cameras may be mounted as part of a driver or passenger side mirror assembly 134.

[0050] Camera 112 can be oriented such that its field of view captures the scene in front of vehicle 100 in the direction in which vehicle 100 is moving forward or in driving mode. In some embodiments, an additional camera may be located at the rear of vehicle 100 and oriented such that its field of view captures the scene behind vehicle 100 in the direction in which vehicle 100 is moving in the opposite direction. Although embodiments of this disclosure may be described with reference to a “forward” camera, aspects of this disclosure can be similarly applied to a “rearward” camera facing the opposite direction of vehicle 100, with reference to camera 112. Thus, when an operator drives vehicle 100 in the opposite direction, the same benefits are obtained when the operator drives vehicle 100 in the forward direction.

[0051] Furthermore, although embodiments of this disclosure may be described with reference to a “forward” camera, aspects of this disclosure can be similarly applied to input received from a camera array mounted around vehicle 100, with reference to camera 112, to provide a larger field of view, which may be up to 360 degrees around a direction parallel to the ground and / or up to 360 degrees around a vertical direction perpendicular to the ground. For example, additional cameras may be mounted on or integrated into the exterior of vehicle 100, such as on or integrated into doors, on or integrated into wheels, on or integrated into bumpers, on or integrated into the hood, and / or on or integrated into the roof.

[0052] Camera 114 can be oriented such that the field of view of camera 114 captures the scene inside the vehicle's cabin and includes the vehicle's user operator, and in particular includes the face of the vehicle's user operator with sufficient detail to discern the user operator's gaze direction.

[0053] Each of cameras 112 and 114 may include one, two, or more image sensors, such as a first image sensor. When multiple image sensors are present, the first image sensor may have a larger field of view (FOV) than the second image sensor, or the first image sensor may have a different sensitivity or a different dynamic range than the second image sensor. In one example, the first image sensor may be a wide-angle image sensor, and the second image sensor may be a telephoto image sensor. In another example, the first sensor is configured to acquire an image through a first lens having a first optical axis, and the second sensor is configured to acquire an image through a second lens having a second optical axis different from the first optical axis. Alternatively or additionally, the first lens may have a first magnification, and the second lens may have a second magnification different from the first magnification. This configuration may occur in a camera module with a lens cluster, wherein multiple image sensors and associated lenses are located at offset positions within the camera module. Additional image sensors with larger, smaller, or the same field of view may be included.

[0054] Each image sensor may include units for capturing data representing a scene, such as image sensors (including charge-coupled device (CCD), Bayer filter sensors, infrared (IR) detectors, ultraviolet (UV) detectors, complementary metal-oxide-semiconductor (CMOS) sensors) and / or time-of-flight detectors. The device may also include units for focusing and / or converging light onto one or more of the image sensors (including simple lenses, compound lenses, spherical lenses, and aspherical lenses). These components can be controlled to capture first, second, and / or more image frames. The image frames can be processed to form a single output image frame, such as through a fusion operation, and the output image frame is further processed according to the aspects described herein.

[0055] As used herein, an image sensor can refer to the image sensor itself and any particular other components coupled to the image sensor that generate image frames for processing by other logic circuitry or storage devices in an image signal processor or memory (whether short-term buffers or long-term non-volatile memory). For example, an image sensor may include other components of a camera, including shutters, buffers, or other readout circuitry for accessing individual pixels of the image sensor. An image sensor can also refer to an analog front-end or other circuitry that converts analog signals into a digital representation of image frames, which is provided to digital circuitry coupled to the image sensor.

[0056] Figure 1The camera, along with other sensors (such as radar sensors, lidar sensors, and others), can provide sensing information to the driver assistance system (SOC) of vehicle 100. The SOC can use this data to provide assisted or autonomous driving capabilities. Specifically, the SOC can receive data from one or more sensors of vehicle 100 and can control one or more vehicle functions based on the received data, such as acceleration, braking, steering, louver activation, headlight activation, wiper activation, and other functions. For example, the SOC can provide driver assistance capabilities up to and exceeding Level 4 autonomous driving capabilities.

[0057] Figure 2 A block diagram of an example driver assistance processing configuration for a vehicle, according to one or more aspects of this disclosure, is shown. For example, Figure 2 The driver assistance system may include a processing system, such as a System-on-a-Chip (SOC) for providing autonomous driving as described herein. Vehicle 100 may include or be otherwise coupled to an image signal processor 212 for processing image frames from one or more image sensors, such as depth and image sensors 240. In some implementations, vehicle 100 may also include or be coupled to a processor (e.g., CPU) 204 and a memory 206 storing instructions 208. In some embodiments, processor 204 may include one or more neural signal processors, one or more graphics processing units (GPUs), one or more application processors, one or more computer vision processors, and one or more other processing units. Device 100 may also include or be coupled to a display 214 and an input / output (I / O) component 216. I / O component 216 may be used for user interaction, such as a touchscreen interface and / or physical buttons. I / O component 216 may also include a network interface for communicating with other devices, such as other vehicles, operator mobile devices, and / or remote monitoring systems. The network interface may include one or more of a wide area network (WAN) adapter 252, a local area network (LAN) adapter 253, and / or a personal area network (PAN) adapter 254. Example WAN adapter 252 is a 4G LTE or 5G NR wireless network adapter. Example LAN adapter 253 is an IEEE 802.11 WiFi wireless network adapter. Example PAN adapter 254 is a Bluetooth wireless network adapter. Each of adapters 252, 253, and / or 254 may be coupled to an antenna, including multiple antennas configured for primary and diversity reception and / or configured to receive a specific frequency band. Vehicle 100 may also include or be coupled to a power source 218, such as a battery or alternator. Vehicle 100 may also include or be coupled to... Figure 2Additional features or components not shown. In one example, a wireless interface that may include one or more transceivers and an associated baseband processor may be coupled to or included in a WAN adapter 252 for a wireless communication device. In another example, an analog front end (AFE) for converting analog image frame data into digital image frame data may be coupled between image sensors 201 and 202 and image signal processor 212.

[0058] Vehicle 100 may include a sensor hub 250 for interfacing with sensors to receive data about the movement of vehicle 100, data about the environment surrounding vehicle 100, and / or other non-camera sensor data. One example non-camera sensor is a gyroscope, a device configured to measure rotation, orientation, and / or angular velocity to generate motion data. Another example non-camera sensor is an accelerometer, a device configured to measure acceleration, which can also be used to determine the speed and distance of travel by appropriately integrating the measured acceleration, and one or more of acceleration, speed, and / or distance can be included in the generated motion data. In further examples, the non-camera sensor may be a Global Positioning System (GPS) receiver, a LiDAR system, a RADAR system, or other ranging system. For example, sensor hub 250 may interface with a vehicle bus for sending configuration commands and / or receiving information from vehicle sensors 272, such as a distance (e.g., ranging) sensor or a vehicle-to-vehicle (V2V) sensor (e.g., a sensor for receiving information from nearby vehicles).

[0059] Image signal processor (ISP) 212 can receive image data, such as data used to form image frames. In one embodiment, a local bus connection couples image signal processor 212 to depth and image sensor 240 (which may correspond to...). Figure 1 The first camera 112 and the second camera 205 (which can correspond to the first camera 112) and the second camera 205 (which can correspond to the second camera 205) Figure 1 (Camera 114). In another embodiment, a wired interface can couple the image signal processor 212 to an external image sensor. In yet another embodiment, a wireless interface can couple the image signal processor 212 to a depth and image sensor 240.

[0060] In some implementations, memory 206 may include a non-transitory or non-transitory computer-readable medium storing computer-executable instructions 208 to perform all or part of one or more of the operations described in this disclosure. In some implementations, instructions 208 include a camera application (or other suitable application) to be executed during operation of vehicle 100 to generate images or videos. Instructions 208 may also include other applications or programs that execute for vehicle 100, such as an operating system, autonomous driving application, mapping application, or entertainment application. For example, execution of a camera application by processor 204 may enable vehicle 100 to generate images using depth and image sensors 240 and image signal processor 212. Memory 206 may also be accessed by image signal processor 212 to store processed frames or may be accessed by processor 204 to obtain processed frames. In some embodiments, vehicle 100 includes a system-on-a-chip (SoC) that integrates image signal processor 212, processor 204, sensor hub 250, memory 206, and input / output components 216 into a single package.

[0061] In some embodiments, at least one of the image signal processor 212 or processor 204 executes instructions to perform various operations described herein, including object detection, risk map generation, driver monitoring, autonomous or assisted driving, and driver alert operations. For example, the execution of instructions may instruct the processing system to isolate one or more domains of processing system 280 when a fault or error is detected in another domain, as described herein. In some embodiments, processor 204 may include one or more general-purpose processor cores 204A capable of executing scripts or instructions of one or more software programs, such as instructions 208 stored in memory 206. For example, processor 204 may include one or more application processors configured to execute a camera application (or other suitable application for generating images or videos) stored in memory 206.

[0062] In some embodiments, in addition to the ability to execute software to enable vehicle 100 to perform multiple functions or operations (such as those described herein), processor 204 may also include an IC or other hardware (e.g., an artificial intelligence (AI) engine 224). In some other embodiments, vehicle 100 does not include processor 204, such as when all the described functions are configured in image signal processor 212. In some embodiments, a processing system 280 including processor 204, image signal processor 212, sensor hub 250, input / output component 216, and memory 206 may be integrated in one or more SOCs, such as an autonomous driving SOC. Such an SOC may, for example, include multiple processors 204, one or more image signal processors 212, and other components. The SOC may also include one or more neural signal processing units, one or more graphics processing units (GPUs), one or more application processors, one or more computer vision processors, one or more display processors, one or more peripheral interfaces (such as Ethernet, Universal Serial Bus, or other interfaces), one or more sensors and / or sensor interfaces, one or more voltage sources, one or more clocks, one or more memory controllers such as one or more oscillators, and other components. In some embodiments, the processing system 280 may include one or more buses connecting components such as processor 204, memory 206, and other components of the processing system 280. In some embodiments, the processing system 280 may include multiple processors and may be divided into multiple domains, as described herein. For example, the multiple domains of the processing system may be configured to meet different security standards, as described herein.

[0063] In some embodiments, display 214 may include one or more suitable displays or screens that allow user interaction and / or presentation of items to the user. In some embodiments, display 214 is a touch-sensitive display. I / O component 216 may be or include any suitable mechanism, interface, or device for receiving input (such as commands) from the user via display 214 and providing output to the user. For example, I / O component 216 may include (but is not limited to) a graphical user interface (GUI), keyboard, mouse, microphone, speaker, squeezable bezel, one or more buttons (such as a power button), slider, switch, etc. In some embodiments involving autonomous driving, I / O component 216 may include an interface to a vehicle bus for providing commands and information to and receiving information from vehicle system 270, including propulsion (e.g., commands to increase or decrease speed or apply braking) and steering systems (e.g., commands to steer wheels, change route, or change final destination).

[0064] Although shown as coupled to each other via processor 204, components such as processor 204, memory 206, image signal processor 212, display 214, and I / O components 216 may be coupled to each other in various other arrangements, such as via one or more local buses (not shown for simplicity). Although image signal processor 212 is shown as separate from processor 204, image signal processor 212 may be the core of processor 204, which is an application processor unit (APU) included in or otherwise incorporated into a system-on-a-chip (SoC). While vehicle 100 is referenced in the examples herein to include aspects of this disclosure, some device components may not be shown in the examples. Figure 2 The details are shown to prevent obscuring aspects of this disclosure. Additionally, other components, the number of components, or combinations of components may be included in a suitable vehicle for performing aspects of this disclosure. Therefore, this disclosure is not limited to a particular device or configuration of components, but includes vehicle 100.

[0065] Vehicle 100 can communicate as a user equipment (UE) within wireless network 300, such as via WAN adapter 252. Figure 3 As shown. Figure 3 This is a block diagram illustrating details of an example wireless communication system according to one or more aspects. Wireless network 300 may, for example, include a 5G wireless network. As those skilled in the art will understand, in Figure 3 The components appearing in this network may have corresponding counterparts in other network arrangements, including, for example, cellular network arrangements and non-cellular network arrangements (e.g., device-to-device, peer-to-peer, or self-organizing network arrangements).

[0066] exist Figure 3The wireless network 300 shown includes base station 305 and other network entities. A base station can be a station communicating with a UE and can also be referred to as an evolved Node B (eNB), a next-generation eNB (gNB), an access point, etc. Each base station 305 can provide communication coverage for a specific geographic area. In 3GPP, the term "cell" can refer to that specific geographic coverage area of ​​a base station or a base station subsystem serving that coverage area, depending on the context in which the term is used. In the implementation of wireless network 300 herein, base station 305 can be associated with the same operator or different operators (e.g., wireless network 300 can include multiple operator wireless networks). Additionally, in the implementation of wireless network 300 herein, base station 305 can use one or more frequencies (e.g., one or more bands of licensed spectrum, unlicensed spectrum, or combinations thereof) from the same frequencies as neighboring cells to provide wireless communication. In some examples, a single base station 305 or UE 315 can be operated by more than one network operating entity. In some other examples, each base station 305 and UE 315 can be operated by a single network operating entity.

[0067] Base stations can provide communication coverage for macrocells or small cells (e.g., picocells or femtocells) or other types of cells. Macrocells typically cover a relatively large geographic area (e.g., a radius of several kilometers) and allow unrestricted access by UEs with service subscriptions to a network provider. Small cells (such as picocells) will typically cover a relatively small geographic area and allow unrestricted access by UEs with service subscriptions to a network provider. Small cells (such as femtocells) will also typically cover a relatively small geographic area (e.g., residential areas) and, in addition to unrestricted access, provide restricted access by UEs associated with the femtocell (e.g., UEs in a Closed Subscriber Group (CSG), UEs for users in a residential area, etc.). Base stations used for macrocells can be referred to as macro base stations. Base stations used for small cells can be referred to as small cell base stations, picocells, femtocells, or home base stations. Figure 3 In the examples shown, base stations 305d and 305e are conventional macro base stations, while base stations 305a-305c are macro base stations implemented using one of 3D MIMO, full-dimensional (FD) MIMO, or massive MIMO. Base stations 305a-305c utilize their higher-dimensional MIMO capabilities to employ 3D beamforming in both elevation and azimuth beamforming to increase coverage and capacity. Base station 305f is a small cell base station, which can be a home node or a portable access point. A base station can support one or more cells (e.g., two cells, three cells, four cells, etc.).

[0068] Wireless Network 300 can support synchronous or asynchronous operation. For synchronous operation, base stations can have similar frame timings, and transmissions from different base stations can be approximately time-aligned. For asynchronous operation, base stations can have different frame timings, and transmissions from different base stations can be time-disaligned. In some scenarios, the network can be enabled or configured to handle dynamic switching between synchronous and asynchronous operation.

[0069] UE 315 is distributed throughout the entire wireless network 300, and each UE can be stationary or mobile. It should be understood that although mobile devices are generally referred to as UEs in the standards and specifications published by 3GPP, such devices may be otherwise referred to by those skilled in the art as mobile station (MS), subscriber station, mobile unit, subscriber unit, radio unit, remote unit, mobile device, radio device, wireless communication device, remote device, mobile subscriber station, access terminal (AT), mobile terminal, radio terminal, remote terminal, mobile phone, terminal, user agent, mobile client, client, gaming device, augmented reality device, vehicle component, vehicle equipment, or vehicle module, or some other suitable term.

[0070] Non-limiting examples of mobile devices may include implementations of one or more of those in UE 315, including mobile devices, cellular phones, smartphones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, laptops, personal computers (PCs), notebooks, netbooks, smartbooks, tablet devices, personal digital assistants (PDAs), and vehicles. Although UE 315a-j is specifically shown as a vehicle, the vehicle may employ a communication configuration described with reference to any UE 315a-315k.

[0071] In one aspect, the UE can be a device that includes a Universal Integrated Circuit Card (UICC). In another aspect, the UE115 can be a device that does not include a UICC. In some aspects, a UE that does not include a UICC can also be referred to as an IoE device. Figure 3 The UEs 315a-315d shown in the implementation are examples of mobile smartphone-type devices accessing the wireless network 300. The UE can also be a machine specifically configured for connected communications, including Machine-Type Communication (MTC), Enhanced MTC (eMTC), Narrowband IoT (NB-IoT), etc. Figure 3 The UE 315e-315k shown is an example of various machines configured for communication in the access network 300.

[0072] Mobile devices (such as UE 315) may be able to communicate with any type of base station (whether macro base station, pico base station, femto base station, repeater, etc.). Figure 3 In this context, a communication link (represented by a lightning bolt) indicates a radio transmission between the UE and a serving base station (which is designated to serve the UE on the downlink or uplink), or a desired transmission between base stations, and a backhaul transmission between base stations. In some scenarios, the UE may operate as a base station or other network node. Backhaul communication between base stations of the wireless network 300 can occur using wired or wireless communication links.

[0073] In operation at wireless network 300, base stations 305a-305c use 3D beamforming and cooperative spatial technologies (such as Cooperative Multipoint (CoMP) or Multi-Connection) to serve UEs 315a and 315b. Macro base station 305d performs backhaul communication with base stations 305a-305c and the small cell (base station 305f). Macro base station 305d also transmits multicast services customized and received by UEs 315c and 315d. Such multicast services may include mobile television or streaming video, or may include other services for providing community information, such as weather emergencies or alerts (such as Amber Alerts or Grey Alerts).

[0074] The implemented wireless network 300 supports mission-critical communication for mission-critical devices such as UE 315e (a drone) using highly reliable and redundant links. Redundant communication links with UE 315e include those from macro base stations 305d and 305e, and from small cell base station 305f. Other machine-type devices (such as UE 315f (thermometer), UE 315g (smart meter), and UE 315h (wearable device)) can communicate directly with base stations (such as small cell base station 305f and macro base station 305e) via the wireless network 300, or via another user device relaying its information to the network (e.g., UE 315f transmitting temperature measurement information to the smart meter (UE 315g), which is then reported to the network via small cell base station 305f) in a multi-hop configuration. The wireless network 300 can also provide additional network efficiency through dynamic, low-latency TDD or low-latency FDD communication, such as in vehicle-to-vehicle (V2V) mesh networks between UEs 315i-315k communicating with macro base station 305e.

[0075] refer to Figure 1 , Figure 2 and Figure 3 Describe and in Figure 1 , Figure 2 and Figure 3The vehicle system shown may include the isolation of one or more domains of an autonomous driving system (SOC) to facilitate the continued operation of a domain unaffected by errors encountered in one or more other domains. This isolation allows the domains of the autonomous driving SOC to continue operating to safely bring the vehicle to a stop or resolve errors encountered in the affected domain without losing critical assistance or autonomous driving capabilities, such as steering, acceleration, braking, and other aids.

[0076] Figure 4An example block diagram of an autonomous driving system 400 is shown. The autonomous driving SOC 402 can be divided into different domains, such as a first domain 406 and a second domain 404. In some embodiments, the autonomous driving SOC 402 can be divided into more than two domains. As an example, the first domain 406 can be a Safety Island (SAIL), and the second domain 404 can be a Main Domain (MD). The different domains of the SOC 402 can be designed to meet different safety standards. For example, the first domain 406, which can be a SAIL domain, can be designed to meet a first safety standard, and the second domain 404, which can be an MD, can be designed to meet a second safety standard lower than the first safety standard. The first safety standard for the first domain 406 can be, for example, the Automotive Safety Integrity Level D (ASIL D) standard, while the second safety standard for the second domain 404 can be the Automotive Safety Integrity Level B (ASIL B) standard according to the International Organization for Standardization 26262 standard. In some embodiments, the second domain 404 may include many or all of the major computing components of the SOC 402, such as one or more processors, such as application processors, computer vision processors, image signal processors or neural signal processors, GPUs, supporting infrastructure (such as one or more memory controllers, system memory management units, and bus interfaces), and other components. The first domain 406 may, for example, include one or more monitoring modules, such as voltage, clock, temperature, built-in self-test control, timestamps, secure boot, encryption, and other monitoring modules and / or one or more processors or processing cores. For example, the SAIL domain may include one or more computing resources, such as one or more processors or processing cores, infrastructure blocks, and input / output interfaces for monitoring one or more safety features of the autonomous driving system (such as temperature, voltage, and other parameters). In some embodiments, the SAIL domain may include one or more I / O interfaces for receiving data from one or more external sensors (such as cameras, LiDAR sensors, radar sensors, and other sensors). Such interfaces may, for example, connect the SAIL domain to other components of the autonomous driving system, such as to one or more Ethernet switches, external microcontrollers, or electronic control units (ECUs). For example, the SAIL domain may be connected to an ECU in a separate location in the vehicle via one or more Ethernet or CAN interfaces. In some embodiments, the autonomous driving SOC 402 may include multiple SAIL domains, or additional SAIL domains may be located outside the autonomous driving SOC 402.

[0077] The second domain 404, which may be an MD, may include or may interface with a dynamic random access memory (DRAM) module 408, one or more camera sensors 412 connected to a serializer / deserializer 414, and / or one or more radar sensors 416 and / or one or more lidar sensors 418 connected to a switch 420. In some embodiments, the camera sensor 412, radar sensor 416, and lidar sensor 418 may be otherwise connected to the SOC 402, such as via Ethernet, PCIe, CAN bus, SPI interface, UART interface, or other interfaces. Additionally, additional sensors may be connected to the SOC 402 via one or more PCIe, Ethernet, or other interfaces. The second domain 404 may include a data acquisition module 428, a data processing module 430, and a fault monitor module 432. The first domain 406 (which may be a SAIL domain) may include a data acquisition module 434, a data processing and verification module 438, and an overall fault monitoring and processing module 440. The first domain 406 may receive information from one or more sensors or other SAIL components 436. The first domain 406 may, for example, perform verification of data and / or calculations performed by the second domain 404. Data can be output from the data processing and verification module 438 of the first domain 406 to an external system monitor 424, such as a microcontroller (MCU). The microcontroller may be, for example, a safety microcontroller. Furthermore, the overall fault monitoring and processing module 440 of the first domain 406 can receive indications of faults from the second domain 404 and can determine a response to such faults. In some cases, the overall fault monitoring and processing module 440 may send notifications of one or more detected faults or errors to the external system monitor 424. Such detected faults or errors may be detected by either the second domain 404 or the first domain 406. In some embodiments, the external system monitor 424 may perform independent monitoring of the SOC 402. The external system monitor 424 may communicate with the vehicle motion control unit 426 to control steering, braking and acceleration actuation or control, or other vehicle functions. In some embodiments, instructions for autonomous or assisted driving may be determined by the second domain 404 and verified by the first domain 406 of the SOC 402, transmitted to the external system monitor 424, and transmitted from the external system monitor 424 to the vehicle control unit 426 to control the vehicle.

[0078] One or more domains of SOC 402 may encounter errors. Some errors may require one or more domains of SOC 402 to be deactivated and / or powered off. Errors can be triggered by a variety of reasons, such as ionizing radiation and associated faults.

[0079] First domain 406 can be isolated from second domain 404 via isolation 442, such that errors encountered by second domain 404 may not require shutting down second domain 406. For example, isolation 442 may include isolation logic such that first domain 406 is isolated from second domain 404 when second domain 404 encounters an error. Isolation 442 can be further provided by using a first independent power supply and clock 422 for the first domain and a second independent power supply and clock 410 for the second domain. Therefore, when second domain 404 encounters an error (such as error 454 in the independent power supply and clock 410 for second domain 404 or error 452 in second domain 404), first domain 406 can be isolated from second domain 404. First domain 406 can then continue operating in degraded mode without assistance from the functionality provided by second domain 404, while second domain 404 is, for example, deactivated and / or powered down. Thus, for example, even when the MD of the autonomous driving SOC is deactivated, the SAIL domain of the autonomous driving SOC can continue to operate in degraded mode.

[0080] If isolation is implemented only in the first direction, such as isolating the first domain 406 when an error occurs in the second domain 404, an error occurring in the first domain 406 may require deactivation of the entire SOC. For example, when the isolation logic is configured to isolate only the SAIL domain from errors that cause MD, without isolating MD from errors occurring in the SAIL domain, an error occurring in the SAIL domain may require deactivation of MD in addition to the SAIL domain. The lack of bidirectional isolation between domains can lead to additional problems in systems without external microcontrollers (MCUs) (such as safety microcontrollers), as errors in the SAIL domain may cause advanced driver assistance or autonomous driving electronic control units to cease operation. Downtime due to SAIL domain errors can be particularly problematic in Level 3 or higher autonomous driving systems, as downtime can affect the ability of such systems to maintain autonomous driving functionality, even in degraded modes, until the driver assumes control of the vehicle. For example, Level 3 or higher autonomous systems may require continuous operation of automated driving systems such as autonomous driving systems for varying durations (e.g., up to and exceeding tens of seconds, depending on the vehicle's operating parameters) to allow the vehicle to warn the driver to take control and / or safely decelerate and pull over to a curb position. As described in this article, the System-on-Chip (SOC) used in autonomous driving systems can also be used to provide Level 1 or Level 2 autonomous driving. The levels of autonomous driving mentioned in this article can correspond to the levels identified by the SAE J3016 standard.

[0081] Therefore, as Figure 4As shown, isolation 442 can also be implemented between errors occurring in the first domain 406 and the second domain 404, such as isolation between MD and errors occurring in the SAIL domain. Such isolation 442 can allow the second domain 404 to continue functioning, even when a critical error requiring deactivation or power-off of the SAIL domain occurs. Errors that may occur in the first domain 406 can include one or more errors 450 in the first domain 406, one or more errors 448 in the independent power and clock module 422 of the first domain 406 (such as errors in one or more power management integrated circuits (PMICs) of the first domain 406), one or more errors 446 in one or more sensors or other SAIL components 436 connected to the first domain 406 (such as errors in serial peripheral interface NOR flash modules or one or more sensor modules), or other errors in the first domain 406. Figure 4 As shown, such errors can be prevented by isolation 442 without requiring the deactivation and / or power-off of the second domain 404. Therefore, isolations 442 and 444 can provide isolation between domains of the autonomous driving SOC 402 or other SOCs to facilitate continued operation of each corresponding domain in the event that another corresponding domain encounters an error requiring shutdown.

[0082] Isolations 442 and 444 can be, for example, isolation between different domains of a System of Origin (SOC) configured to different levels of safety standards, such as ASIL B domains, ASIL D domains, domains configured to Quality Management (QM) levels, and other domains configured according to different safety levels according to the ISO 26262 standard or other standards. As a specific example, isolations 442 and 444 can provide isolation between domains configured according to QM levels or not configured according to safety standards or levels, and domains configured according to one or more safety levels identified by the ISO 26262 standard (such as ASIL A, ASIL B, ASIL D, or other domains). Isolations 442 and 444 can be facilitated by independent power and clock modules 422 and 410, respectively assigned to the first domain 406 and the second domain 404. Isolations 442 and 444 can also include implementations of isolation logic to prevent cascading failures from the first domain 406 or the second domain 406 from requiring the shutdown of another corresponding domain. Isolations 442 and 44 can, for example, include electrical isolation between the first domain 406 and the second domain 404.

[0083] Figure 5Example flowchart 500 is shown, illustrating the corresponding states of the SAIL field (which may be the first field as described herein) and MD (which may be the second field as described herein). In block 502, the SOC can be powered down. The power-down state can be considered a safe state, such as a safe state according to ISO 26262. At block 504, the SOC can be powered up and a determination can be made regarding the SM_ERROR_0 / 1 error report status. Such a report can, for example, be sent to an external safety monitoring device 506, such as an electronic control unit (ECU) or microcontroller (MCU).

[0084] The error reporting status in box 504 can indicate that an error exists at box 508. Such error indications can place one or more domains of the SOC into a safe state, such as a safe state according to ISO 26262.

[0085] Errors can be, for example, errors in the SOC's SAIL domain. For instance, at box 510, the SAIL Built-in Self-Test (BIST) may have failed and / or is still incomplete. At box 512, booting of the SAIL domain may have failed and / or is still incomplete. At box 514, the SAIL security subsystem may report an error. In some embodiments, the error state indicated at box 508 may be caused by one or more of the errors described in boxes 510-514. As a result of an error in the SAIL domain, at box 516, the SOC's MD may enter a degraded mode, such as a mode in ASIL B operation. For example, isolation between the SAIL domain and the MD may be activated, and the MD may continue to operate in degraded mode without communicating with the SAIL domain.

[0086] As another example, the error could be an error in the MD of the SOC. For example, at box 518, the MD's built-in self-test (BIST) may have failed and / or is still incomplete. At box 520, the MD's boot process may have failed and / or is still incomplete. At box 522, the MD security subsystem may report an error. In some embodiments, the error state indicated at box 508 may be caused by one or more of the errors described in boxes 518-522. As a result of an error in the MD, at box 524, the SOC's SAIL domain may enter a degraded mode, such as a mode in ASIL D operation. For example, isolation between the MD and the SAIL domain may be activated, and the SAIL domain may continue to operate in degraded mode without communicating with the MD.

[0087] Error reporting status box 504 can indicate that no error occurred at box 526. Therefore, at box 528, the SAIL domain can enter task mode, and at box 530, the MD can enter task mode. The SAIL domain and the MD can thus continue to communicate with each other, such as via a shared mailbox when no error is detected.

[0088] exist Figure 6 Example state flowchart 600 of a SOC with a SAIL domain and MD is shown. When the SOC is powered on, the SOC can progress from a power-down state 640 to a hypothetical SAIL domain error state 602. In SAIL domain error state 602, the SAIL domain can be in SAIL error state 608, and the MD can be in MD error state 610. The SAIL domain can be placed in SAIL error state 608 due to a SAIL-to-boot failure, unresponsiveness of the SAIL domain, or detection of an error (such as an uncorrectable error) in the SAIL domain. Alternatively or additionally, in SAIL domain error state 602, the SAIL domain can be in SAIL boot state 604 (where the SAIL domain can be in the boot process and can respond to CAN or ETH discovery messages) or in SAIL BIST state 606 (where the SAIL domain may be undergoing a BIST sequence).

[0089] If the SAIL domain is not determined to be faulty, a SAIL_OK message can be generated and sent to a controller, such as an external system monitor, ECU, or MCU, and the SOC can proceed to the MD hypothetical fault state 612. In MD fault state 612, the MD can be in MD fault state 618, and the SAIL domain can be in SAIL degraded mode state 616. SAIL degraded mode 616 can also be referred to as island mode. In SAIL degraded mode state 616, the SAIL domain can operate under limited security policy definition conditions, such as conditions defined by the manufacturer. For example, in SAIL degraded mode state 616, SAIL can perform minimal-risk operations, such as operations selected by the manufacturer's security policy. In some embodiments, if entering SAIL degraded mode state 616 due to a fault in the MD, the SAIL domain can attempt to restart the MD to recover the MD from the fault back to MD task state 626. Figure 6As shown, when the SOC's primary domain is still in the boot or BIST state, it can enter the SAIL degraded mode state 616. In SAIL degraded mode state 616, the SAIL domain can be limited to using computing and resource capabilities within the SAIL domain, as the MD may be in an error, boot, or BIST state. In the MD hypothetical error state 612, the MD can be in the MD boot state 614 (where the MD can be in the boot process and can respond to CAN or ETH discovery messages) or the MD BIST state 620 (where the MD can undergo one or more BIST sequences). In the MD hypothetical error state 612, the MD can be deactivated or in the process of initialization, and the SAIL domain can be isolated from the MD. Therefore, the SAIL domain can operate in degraded mode without input from the MD. If a SAIL error is detected while the SAIL domain is operating in degraded mode, a SAIL-NOK notification can be issued, and the SOC can continue operating in the SAIL error state 602. If the SAIL error is uncorrectable, the SOC can continue to issue SOC_PowerOff notifications to the controller and can return to the power-off state 640.

[0090] If the MD is not determined to be faulty, a Main Domain_OK notification can be sent to the controller, and the SOC can advance to SOC task state 622. In the SOC task state, the SAIL domain can operate in SAIL task state 624, and the MD can operate in the main domain task state 626. In SOC task state 622, the SAIL domain and MD can communicate with each other via mailboxes. In SOC task state 622, booting of the SAIL domain and MD can be completed, security mechanisms can be activated, fault monitoring of the MD and SAIL domains can be active, and the MD and SAIL domains can execute defined security functions. If the MD encounters an error while the SOC is in SOC task state 622, a Main Domain_NOK notification can be sent to the controller, and the SOC can advance to MD error state 612, where the SAIL domain can be placed in SAIL degraded mode state 616.

[0091] In some configurations, if the SAIL domain fails during SOC mission mode, the SAIL domain can issue a SAIL_Failsafe notification, and the SOC can advance to SOC failsafe mode 634. In SOC failsafe mode, the SAIL domain can be placed in SAIL failsafe state 636, and the MD can be placed in MD error state 638. In SAIL failsafe state 636, SAIL can operate with limited capabilities (such as capabilities set by the manufacturer's safety policy) to place the vehicle in a safe state to perform minimum-risk maneuvers. Once the minimum-risk maneuvers are completed, such as once the vehicle reaches a stop at a safe roadside location, the SOC can issue a SOC_PowerOff notification and advance to power-off state 640. In SAIL failsafe state 636, for example, the SAIL domain can perform minimum-risk maneuvers and shut down the SOC, while in SAIL degraded mode state 616, SAIL can attempt to recover and / or restart the MD.

[0092] As another example, if no isolation is provided between errors in the SAIL domain and the MD, when the SAIL domain encounters an error in SOC task mode 622, SAIL_NOK can be issued, and the SOC can proceed to SAIL error state 602. However, if isolation is provided between errors in the SAIL domain and the MD, when the SAIL domain encounters an error in SOC task mode 622, SAIL_NOK can be issued, and the SOC can proceed to alternative SAIL error state 628. In alternative SAIL error state 628, the SAIL domain can be in SAIL error state 630, and the MD can enter MD degraded mode state 632. In MD degraded mode state 632, the MD can operate in degraded mode. If the SAIL domain error is resolved in alternative SAIL error state 628, such as by restarting the SAIL domain and performing SAIL BIST on the SAIL domain during the restart process, SAIL_OK can be issued, and the SOC can continue to return to SOC task state 622. Specifically, the SAIL domain (which resolves errors by restarting the SAIL domain) can be reinitialized to SAIL task mode 624. Therefore, isolation between the MD and errors in the SAIL domain allows for the resolution of errors in the SAIL domain and the restoration of the SOC task state 622, rather than requiring the eventual shutdown of the SOC. If the SAIL domain error is not resolved, the MD can continue operating in degraded mode until a safe state for the vehicle is achieved, such as when the driver assumes control of the vehicle or the vehicle comes to a safe stop at the roadside. Then, a SOC_PowerOff can be issued, and the SOC can proceed to the power-off state 640. Therefore, when isolation is provided between errors in the MD and SAIL domains, and between errors in the SAIL and MD domains, when the MD or SAIL domain encounters an error, the other of the MD or SAIL domains can operate in degraded mode, rather than requiring the SOC to shut down when an error is encountered in the SAIL domain.

[0093] exist Figure 7 An example layout of an autonomous driving system 700 is shown. The autonomous driving system may include an MD 702 and a SAIL domain 704, which together may form part or all of the autonomous driving SOC of the autonomous driving system 700. The autonomous driving system 700 may also include one or more MD PMICs 710 and an external controller 708, which may be a microcontroller (MCU), electronic control unit (ECU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other controller. The MD 702 and SAIL domain 704 can be configured according to different safety standards. For example, the SAIL domain 704 can be configured according to a higher safety standard than the MD 702, as described herein.

[0094] MD 702 may include one or more processors, such as application processor 712. Application processor 712 may execute one or more security-critical applications 714, MD security monitoring processes 716, and one or more drivers, such as an image signal processing (ISP) security driver 718, a neural signal processing (NSP) security driver 720, and one or more other security drivers 722. The drivers of application processor 712 may communicate with one or more other hardware modules of MD 702. For example, ISP security driver 718 may communicate with ISP hardware 724 (such as one or more image signal processors), NSP security driver 720 may communicate with NSP hardware 726 (such as one or more neural signal processors), and other security drivers 722 may communicate with other hardware components 728 (such as sensors, GPUs, other processing units, and other hardware components). Communication between security drives 718, 720, 722 and hardware components 724, 726, 728 may include communication of errors, warnings, or other interrupts occurring in or related to hardware components 724, 726, 728. Error and warning interrupts may also be sent from hardware components 724, 726, 728 to the error aggregation module 742 of SAIL domain 704.

[0095] SAIL domain 704 may include SAIL processor 736, which can execute SAIL safety monitoring process 734. SAIL domain 704 may also include other hardware components 740, such as Ethernet interfaces, CAN interfaces, and other hardware components, such as other interfaces, sensors, or hardware components. Error aggregation module 742 may be a hardware error aggregation module and may transmit interrupts to SAIL safety monitoring process 734 via one or more interfaces. SAIL safety monitoring process 734 may communicate with MD safety monitoring process 716 via mailbox 730. SAIL safety monitoring process 734 may also communicate with the security application of external controller 708 via a serial bus interface, such as a Universal Asynchronous Receiver-Transmitter (UART) interface, Ethernet interface, CAN bus, or Serial Peripheral Interface (SPI). In some embodiments, error aggregation module 742 may communicate with external controller 708 via one or more error pins.

[0096] The MD 702 can be isolated from errors occurring in the SAIL domain 704 via isolation 706, which may include isolation logic and / or electrical isolation between the two domains. To facilitate communication between the MD 702 and an external controller 708 when an error is encountered in the SAIL domain 704 and isolation 706 is activated, the MD 702 can be connected to the external controller 708 via an interface 732 (such as a CAN interface or other interfaces).

[0097] As an example of an error in MD 702, the error can be detected in ISP hardware 724 (such as in ISP on-chip memory). Alternatively or additionally, the error can be detected in application processor 712 or another component of MD 702. Error aggregation module 742 can receive functional safety error signals, which notify the error to error aggregation module 742. Similarly, ISP safety driver 718 can receive an interrupt to warn the error to safety monitoring process 716 of application processor 712. Error aggregation module 742 can notify the error to external controller 708 and SAIL safety monitoring process 734. Safety monitoring process 716 of application processor 712 can determine the cause of the error and can provide details about the error, such as the cause of the error, to SAIL safety monitoring process 734 via mailbox 730. When error aggregation module 742 receives an error signal, error aggregation module 742 can start a timer set to a first time period. If no error information is received via mailbox 730 before the timer expires, the error indication may time out. However, if the SAIL safety monitoring process 734 receives information about an error via mailbox 730 before the timer expires, the SAIL safety monitoring 734 can send the error and / or health information to the external controller 708 via serial bus 744. Therefore, when MD 702 and SAIL domain 704 are operating in task mode, error information about MD 702 can be sent from MD 702 to SAIL domain 704, and from SAIL domain 704 to the external controller 708. For example, when both MD 702 and SAIL domain 704 are operating, MD 702 can communicate with the external controller 708 via SAIL domain 704 instead of via bus 732.

[0098] If it is determined that an error detected in MD 702 requires isolation of SAIL domain 704 from MD 702, for example, if the error is determined to be uncorrectable, SAIL domain 704 and / or MD 702 can activate isolation 706, such as by activating isolation logic. In some embodiments, the activation of isolation logic can be initiated before or during the transmission of error interruption and / or error information from MD 702 to SAIL domain 704. After isolating SAIL domain 704 from MD 702, SAIL domain 704 can operate in degraded mode without accessing the computing power and / or other resources of MD 702. In SAIL degraded mode, SAIL security monitoring process 734 can continue to communicate with security applications of external controller 708 via serial interface 744, such as to notify the external controller of any errors detected in SAIL domain 704 or to maintain continuous heartbeat checks between SAIL domain 704 and external controller 708. When the vehicle in which the autonomous driving system 700 is located has reached a safe state, such as after performing manufacturer-defined minimum risk maneuvers, the SAIL domain 704 (such as the SAIL domain processor 736) can send instructions to one or more SAIL PMICs to power down the SOC, including MD 702 and the SAIL domain 704, such as via a SAIL_PS_HOLD shutdown request. Therefore, when MD 702 encounters an error (such as an uncorrectable error), the SAIL domain 704 can isolate itself from MD 702 and continue operating in degraded mode until the SOC can be safely powered down.

[0099] Isolation 706 can also isolate MD 702 from errors detected in SAIL domain 704. In some embodiments, isolation 706 may include logical and electrical isolation to provide bidirectional isolation between MD 702 and SAIL domain 704. Isolation 706 can be activated to isolate MD 702 from SAIL domain 704 when an error 738 (such as an error in processor 736 or another component of SAIL domain 704) is detected in SAIL domain 704. For example, MD 702 may receive notification of errors in the SAIL domain, such as a SAIL-ERROR signal, via mailbox 730, via an internal SOC signal between SAIL and MD, or via another interface between SAIL processor 734 and application processor 712 of MD 702, and isolation 706, such as isolation logic between MD 702 and SAIL domain 704, can be activated. This isolation 706 can also allow MD 702 to continue operating in degraded mode even if SAIL domain 704 is in an error state. For example, when an error such as error 738 is detected in SAIL domain 704, isolation 706, such as isolation logic, can be activated between MD 702 and SAIL domain 704. In some embodiments, application processor 712 or another MD subsystem can activate isolation 706 when it receives notification of an error occurring in SAIL domain 704 via mailbox 730 through security monitoring process 716 or other signaling between SAIL domain 704 and MD. When isolation 706 is activated, security monitoring process 716 may no longer rely on or communicate with SAIL security monitoring process 734 executed by SAIL processor 736. For example, when isolation 706 is activated, security monitoring process 716 of MD 702 and SAIL security monitoring process 734 of SAIL domain 704 may cease communication via mailbox 730.

[0100] To maintain communication between the SOC, including MD 702 and SAIL domain 704, and the external controller 708, MD 702 can begin communicating with the external controller 708 via bus 732 when isolation 706 is activated. Bus 732 can be a CAN bus, Ethernet interface, serial interface (such as SPI or UART interface), or other interface. For example, when SAIL domain 704 and MD 702 are operating in task mode, MD 702 (such as MD 702's security monitoring process 716) can communicate with the external controller 708 via SAIL domain 704's SAIL security monitoring process 734 without using bus 732. Therefore, once isolation 706 is activated in response to an error in SAIL domain 704, MD 702 can operate in degraded mode and can communicate with the external controller 708 via bus 732. In some embodiments, application processor 712's security monitoring process 716 can communicate with the external controller 708 via bus 732. Information transmitted via bus 732 may include MD health information, such as information about one or more errors encountered by MD 702, and information for maintaining continuous heartbeat checks between MD 702 and external controller 708. When the vehicle, including the autonomous driving system 700, has reached a safe state (such as a safe state determined by the vehicle manufacturer), MD application processor 712 may send a power-down request to one or more MD PMICs 710, such as an MD_PS_HOLD power-down request to power down the SOC including MD 702 and SAIL domain 704. Alternatively or additionally, MD 702 may restart SAIL domain 704, such as by restarting the processor 736 of SAIL domain to attempt to correct errors detected in SAIL domain 704. If the errors are corrected, isolation 706 can be deactivated, and MD 702 and SAIL domain 704 can continue to operate in task mode. Therefore, the isolation 706 between SAIL domain 704 and MD 702 allows MD 702 to continue operating in degraded mode when SAIL domain 704 encounters an error that triggers an error state for SAIL domain 704 (such as a state that requires deactivation of SAIL domain 704). SAIL domain 704 can be configured to avoid shutting down MD 702 when SAIL domain 704 enters an error state.

[0101] When MD 702 operates in degraded mode, such as when isolation 706 is activated and SAIL domain 704 is in an error state, various continuing functions can be performed by MD 702 in degraded mode, such as allowing the autonomous driving system 700 to continue providing assisted or autonomous driving functions until the functionality of SAIL domain 704 is restored or until the vehicle is brought to a safe state and the SOC including SAIL domain 704 and MD 702 can be safely deactivated. As an example, MD 702 can continue to perform functional safety (FUSA) error monitoring of MD 702. Such monitoring may include monitoring subsystems of MD 702, such as monitoring one or more GPUs of MD 702, one or more NSPs 726 of MD 702, one or more ISPs 724 of MD 702, or one or more other components 728 of MD 702. Indications of errors, warnings, or other interruptions of such components can continue to be sent to application processor 712, as described herein, and detected by drivers 718, 720, 722 of application processor 712. In addition, the safety monitoring process 716 can continue to process such errors and / or warnings, and send indications of such errors, warnings or other interruptions to the ECU 708 via the bus 732.

[0102] In some embodiments, in addition to SAIL domain 704, the autonomous driving system 700 may also include other SAIL domains. In some embodiments, the additional SAIL domains may reside on the same SoC, such as on the same die as MD 702 and SAIL domain 704. In some embodiments, the additional SAIL domains may reside on other SoCs, such as on other dies within the same package. In some embodiments, SAIL domain 704 may be the SAIL domain of the primary die, while the other SAIL domain may be the SAIL domain of the secondary die. If the SAIL domain 704 of the primary die fails, the SAIL domain of the secondary die can communicate with MD 702 and allow MD 702 to continue operating in mission mode. If the SAIL domain of the secondary die fails, the SAIL domain 704 of the primary die can continue operating in mission mode together with MD 702. If all alternative SAIL domains fail, MD can be isolated from all SAIL domains and can operate in degraded mode.

[0103] When the MD 702 operates in degraded mode, one or more voltage rails of the MD 702 can also be monitored, for example, by one or more PMICs 710 of the MD 702. For example, one or more PMICs 710 of the MD 702 can provide voltage monitoring of one or more MD rails. In some embodiments, one or more PMICs 710 of the MD 702 can communicate directly with an external controller 708, for example, by sending one or more error signals when one or more errors are detected relative to one or more MD rails. Such signals can be sent, for example, via interface 760. In some embodiments, a timing protection band monitor can be provided when the MD 702 is in degraded mode, which can also provide undervoltage fault detection. Furthermore, adaptive clock distribution circuitry within one or more safety subsystems of the MD 702 can monitor voltage drop events when the MD is in degraded mode. However, when the MD 702 is isolated from the SAIL domain 704 and operates in degraded mode, on-die voltage monitoring of the voltage rails of the SOC may not be provided.

[0104] When the MD 702 operates in degraded mode, one or more clocks associated with the MD 702 can also be monitored, such as via the MD PMIC 710. For example, the MD PMIC 710 can perform clock monitoring of one or more crystal oscillators for the MD clock input. In some embodiments, one or more PMICs 710 of the MD 702 can communicate directly with an external controller 708, for example, by sending one or more error signals when one or more errors are detected relative to one or more MD clocks. In some embodiments, on-die clock monitoring may not be performed when the MD 702 is in degraded mode. In some embodiments, one or more clock monitoring units (such as advanced clock monitoring units) may be included in the MD 702 of the SOC to perform on-die clock monitoring when the MD 702 is in degraded mode.

[0105] The MD application processor 712 can also monitor one or more temperatures associated with the MD 702. For example, one or more temperature sensing controllers in the MD 702 can monitor one or more temperatures and can report sensed temperatures outside one or more temperature ranges to the security monitoring process 716 of the application processor 712.

[0106] When the MD 702 is in degraded mode, the MD application processor 712 can also perform hardware and software diagnostics within the MD 702. Such diagnostics may include hardware and software diagnostics for the application processor 712, hardware and software diagnostics for the neural signal processor 726, GPU diagnostics, memory diagnostics, network diagnostics, and other diagnostics. For example, when the MD 702 is in degraded mode, error correction codes, parity checks, watchdog timers, and other diagnostics can continue to be performed within the MD 702. As another example, when the MD 702 is in degraded mode, one or more software test libraries for the application processor 712, NSP hardware 726, ISP hardware 724, or other hardware 728 can continue to be executed. Such software test libraries may include, for example, one or more software tests running in the background to perform one or more tests on security subsystems, such as the GPU, NSP, and other subsystems within the MD. Such tests can compare the outputs from the performed tests with threshold values ​​(e.g., the gold standard) to determine whether the subsystem is operating correctly. In some embodiments, when MD 702 enters degraded mode, the MD application processor 712 may perform additional health checks and diagnostics that are not performed when MD is in task mode. In some embodiments, when MD 702 is in degraded mode, MD 702 may provide additional periodic health check status information to ECU 708 via bus 732.

[0107] In some embodiments, isolation 706 may allow updates to the firmware or software of MD 702 or SAIL domain 704. For example, an autonomous driving system 700 may receive updates, such as over-the-air updates to SAIL domain 704 or MD 702, and may activate isolation 706. Such an update may, for example, be an update that reprograms the software image of the SAIL external memory. If the update is for SAIL domain 704, isolation 706 may be activated to isolate MD 702 from the SAIL domain, and MD 702 may continue to operate in degraded mode while SAIL domain 704 is updated. Similarly, if the update is for MD 702, isolation 706 may be activated to isolate SAIL domain 704 from MD 702, and SAIL domain 704 may continue to operate in degraded mode while MD 702 is updated. When the updated application is complete, isolation 706 may be deactivated, and MD 702 and SAIL domain 704 may resume operation in task mode.

[0108] exist Figure 8 The diagram illustrates a method for domain isolation in a vehicle SOC according to an embodiment described above. Figure 8This is a flowchart illustrating an example method 800 for isolating a second domain of a System-on-a-Chip (SOC) from the first domain when an error is detected in the first domain of the SOC. Method 800 includes, at block 802, detecting an error in the first domain of the SOC of an autonomous driving system. For example, the first domain may be a SAIL domain of the SOC, such as an autonomous driving SOC. The SOC may also include a second domain, such as the MD of the SOC. In some embodiments, the first domain may be configured according to a first safety standard, and the second domain may be configured according to a second safety standard. The first safety standard may be higher than the second safety standard. For example, the first domain may be configured to meet the ASIL D safety standard, while the second domain may be configured to meet the ASIL B safety standard. In some embodiments, detecting an error in the first domain may include determining that the error has a specific type. For example, detecting an error may include determining that the error is uncorrectable. In some embodiments, method 800 may be performed by one or more processors of the second domain of the SOC (such as one or more processors of the MD of the SOC). Detection of an error in the first domain may, for example, include receiving notification of an error detected in the first domain by a processor of the second domain, such as via a mailbox or other signaling between the processor of the second domain and the processor of the first domain.

[0109] At block 804, a second domain of the SOC (such as the SOC's MD) may be isolated from the SOC's first domain. Isolating the second domain of the SOC from the first domain may include activating isolation logic between the second domain and the first domain of the SOC. For example, activating the isolation logic may include stopping communication between the first domain and the second domain of the SOC. Furthermore, isolating the second domain of the SOC from the first domain may include activating electrical isolation between the second domain and the first domain of the SOC. Isolation between the second domain and the first domain may be performed, for example, by the processor of the second domain in response to the detection of an error in the first domain. In some embodiments, the first domain may be deactivated or disabled after the second domain is isolated from the first domain.

[0110] At block 806, operation of the second domain of the SOC can be maintained after isolating the second domain from the first domain. For example, after isolating the second domain from the first domain, the second domain of the SOC can continue to operate in a degraded mode. In degraded mode, the second domain can continue to perform one or more functions of the second domain (such as one or more MD functions) without information input from the first domain. Maintaining the operation of the second domain may include: monitoring the voltage of one or more voltage rails associated with the second domain by the second domain (such as by the processor of the second domain); monitoring the clock associated with the second domain by the second domain; monitoring the temperature associated with the second domain by the second domain; performing hardware diagnostics on the second domain by the second domain; or performing software diagnostics on the second domain by the second domain. In some embodiments, maintaining the operation of the second domain may include: generating instructions for performing minimal-risk maneuvers of the vehicle by one or more processors or other components of the second domain. For example, in degraded mode, the second domain may be configured to safely perform one or more assisted or autonomous driving functions. In some embodiments, such functions may be performed to safely park the vehicle on the roadside. In some embodiments, after a minimum-risk maneuver has been performed, such as after the vehicle has been safely pulled over to the side of the road or after the driver has assumed control of the vehicle, the second domain (such as the processor of the second domain) may send a shutdown request to one or more power regulators (such as one or more PMICs of the second domain) to deactivate the SOC comprising the first and second domains.

[0111] At block 808, the second domain can bypass the first domain to send one or more notifications to an external controller via a first communication interface between the second domain and an external controller after isolating the second domain of the SOC from the first domain. In some embodiments, the external controller may be a microcontroller of an autonomous driving system, such as a safety microcontroller. The second domain may communicate with the external controller via the first domain before an error is detected in the first domain, such as via one or more safety processes executed by the processor of the second domain. Therefore, the second domain may not communicate directly with the external controller before an error is detected in the first domain. However, after isolation is activated at block 804, the second domain may not be able to communicate with the external controller via the first domain because the second domain may be isolated from the first domain. Therefore, the second domain can bypass the first domain to communicate with the external controller via the first communication interface. The first communication interface may include, for example, a CAN interface, a serial interface, or another interface. One or more notifications may include indications of the health of the second domain, indications of one or more errors detected in the second domain, or one or more heartbeat check notifications. Therefore, in order to facilitate the continued operation of the second domain when it is isolated from the first domain, the second domain can bypass the first domain after it is isolated from the first domain and communicate with the external controller via the first communication interface.

[0112] exist Figure 9The diagram illustrates one method for domain isolation in an automotive SOC according to the embodiments described above. The operation of method 900 can be performed in conjunction with one or more blocks of method 800 or method 1000. Figure 9 This is a flowchart illustrating an example method for isolating a second domain of a System-on-a-Chip (SOC) from the first domain of the SOC when an error is detected in the first domain. Method 900 includes, at block 902, detecting an error in the SAIL domain of an autonomous driving SOC. As described herein, an autonomous driving SOC may include an MD and a SAIL domain. Method 900 may be performed, for example, by an application processor of the SOC's MD and / or one or more other subsystems of the MD (such as a control processor). Detecting an error in the SAIL domain may, for example, include receiving an indication of the error via a mailbox or other signaling between the MD and the SAIL domain.

[0113] At position 904, primary domain isolation logic can be activated. For example, in response to detecting an error in the SAIL domain, the MD application processor can activate isolation between the MD and the SAIL domain. Activation of the MD isolation logic can include activating electrical isolation between the SAIL domain and the MD.

[0114] At point 906, one or more safety tasks can be performed independently of the SAIL domain. For example, after activating the MD isolation logic, the MD can continue to operate in degraded mode without input from the SAIL domain. The MD can continue to perform one or more autonomous or assisted driving functions, such as performing minimum-risk maneuvers. The MD can, for example, communicate with an external controller via a communication interface, bypassing the SAIL domain, as per [reference to...]. Figure 8 As described. Furthermore, as stated in this article, MD can continue to perform one or more of its monitoring functions.

[0115] At point 908, the SAIL domain can be recovered from an error state, or a shutdown can be performed. For example, the MD can attempt to recover the SAIL domain from a detected error, such as by restarting the SAIL domain. If the MD cannot recover the SAIL domain from the error, it can perform a shutdown once the vehicle is in a safe state, as described herein, such as after performing a minimum-risk maneuver. If the MD is able to recover the SAIL domain from the error, it can detect that the error has been resolved.

[0116] At 910, if the SAIL domain recovers from an error state, the MD can deactivate the primary domain isolation logic. For example, the MD can deactivate the isolation logic between the MD and the SAIL domain based on the resolution of the error. In some embodiments, the MD can begin communicating with the SAIL domain via a mailbox. Furthermore, after isolation is deactivated, the MD can cease operation in degraded mode and can continue operation in task mode along with the SAIL domain, as described herein. After the MD isolation logic is deactivated, the MD can cease direct communication with the external controller via the communication interface and can communicate with the external controller via the SAIL domain. Therefore, when the SAIL domain encounters an error requiring isolation from the MD to continue MD functionality, the isolation between the MD and the SAIL domain allows the MD to attempt to recover the SAIL domain, such as by restarting the SAIL domain.

[0117] As discussed in this article, isolation between domains can also facilitate the continued operation of another domain of the SOC while one domain of the SOC is being updated. Figure 10 The diagram illustrates a method 1000 for domain isolation in an automotive SOC according to an embodiment described above. The operation of method 1000 can be performed in conjunction with one or more blocks of method 800 or method 900. Figure 10 This is a flowchart illustrating an example method for isolating a second domain of a System-on-a-Chip (SOC) from the first domain when updating the first domain of the SOC. Method 1000 includes, at block 1002, receiving a notification of an update to the first domain. For example, an autonomous driving SOC may receive a notification of an update to the first domain, such as an over-the-air update notification. The first domain may, for example, be the SOC's SAIL domain. The SOC may also include a second domain, which may be the SOC's MD. In some embodiments, method 1000 may be executed by a processor of the second domain of the SOC.

[0118] At box 1004, a second domain (such as MD) can be isolated from the first domain based on a received update notification. For example, isolation logic between the second and first domains can be activated. This isolation can be performed in response to receiving a notification of an update to the first domain. For example, when an update is applied to the first domain, the second domain can be isolated from the first domain to allow the second domain to operate in a degraded mode when the update is applied to the first domain.

[0119] At box 1006, an update can be applied to the first domain. For example, the software or firmware of the first domain (which could be a SAIL domain) can be updated while the second domain is isolated from the first domain. Such an update could, for example, include rebooting the SAIL domain. When the update is applied, the second domain can continue to operate. For example, the second domain can continue to operate in degraded mode.

[0120] At box 1008, isolation between the second and first domains can be stopped after an update is applied. For example, as described herein, isolation between the second and first domains can be deactivated when the update to the first domain is complete. After the update to the first domain is complete, the second and first domains can continue to communicate with each other via a shared mailbox and operate in task mode. In some embodiments, the first domain can be reset or restarted before a recovery operation after the update to the first domain is complete. Such a reset or restart may include performing BIST on the first domain, reinitializing the domain and associated security features and mechanisms, and activating the first domain. The procedures of method 1000 described herein can be applied to update the SAIL domain or MD, as described for the first domain. Therefore, in addition to facilitating the continued operation of the first domain when the second domain encounters an error, isolation can also facilitate the application of updates to the first domain while the second domain continues to operate in degraded mode.

[0121] Note, reference Figure 8 , Figure 9 and Figure 10 One or more boxes (or operations) described can be combined with one or more boxes (or operations) described in another figure in the reference diagram. For example, Figure 8 One or more boxes (or operations) can be associated with Figure 1-7 A combination of one or more boxes (or actions). As another example, with... Figure 9 One or more associated boxes can be used with Figure 6 A combination of one or more related boxes.

[0122] In one or more aspects, the technology for supporting vehicle operation may include additional aspects, such as any single aspect or any combination of aspects described below or in conjunction with one or more other processes or devices described elsewhere in this document. In a first aspect, an apparatus may be configured to perform operations including: detecting an error in a first domain of the autonomous driving system, wherein the autonomous driving system includes the processor; isolating a second domain of the autonomous driving system from the first domain; maintaining operation of the second domain of the autonomous driving system after isolating the second domain from the first domain; and, after isolating the second domain of the autonomous driving system from the first domain, sending one or more notifications to the external controller via a first communication interface between the second domain and the external controller, bypassing the first domain. In some implementations, the apparatus may be a vehicle or an autonomous driving system of a vehicle. In some implementations, the apparatus may include at least one processor and memory coupled to the processor. The processor may be configured to perform the operations described herein with respect to the apparatus. In some other implementations, the apparatus may include a non-transitory computer-readable medium on which program code is recorded, and the program code may be executable by a computer to cause the computer to perform the operations described herein with reference to the apparatus. In some implementations, the apparatus may include one or more units configured to perform the operations described herein. In some implementations, a method of wireless communication may include one or more operations described herein with reference to the apparatus.

[0123] In a second aspect, in conjunction with the first aspect, the device is further configured to perform an operation including: after maintaining operation of the second domain of the autonomous driving system, sending a shutdown request to one or more power regulators.

[0124] In a third aspect, in conjunction with one or more of the first or second aspects, the apparatus is further configured to perform operations including: receiving a notification of an update to the first domain; isolating the second domain from the first domain based on the received notification of the update; applying the update to the first domain; and stopping the isolation of the second domain from the first domain after the application of the update is completed.

[0125] In the fourth aspect, in conjunction with one or more of the first to third aspects, the first domain is configured according to a first security integrity level, and the second domain is configured according to a second security integrity level lower than the first security integrity level.

[0126] In the fifth aspect, in conjunction with one or more of the first to fourth aspects, the first domain includes the security island domain, and the second domain includes the main domain.

[0127] In the sixth aspect, in conjunction with one or more of the first to fifth aspects, bypassing the first domain to send one or more notifications to the external controller via the first communication interface includes: sending at least one of the following via the first communication interface: an indication of the health of the second domain, an indication of one or more errors detected in the second domain, or a heartbeat check notification.

[0128] In the seventh aspect, in conjunction with one or more aspects from the first to the sixth aspect, maintaining the operation of the second domain includes at least one of the following: monitoring one or more errors in the second domain, monitoring the voltage of one or more voltage rails associated with the second domain, monitoring the clock associated with the second domain, monitoring the temperature associated with the second domain, performing hardware diagnostics on the second domain, or performing software diagnostics on the second domain.

[0129] In the eighth aspect, in conjunction with one or more of the first to seventh aspects, the device is further configured to perform operations including: detecting that the error in the first domain has been resolved; and stopping the isolation of the second domain of the autonomous driving system from the first domain based on the resolution of the error.

[0130] In the ninth aspect, in conjunction with one or more of the first to seventh aspects, the first domain is located on a first die of the system-on-a-chip (SOC) of the autonomous driving system, and the second domain is located on a second die of the SOC of the autonomous driving system.

[0131] about Figure 1-4 The components, functional blocks, and modules described include processors, electronic devices, hardware devices, electronic components, logic circuits, memory, software code, firmware code, and other examples, or any combination thereof. Whether referred to as software, firmware, middleware, microcode, hardware description language, or other terms, software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures and / or functions, and other examples. Furthermore, the features discussed herein can be implemented via dedicated processor circuitry, via executable instructions, or a combination thereof.

[0132] Those skilled in the art will also understand that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of the various illustrative components, blocks, modules, circuits, and steps has been generally described above. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in varying ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure. Those skilled in the art will also readily recognize that the order or combination of components, methods, or interactions described herein is merely illustrative, and that components, methods, or interactions of various aspects of this disclosure can be combined or performed in ways different from those shown and described herein.

[0133] The various illustrative logics, logic blocks, modules, circuits, and algorithmic processes described herein can be implemented as electronic hardware, computer software, or a combination of both. Hardware and software interchangeability has been demonstrated based on the overall functional description and in the various illustrative components, blocks, modules, circuits, and processes described above. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole.

[0134] Hardware and data processing apparatuses for implementing the various illustrative logics, logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein may be implemented or executed using general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor, controller, microcontroller, or state machine. In some implementations, the processor may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In some implementations, specific processes and methods may be executed by circuitry specific to a given function.

[0135] In one or more aspects, the described functionality can be implemented using hardware, digital electronic circuits, computer software, firmware (including the structures disclosed in this specification and their structural equivalents), or any combination thereof. Implementation of the subject matter described in this specification can also be implemented as one or more computer programs (which are one or more modules of computer program instructions) encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus.

[0136] If implemented in software, the functionality can be stored on or transmitted via a computer-readable medium as one or more instructions or code. The processes of the methods or algorithms disclosed herein can be implemented in a processor-executable software module that can reside on a computer-readable medium. Computer-readable media include both computer storage media and communication media, wherein the communication medium includes any medium capable of transferring a computer program from one place to another. Storage media can be any available medium accessible by a computer. By way of example, and not limitation, such computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection may be appropriately referred to as a computer-readable medium. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while optical discs utilize lasers to optically copy data. Combinations of the above should also be included within the scope of computer-readable media. In addition, the operation of a method or algorithm may exist as one or any combination or set of code and instructions on a machine-readable medium and a computer-readable medium, which may be incorporated into a computer program product.

[0137] Various modifications to the implementations described in this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other implementations without departing from the spirit or scope of this disclosure. Therefore, the claims are not intended to limit them to the implementations shown herein, but are intended to impose the broadest scope consistent with this disclosure, the principles disclosed herein, and the novel features.

[0138] Some features described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, individual features described in the context of a single implementation can also be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as taking action in certain combinations, and even originally claimed in this manner, in some cases, one or more features from the claimed combination may be removed from that combination, and the claimed combination may involve sub-combinations or variations of sub-combinations.

[0139] Similarly, although operations are depicted in a specific order in the figures, this should not be construed as requiring such operations to be performed in the shown specific order or sequential order, or to perform all of the shown operations to achieve the desired result. Furthermore, the figures may schematically depict one or more example processes in the form of flowcharts. However, other operations not depicted may be incorporated into the schematically shown example processes. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the shown operations. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the above implementation should not be construed as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Additionally, some other implementations are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result.

[0140] The foregoing description of this disclosure is provided to enable those skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but is to be given the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for isolation in an autonomous driving system, comprising: Detect errors in the first domain of the autonomous driving system; The second domain of the autonomous driving system is isolated from the first domain; After isolating the second domain from the first domain, the operation of the second domain of the autonomous driving system is maintained; as well as After isolating the second domain of the autonomous driving system from the first domain, the second domain bypasses the first domain to send one or more notifications to the external controller via a first communication interface between the second domain and the external controller.

2. The method according to claim 1, further comprising: After maintaining the operation of the second domain of the autonomous driving system, a shutdown request is sent to one or more power regulators.

3. The method according to claim 1, further comprising: Receive notification of updates to the first domain; The second domain is isolated from the first domain based on the received update notification; Apply the update to the first domain; as well as After completing the application of the update, stop isolating the second domain from the first domain.

4. The method according to claim 1, wherein, The first domain is configured according to a first security integrity level, and the second domain is configured according to a second security integrity level that is lower than the first security integrity level.

5. The method according to claim 4, wherein, The first domain includes the security island domain, and the second domain includes the primary domain.

6. The method according to claim 1, wherein, Bypassing the first domain to send one or more notifications to the external controller via the first communication interface includes: Send at least one of the following via the first communication interface: Indications regarding the health of the second domain; An indication of one or more errors detected in the second field; or Heart rate check notification.

7. The method according to claim 1, wherein, Maintaining the operation of the second domain includes at least one of the following: Monitor one or more errors in the second domain; Monitor the voltage of one or more voltage rails associated with the second domain; Monitor the clock associated with the second domain; Monitor the temperature associated with the second domain; Perform hardware diagnostics on the second domain; or Perform software diagnostics on the second domain.

8. The method according to claim 1, further comprising: It has been detected that the error in the first domain has been resolved; as well as Based on the solution to the error, stop isolating the second domain of the autonomous driving system from the first domain.

9. The method according to claim 1, wherein, The first domain is located on the first die of the system-on-a-chip (SoC) of the autonomous driving system, and the second domain is located on the second die of the SoC of the autonomous driving system.

10. An autonomous driving system, comprising: Memory that stores processor-readable code; as well as At least one processor coupled to the memory, the at least one processor being configured to execute processor-readable code to cause the at least one processor to perform operations including: Detecting errors in a first domain of the autonomous driving system, wherein the autonomous driving system includes the processor; The second domain of the autonomous driving system is isolated from the first domain; After isolating the second domain from the first domain, the operation of the second domain of the autonomous driving system is maintained; and After isolating the second domain of the autonomous driving system from the first domain, the second domain bypasses the first domain to send one or more notifications to the external controller via a first communication interface between the second domain and the external controller.

11. The apparatus according to claim 10, wherein, The at least one processor is further configured to execute processor-readable code to cause the at least one processor to perform operations including the following: After maintaining the operation of the second domain of the autonomous driving system, a shutdown request is sent to one or more power regulators.

12. The apparatus according to claim 10, wherein, The at least one processor is further configured to execute processor-readable code to cause the at least one processor to perform operations including the following: Receive notification of updates to the first domain; The second domain is isolated from the first domain based on the received update notification; Apply the update to the first domain; as well as After completing the application of the update, stop isolating the second domain from the first domain.

13. The apparatus according to claim 10, wherein, The first domain is configured according to a first security integrity level, and the second domain is configured according to a second security integrity level that is lower than the first security integrity level.

14. The apparatus according to claim 13, wherein, The first domain includes the security island domain, and the second domain includes the primary domain.

15. The apparatus according to claim 10, wherein, Bypassing the first domain to send one or more notifications to the external controller via the first communication interface includes: Send at least one of the following via the first communication interface: Indications regarding the health of the second domain; An indication of one or more errors detected in the second field; or Heart rate check notification.

16. The apparatus according to claim 10, wherein, Maintaining the operation of the second domain includes at least one of the following: Monitor one or more errors in the second domain; Monitor the voltage of one or more voltage rails associated with the second domain; Monitor the clock associated with the second domain; Monitor the temperature associated with the second domain; Perform hardware diagnostics on the second domain; or Perform software diagnostics on the second domain.

17. The apparatus according to claim 10, wherein, The at least one processor is further configured to execute processor-readable code to cause the at least one processor to perform operations including the following: It has been detected that the error in the first domain has been resolved; as well as Based on the solution to the error, stop isolating the second domain of the autonomous driving system from the first domain.

18. The apparatus according to claim 10, wherein, The first domain is located on the first die of the system-on-a-chip (SoC) of the autonomous driving system, and the second domain is located on the second die of the SoC of the autonomous driving system.

19. A non-transitory computer-readable medium storing instructions, which, when executed by a processor, cause the processor to perform operations including: Detecting errors in the first domain of the autonomous driving system, where... The autonomous driving system includes the processor; The second domain of the autonomous driving system is isolated from the first domain; After isolating the second domain from the first domain, the operation of the second domain of the autonomous driving system is maintained; as well as After isolating the second domain of the autonomous driving system from the first domain, the second domain bypasses the first domain to send one or more notifications to the external controller via a first communication interface between the second domain and the external controller.

20. The non-transitory computer-readable medium of claim 19, further storing instructions that, when executed by a processor, cause the processor to perform operations including: After maintaining the operation of the second domain of the autonomous driving system, a shutdown request is sent to one or more power regulators.

21. The non-transitory computer-readable medium of claim 19, further storing instructions that, when executed by a processor, cause the processor to perform operations including: Receive notification of updates to the first domain; The second domain is isolated from the first domain based on the received update notification; Apply the update to the first domain; as well as After completing the application of the update, stop isolating the second domain from the first domain.

22. The non-transitory computer-readable medium according to claim 19, wherein, The first domain is configured according to a first security integrity level, and the second domain is configured according to a second security integrity level that is lower than the first security integrity level.

23. The non-transitory computer-readable medium according to claim 22, wherein, The first domain includes the security island domain, and the second domain includes the primary domain.

24. The non-transitory computer-readable medium according to claim 19, wherein, Bypassing the first domain to send one or more notifications to the external controller via the first communication interface includes: Send at least one of the following via the first communication interface: Indications regarding the health of the second domain; An indication of one or more errors detected in the second field; or Heart rate check notification.

25. The non-transitory computer-readable medium of claim 19, further storing instructions that, when executed by a processor, cause the processor to perform operations including: It has been detected that the error in the first domain has been resolved; and Based on the solution to the error, stop isolating the second domain of the autonomous driving system from the first domain.

26. A vehicle comprising: Autonomous driving systems, including: At least one processor, wherein the at least one processor is configured to perform operations including the following: Detect errors in the first domain of the autonomous driving system; The second domain of the autonomous driving system is isolated from the first domain; After isolating the second domain from the first domain, the operation of the second domain of the autonomous driving system is maintained; and After isolating the second domain of the autonomous driving system from the first domain, the second domain bypasses the first domain to send one or more notifications to the external controller via a first communication interface between the second domain and the external controller.

27. The vehicle according to claim 26, wherein, The at least one processor is also configured to perform operations including the following: After maintaining the operation of the second domain of the autonomous driving system, a shutdown request is sent to one or more power regulators of the driver assistance system.

28. The vehicle according to claim 26, wherein, The first domain includes a security island domain and is configured according to a first security integrity level, and the second domain includes a primary domain and is configured according to a second security integrity level lower than the first security integrity level.

29. The vehicle according to claim 26, wherein, The at least one processor is also configured to perform operations including the following: Receive notification of updates to the first domain; The second domain is isolated from the first domain based on the received update notification; Apply the update to the first domain; as well as After completing the application of the update, stop isolating the second domain from the first domain.

30. The vehicle according to claim 26, wherein, Bypassing the first domain to send one or more notifications to the external controller via the first communication interface includes: Send at least one of the following via the first communication interface: Indications regarding the health of the second domain; An indication of one or more errors detected in the second field; or Heart rate check notification.

Images