System-on-chip automotive safety monitoring

Abstract

The runtime program verification method involves receiving the latest values ​​of a set of system health indicators from an external system through the preemptive mechanism of the operating system running on the external system. The latest values ​​are compared to the calibration values ​​of the set of system health indicators, and the normal operation of the external system is verified based on whether the comparison between the latest values ​​and the calibration values ​​falls within a predetermined error threshold.

Description

Technical Field

[0001] Universal Chiplet Interconnect Express (UCIe) is an open standard for the interconnection and serial bus between multiple chiplets that enables the manufacture of large-scale system-on-chip (SoC) packages with components from various semiconductor manufacturers mixed. An autonomous vehicle computing system may operate using a chiplet layout that complies with the UCIe standard. One goal in creating such a computing system is to achieve a robust safety level (Safety Integrity Level: SIL) for other important electrical and electronic (E / E) automotive components of the vehicle.

Summary of the Invention

[0002] This specification describes an on-board vehicle computing system including a system-on-chip (SoC) having several dedicated chiplets for acquiring data from sensors in a vehicle and making autonomous driving decisions in a functionally safe manner.

[0003] As used herein, a system-on-a-chip (SoC) is an integrated circuit that combines multiple components of a computer or electronic system onto a single chip, providing a compact and efficient solution for a wide range of applications. The main advantage of an SoC is its compactness and reduced complexity, as all components are integrated onto a single chip. This reduces the need for additional circuit boards and other components, saving space, lowering power consumption, and reducing overall cost. The components of an SoC are often called chiplets, which are small, embedded semiconductor components that can be combined with other chiplets to form an SoC. Chiplets are designed to be highly modular and scalable, with the aim of creating complex systems from smaller, simpler components, and are typically designed to perform specific functions or tasks, such as memory, graphics processing, or input / output (I / O) functions. They are usually interconnected with each other and with the main processor or controller using high-speed interfaces. Compared to traditional monolithic chip designs, chiplets offer not only increased modularity, scalability, and manufacturing efficiency, but also the ability to be individually tested before being integrated into larger systems.

[0004] In various implementations, a computer monitoring system receives the latest values ​​of a set of system health indicators from an external system through the preemptive mechanism of the operating system running on the external system. The latest values ​​are compared to the calibration values ​​of the set of system health indicators, and the normal operation of the external system is verified based on whether the comparison between the latest values ​​and the calibration values ​​falls within a predetermined error threshold. [Brief explanation of the drawing]

[0005] The disclosures herein are illustrative and not limiting, and in the figures of the accompanying drawings, similar reference numbers refer to similar elements. [Figure 1]This is a block diagram showing an example of a computing system in which an embodiment described herein may be implemented, relating to an example described herein. [Figure 2] This block diagram shows an example of a computing system implementing multiple systems-on-a-chip (SoCs) as described in this specification. [Figure 3] This is a block diagram showing a system-on-chip example relating to the examples described herein. [Figure 4] A block diagram showing a central chiplet with an exemplary external system and health monitor relating to an example described herein. [Figure 5] This flowchart illustrates a method for verifying the normal operation of an external system, as described in this specification. [Modes for carrying out the invention]

[0006] In experimental and controlled test environments, system redundancy and Automotive Safety Level (ASIL) assessments of autonomous systems are typically not priority considerations. As autonomous driving capabilities continue to advance (e.g., beyond Level 3 autonomy) and autonomous vehicles become commonplace on public road networks, the qualification and certification of E / E components related to the autonomous operation of vehicles will be advantageous in ensuring the operational safety of these vehicles. Furthermore, novel methods for determining and certifying the qualification of hardware, software, and / or hardware / software combinations will also be advantageous in increasing public confidence and assurance that autonomous driving systems are safer than current standards. For example, certain safety standards for autonomous driving systems include safety thresholds corresponding to average human ability and attention. However, these statistics include vehicle incidents associated with drivers with reduced driving ability or distracted drivers, and do not take into account certain time windows in which the inherent risk of vehicle operation is increased (e.g., bad weather conditions, nighttime driving, winding mountain roads, etc.).

[0007] The Automotive Safety Level (ASIL) is a risk classification framework defined by ISO 26262 (Functional Safety Standards for Automotive), typically established for the E / E components of a vehicle by conducting a risk analysis of potential hazards. This analysis includes determining the severity (i.e., the degree of injury that the hazard is expected to cause, classified from S0 (no injury) to S3 (life-threatening injury)), the frequency (i.e., the relative expected frequency of operating conditions in which injury may occur, classified from E0 (very unlikely) to E4 (high probability of injury under most operating conditions)), and the avoidability (i.e., the relative likelihood that the driver can act to avoid injury, classified from C0 (generally avoidable) to C3 (difficult or impossible to avoid)). Therefore, any safety objective for any potential hazard event includes a set of ASIL requirements.

[0008] Hazards identified as quality management (QM) do not indicate safety requirements. These QM hazards may be any combination of low frequency of occurrence, low severity of potential injury resulting from the hazard, and high likelihood of avoidance by the driver in preventing the hazard and / or injury. Other hazard events are classified as ASIL-A, ASIL-B, ASIL-C, or ASIL-D depending on the varying levels of severity, frequency, and avoidability corresponding to the potential hazard. ASIL-D events correspond to the highest integrity requirements (ASIL requirements) for safety systems or E / E components of safety systems, while ASIL-A includes the lowest integrity requirements. For example, vehicle airbags, anti-lock brakes, and power steering systems typically have an ASIL-D grade, and the risks associated with failure of these components (e.g., the likely severity of injury and lack of vehicle control to prevent those injuries) are relatively high.

[0009] As described herein, ASIL relates to both the risk and the requirements that depend on that risk, and various combinations of severity, frequency, and avoidability are quantified to form a representation of the risk (for example, a vehicle airbag system may have a relatively low frequency classification but high values ​​for severity and avoidability). As stated above, the amounts of severity, frequency, and avoidability of a given hazard have traditionally been determined using values ​​for severity (e.g., S0-S3), frequency (e.g., E0-E4), and avoidability (e.g., C0-C3) in the ISO 26262 series, and these values ​​are then used to classify the ASIL requirements for a particular safety system component. As described herein, a safety system can implement a variety of mitigation measures, which may range from warnings (e.g., visual, auditory, or tactile warnings) to mild interventions (e.g., brake assistance or steering assistance), severe interventions and / or evasive maneuvers (e.g., taking over control of one or more control mechanisms such as the steering system, acceleration system, or braking system), and fully autonomous control of the vehicle.

[0010] Current fully autonomous driving systems can feature non-deterministic reasoning models in which the system performs one or more perception, object detection, object classification, motion prediction, motion planning, and vehicle control techniques based on, for example, two-dimensional image data, to perform all autonomous driving tasks. In such implementations, it is anticipated that certifying the entire autonomous driving system and assigning an ASIL rating may be difficult or impossible. To address these shortcomings in current implementations, this specification describes an autonomous driving system capable of performing deterministic immediate reasoning operations for a specific hardware configuration, which enables certification and ASIL grading of various components, software configurations of the system, and / or the autonomous driving system itself.

[0011] To properly certify a system, the first option is to test, conform (if necessary), and document the software to demonstrate compliance with the standard (ISO 26262-8:2018, see Section 12). For very large codebases like the Linux® operating system, this is simply not a practical option. The second option is to develop a suitable monitoring system for the operating system components. Thus, through the use of runtime program verification, software that runs compute chiplets on an SoC architecture can be certified even if, without such runtime program verification, they would not meet the required level of safety requirements (e.g., ASIL-D).

[0012] As illustrated in the examples described herein, the use of a dual SoC configuration in which each SoC in a pair alternates between primary and backup roles can facilitate overall certification and ASIL grading of a vehicle's autonomous driving system. In this configuration, the first and second SoCs utilize isolated power supplies and can be electrically coupled to each other via electronic fuses (eFuses; for example, active circuit protection devices having integrated field-effect transistors (FETs) used to limit current and voltage to safe levels during fault conditions), thereby further enhancing the ASIL grading of the configuration. The SoCs can have direct memory access to each other (for example, via the functional safety components of each SoC), which can facilitate dynamic health monitoring, error checking, and seamless transitions between primary and backup statuses.

[0013] In some implementations, a computing system can perform one or more of the functions described herein using a learning-based approach, such as running an artificial neural network (e.g., a recurrent neural network, a convolutional neural network, etc.) or one or more machine learning models. Such learning-based approaches can further be adapted to computing systems that store or include one or more machine learning models. In one embodiment, the machine learning models may include unsupervised learning models. In one embodiment, the machine learning models may include neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear and / or linear models. The neural networks may include feedforward neural networks, recurrent neural networks (e.g., long-short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some exemplary machine learning models may leverage attentional mechanisms such as self-attention. For example, some exemplary machine learning models may include multi-head self-attention models (e.g., transformer models).

[0014] As described herein, “Network” or “one or more networks” may include any type of network or combination of networks that enable communication between devices. In one embodiment, a network may include one or more of the following: a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, or a combination thereof, and may include any number of wired or wireless links. Communication over one or more networks may be achieved via a network interface using, for example, any type of protocol, protection scheme, encoding, format, packaging, etc.

[0015] As further described herein, an “autonomous map” or “autonomous driving map” includes a ground truth map recorded by a mapping vehicle using various sensors (e.g., LiDAR sensors and / or a set of cameras or other imaging devices) and labeled to show traffic objects and / or traffic right-of-way rules at a given location. For example, a given autonomous map may be labeled by a human based on observed traffic signs, traffic signals, and lane markings within the ground truth map. In a further example, reference points or other points of interest may be further labeled on the autonomous map for additional assistance to the autonomous vehicle. The autonomous vehicle or autonomous driving vehicle can then utilize the labeled autonomous map to perform positioning, attitude determination, change detection, and various other operations required for autonomous driving on public roads. For example, the autonomous vehicle may refer to the autonomous map to determine traffic rules (e.g., speed limits) at the vehicle’s current location and may dynamically compare live sensor data from its onboard sensor suite with the corresponding autonomous map to safely navigate along the current route.

[0016] Among other advantages, the examples described herein achieve technical benefits such as providing redundancy and functional safety monitoring for SoCs to improve the safety level of autonomous vehicle computing systems.

[0017] In one or more examples described herein, the methods, techniques, and operations performed by the computing device are performed by program or as a computer implement. "By program" as used herein means using code or computer executable instructions. These instructions can be stored in one or more memory resources of the computing device. The steps performed by program may be automatic or not.

[0018] One or more examples described herein can be implemented using a programmatic module, engine, or component. A programmatic module, engine, or component may include a program, a subroutine, a portion of a program, or a software or hardware component capable of performing one or more defined tasks or functions. Where used herein, a module or component may reside on a hardware component independent of other modules or components. Alternatively, a module or component may be a shared element or shared process of other modules, programs, or machines.

[0019] Some of the examples described herein may generally require the use of a computing device, including processing and memory resources. For example, one or more of the examples described herein may be implemented entirely or partially using network equipment (e.g., a router) on a computing device such as a server and / or a personal computer. Memory resources, processing resources, and network resources may be used in connection with the establishment, use, or implementation of any of the examples described herein (including the implementation of any method or any system).

[0020] Furthermore, one or more examples described herein may be implemented through the use of instructions executable by one or more processors. These instructions may be carried on non-temporary computer-readable media. The machines shown in or described using the following figures provide examples of processing resources and computer-readable media capable of carrying and / or executing instructions for implementing the examples disclosed herein. In particular, many machines shown with the examples of the present invention include processors and various forms of memory for holding data and instructions. Examples of non-temporary computer-readable media include persistent memory storage devices such as hard drives in personal computers or servers. Other examples of computer storage media include portable storage units such as flash memory or magnetic memory. Computers, terminals, and network-enabled devices are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable media. Furthermore, examples of computer storage media may be implemented in the form of computer programs.

[0021] Examples of computing systems Figure 1 is a block diagram showing a computing system 100 as an example in which embodiments described herein may be implemented, relating to examples described herein. In one embodiment, the computing system 100 may include one or more control circuits 110, which may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuits (PLCs), or programmable logic / gate arrays (PLAs / PGAs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), system-on-a-chip (SoCs), or any other control circuits. In some implementations, the control circuits 110 and / or the computing system 100 may be part of, or form part of, a vehicle control unit (also called a vehicle controller) that is embedded in or located in a vehicle (e.g., a Mercedes-Benz® vehicle, truck, or van). For example, the vehicle controller may be an infotainment system controller (e.g., an infotainment head unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a central exterior and interior controller (CEIC), a zone controller, an autonomous vehicle control system, or any other controller (the term "or" is used herein interchangeably with "and / or"), or may include such controllers.

[0022] In one embodiment, the control circuit 110 may be programmed by one or more computer-readable instructions or computer-executable instructions stored in a non-temporary computer-readable medium 120. The non-temporary computer-readable medium 120 may be a memory device, also called a data storage device, and the memory device may include electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any preferred combination thereof. The non-temporary computer-readable medium 120 may, for example, form a floppy disk, a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), dynamic random access memory (DRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), and / or memory stick. In some cases, the non-temporary computer-readable medium 120 may store computer-executable instructions or computer-readable instructions.

[0023] In various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, when computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a set of software instructions or code configured to cause the control circuit 110 to perform one or more functional tasks. Modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 110 or other hardware components execute the module or computer-readable instructions.

[0024] In further embodiments, the computing system 100 can include a communication interface 140 that enables communication via one or more networks 150 to send and receive data. In various examples, the computing system 100 can use the communication interface 140 to communicate with fleet vehicles via one or more networks 150 to receive sensor data and perform intersection classification as described throughout this disclosure. In certain embodiments, the communication interface 140 may be used to communicate with one or more other systems. The communication interface 140 may include any circuitry, component, software, etc. for communicating via one or more networks 150 (e.g., local area network, wide area network, Internet, secure network, cellular network, mesh network, and / or peer-to-peer communication link). In some implementations, the communication interface 140 may include one or more of, for example, a communication controller, receiver, transceiver, transmitter, port, conductor, software, and / or hardware for communicating data / information.

[0025] As an illustrative embodiment, the control circuit 110 of the computing system 100 may include a dual SoC arrangement that facilitates the various methods and techniques described throughout this disclosure. In various examples, the SoC may execute a set of tasks in an arrangement of a primary SoC and a backup SoC, where the primary SoC performs the set of tasks and the backup SoC maintains a standby state and monitors the status and / or state of the primary SoC. In various implementations, the set of tasks may include a set of autonomous driving tasks, such as autonomously operating the vehicle along a route of travel, perception, object detection and classification, grid occupancy determination, sensor data fusion and processing, motion prediction (e.g., motion prediction of dynamic external entities), motion planning, and vehicle control tasks. As described herein, multiple dual SoC arrangements may be implemented to perform these tasks, and each SoC pair is configured in a manner described in detail below.

[0026] Description of the System FIG. 2 is a block diagram showing an example of a multi-system-on-chip (MSoC) according to an example described herein. In various examples, the Msoc200 can include a first SoC210 having a first memory 215 and a second SoC220 having a second memory 225 coupled by an interconnect 240 (e.g., an inter-chip interconnect with ASIL-D evaluation), which enables each of the first SoC210 and the second SoC220 to read each other's memories 215, 225. During a given session, the first SoC210 and the second SoC220 can alternate roles between a primary SoC and a backup SoC. As described herein, the primary SoC can perform various autonomous driving tasks, such as perception, object detection and classification, grid occupancy determination, sensor data fusion and processing, motion prediction (e.g., motion prediction of dynamic external entities), motion planning, and vehicle control tasks. The backup SoC can maintain a set of computing components (e.g., a CPU, an ML accelerator, and / or a memory dielet) in a low-power state and can continuously or periodically read the memory of the primary SoC.

[0027] For example, if the first SoC210 is the primary SoC and the second SoC220 is the backup SoC, the first SoC210 performs a set of autonomous driving tasks and exposes state information corresponding to these tasks to the first memory 215. The second SoC220 reads the exposed state information from the first memory 215 and continuously checks whether the first SoC210 is operating within nominal thresholds (e.g., temperature threshold, voltage threshold, bandwidth and / or memory threshold) and whether the first SoC210 is performing the set of autonomous driving tasks without errors. In some examples, the nominal thresholds may be obtained from the manufacturer's specifications or determined through a testing process of the underlying hardware. Thus, the second SoC220 performs health monitoring (i.e., comparing the latest values ​​from the first SoC210 with the nominal thresholds) and error management tasks for the first SoC210 and takes over control of the set of autonomous driving tasks when trigger conditions are met. As described herein, trigger conditions may correspond to failures, malfunctions, or other errors that the first SoC210 experiences, such as exceeding one of the nominal thresholds, which may affect the performance of the set of tasks performed by the first SoC210.

[0028] In various implementations, the second SoC220 may expose state information corresponding to the fact that the computing components are being kept in a standby state (for example, a low-power state in which the second SoC220 maintains preparation for taking over a set of tasks from the first SoC210). In such an example, the first SoC210 may monitor the state information of the second SoC220 by continuously or periodically reading the memory 225 of the second SoC220, and may also perform health check monitoring and error management for the second SoC220. For example, if the first SoC210 detects a fault, failure, or other error in the second SoC220, the first SoC210 may trigger the second SoC220 to perform a system reset or reboot.

[0029] In one example, the first SoC210 and the second SoC220 may each include a Functional Safety (FuSa) component that performs health monitoring and error management tasks. The FuSa component can maintain power on each SoC regardless of whether the SoC is operating in primary or backup mode. Thus, the backup SoC may keep other components in a low-power state while the FuSa component is powered up and performing the health monitoring and error management tasks described herein.

[0030] In various embodiments, when the first SoC210 operates as a primary SoC, the state information exposed in the first memory 215 may correspond to a set of tasks performed by the first SoC210. ​​For example, the first SoC210 may expose any information corresponding to the vehicle's surrounding environment (e.g., external entities identified by the first SoC210, their locations and predicted trajectories, and detected objects such as traffic signals, signs, lane markings, and crosswalks). The state information may further include the operating temperature of the computing components of the first SoC210, the bandwidth usage and available memory of the chiplets of the first SoC210, and / or faults or errors in these components, or information indicating faults or errors.

[0031] In a further embodiment, when the second SoC220 operates as a backup SoC, the state information exposed in the second memory 225 may correspond to the state of each computing component of the second SoC220. In particular, these components may operate in a low-power state in which they are ready to take over the set of tasks being performed by the first SoC210. ​​The state information may include whether the components are operating within a nominal temperature and other nominal ranges (e.g., available bandwidth, power, memory, etc.).

[0032] As described throughout this disclosure, the first SoC210 and the second SoC220 can switch between operating as primary SoCs and backup SoCs (for example, each time the system 200 is rebooted). For example, in a computing session following a session in which the first SoC210 operated as the primary SoC and the second SoC220 operated as the backup SoC, the second SoC220 may assume the role of the primary SoC and the first SoC210 may assume the role of the backup SoC. This process of switching roles between the two SoCs is intended (or thought to be intended) to cause the hardware components of each SoC to degrade substantially evenly, thereby extending the overall lifespan of the computing system 200.

[0033] The aim is to improve the safety level (e.g., ASIL rating) of the computing system 200 and the overall autonomous driving system of the vehicle by providing an MSoC configuration of the computing system 200. As described herein, the autonomous driving system may include any number of dual SoC configurations, each capable of performing a set of autonomous driving tasks. In this configuration, the backup SoC dynamically monitors the health of the primary SoC according to a set of functional safety operations, so that when a fault, failure, or other error is detected, the backup SoC can immediately power up its components and take over the set of tasks from the primary SoC. Further descriptions of the SoCs and their computing components are provided below with reference to Figure 3.

[0034] Exemplary SoC Figure 3 is a block diagram showing an exemplary system-on-chip 300 relating to an example described herein. The system-on-chip 300 may comprise either a first SoC 210 or a second SoC 220, as illustrated and described in relation to Figure 2. Furthermore, the system-on-chip example 300 shown in Figure 3 may include additional components, and the components of the system-on-chip 300 may be arranged in various alternative configurations other than those shown. Accordingly, the system-on-chip 300 in Figure 3 is described herein as an example arrangement for illustrative purposes and is not intended to limit the scope of this disclosure in any way.

[0035] Referring to Figure 3, the sensor data input chiplet 310 of the system-on-chip 300 can receive sensor data from various vehicle sensors 305 of the vehicle. These vehicle sensors 305 may include any combination of image sensors (e.g., single camera, binocular camera, fisheye lens camera, etc.), LIDAR sensors, radar sensors, ultrasonic sensors, proximity sensors, etc. Simultaneously with receiving the sensor data, the sensor data input chiplet 310 may automatically dump the sensor data into the cache memory 331 of the central chiplet 320. The sensor data input chiplet 310 may also include an image signal processor (ISP) responsible for capturing, processing, and improving images acquired from the various vehicle sensors 305. The ISP acquires raw image data and performs a series of complex image processing operations, such as color, contrast, and brightness correction, noise reduction, and image enhancement, to create higher quality images ready for further processing or analysis by other chiplets of the SoC 300. The ISP may also include features such as autofocus, image stabilization, and advanced scene recognition to further improve the quality of the captured images. The ISP can then store the high-quality images in the cache memory 331.

[0036] In some embodiments, the sensor data input chiplet 310 exposes identification information for each item of sensor data (e.g., image, point cloud map, etc.) to the shared memory 330 of the central chiplet 320, which functions as a central mailbox for synchronizing the workloads of various chiplets. The identification information may include details such as the address where the data is stored in the cache memory 331, the type of sensor data, which sensor captured the data, and the timestamp when the data was captured.

[0037] To communicate with the central chiplet 320, the sensor data input chiplet 310 transmits data through interconnect 311a. Interconnections 311a-311f each represent die-to-die (D2D) interfaces between chiplets of the SoC 300. In some embodiments, the interconnects include a high-bandwidth data path used for general data purposes to the cache memory 331 and a high-reliability data path for transmitting functional safety and scheduler information to the shared memory 330. Network-on-chip (NoC) network interface units (NIUs) on the chiplets may be configured to generate error correction code (ECC) data on both the high-bandwidth and high-reliability data paths. Each corresponding NIU on its paired die has the same ECC configuration and generates and checks the ECC data to ensure end-to-end error correction coverage. In the case of a highly reliable data path, the NIU can transmit functional safety and scheduler information in two redundant transactions, with the second transaction ordering the bits in reverse order of the first transaction (for example, from bit 31 to 0 on a 32-bit bus). Furthermore, if an error is detected during data transfer between chiplets on a highly reliable data path, the NIU can reduce the transmission rate (transfer speed) to improve reliability.

[0038] Depending on bandwidth requirements, the interconnect may include two or more die-to-die interfaces. For example, interconnect 311a may include two interfaces to support relatively high-bandwidth communication between the sensor data input chiplet 310 and the central chiplet 320.

[0039] In one embodiment, interconnects 311a-311f implement the Universal Chiplet Interconnect Express (UCIe) standard and communicate via indirect mode, allowing each of the chiplet's host processors to access remote memory as if it were local memory. This is achieved by using dedicated NoC NIUs that provide hardware-level support for Remote Direct Memory Access (RDMA) operation. These NIUs also enable freedom from interference between network-connected devices. In UCIe indirect mode, the host processor sends a request to the NIU, which then accesses the remote memory and returns the data to the host processor. This approach enables efficient, low-latency access to remote memory, which can be particularly useful in distributed computing and data-intensive applications. Furthermore, UCIe indirect mode can be used with a wide range of different network topologies and protocols, providing a high degree of flexibility.

[0040] In various examples, the system-on-chip 300 may include additional chiplets that can store, modify, or otherwise process sensor data cached by the sensor data input chiplet 310. The system-on-chip 300 may include an autonomous driving chiplet 340 that can perform actions to determine the physical characteristics of the environment around the sensor. These actions may include autonomous vehicle perception, sensor fusion, trajectory prediction, and / or other autonomous driving algorithms. To perform these actions, the autonomous driving chiplet 340 may include dedicated hardware such as a digital signal processor (DSP), a direct memory access (DMA) engine, and a neural network (NN) accelerator. The autonomous driving chiplet 340 may be connected to a dedicated HBM-RAM chiplet 335, which may expose all status information, variables, statistics, and / or processed sensor data processed by the autonomous driving chiplet 340 to the dedicated HBM-RAM chiplet 335.

[0041] In various applications, the system-on-chip 300 may further include a machine learning (ML) accelerator chiplet 340, specifically designed to accelerate AI workloads such as image inference or other sensor inference using machine learning to achieve high performance and low power consumption for those workloads. The ML accelerator chiplet 340 may include an engine designed to efficiently process graph-based data structures commonly used in AI workloads, and a highly parallel processor that enables efficient processing of large amounts of data. The ML accelerator chiplet 340 may also include a dedicated hardware accelerator for common AI operations such as matrix multiplication and convolution, as well as a memory hierarchy designed to optimize memory access for AI workloads that often have complex memory access patterns.

[0042] The general-purpose computing chiplet 345 can provide general-purpose computing for the system-on-chip 300. For example, the general-purpose computing chiplet 345 may include a high-power central processing unit and / or a graphical processing unit, which can support computing tasks for the central chiplet 320, the autonomous driving chiplet 340, and / or the ML accelerator chiplet 350.

[0043] In various implementations, the shared memory 330 can store programs and instructions for performing autonomous driving tasks. The shared memory 330 of the central chiplet 320 may further include a reservation table that provides various chiplets with the information necessary to perform individual tasks (e.g., sensor data items and their locations in memory). A further description of the shared memory 330 in the context of the dual SoC configuration described herein is provided below with reference to Figure 4. The central chiplet 320 also includes a large cache memory 331 that supports invalidation and flashing of stored data.

[0044] Cache misses and evicting from the cache memory 331 are transmitted by a high-bandwidth memory (HBM) RAM chiplet 355 connected to the central chiplet 320. The HBM-RAM chiplet 355 may contain status information, variables, statistics, and / or sensor data for all other chiplets. In one example, the information stored in the HBM-RAM chiplet 355 may be stored for a predetermined period (e.g., 10 seconds), after which the data is deleted or flushed. For example, if an autonomous vehicle experiences a malfunction, the information stored in the HBM-RAM chiplet 355 may contain all the information necessary to diagnose and resolve the malfunction. The cache memory 331 holds new data that is available with lower latency and lower power consumption compared to accessing data from the HBM-RAM chiplet 355.

[0045] Figure 4 is a block diagram showing a central chiplet with an exemplary external system and health monitor. In various implementations, the health monitor 440 on the central chiplet 420 receives the latest values ​​of a set of system health indicators from the external system 410. The latest values ​​are compared to the calibrated values ​​of the set of system health indicators, and the normal operation of the external system 440 is verified based on whether the comparison between the latest values ​​and the calibrated values ​​falls within a predetermined error threshold. In some examples, the external system 410 is one of the general-purpose computing chiplets 345, as described in the context of Figure 3.

[0046] In the external system 410, the operating system 412 acts as an intermediary between hardware components such as memory 416 and processor 412. The operating system 412 handles memory management tasks, including allocating and deallocating memory, managing virtual memory, and implementing memory protection mechanisms to prevent unauthorized access. The aspects of memory management include various software components such as the stack, heap, and operating system-managed memory. The stack is an area of ​​memory used to manage function calls, local variables, and store return addresses. The operating system 412 maintains a stack pointer that tracks the current position in the stack. This ensures the creation and destruction of appropriate stack frames during function calls. The heap, on the other hand, is a dynamically allocated area of ​​memory used to manage dynamic memory requests made by processes running on the external system 410. The operating system 412 facilitates heap allocation by providing system calls or memory management functions that enable these processes to request and free memory blocks on the heap.

[0047] Similarly, the operating system 412 manages the processor 418 by scheduling its resources and allocating them to different processes. This ensures fairness and efficiency by using scheduling algorithms to determine which process receives CPU time, switching between processes via context switching, and providing mechanisms for process synchronization and intercommunication through the use of the operating system (OS) scheduler 414. When a process exhausts its allocated CPU time slice or voluntarily relinquishes its use of one of the processors 418, the OS scheduler 414 initiates a context switch. During this switch, the OS scheduler 414 stores the context of the current process, including the register value program counter, in its process control block (PCB). It then selects the next process to execute from the scheduling queue, restores its context from its PCB, and transfers control to it. This process switching is achieved through a combination of hardware support (such as context switching instructions provided by the processor 418) and the scheduler's management of the scheduling queue and process state.

[0048] In some embodiments, the OS scheduler 414 includes the ability to report values ​​of a set of health indicators to a separate system, such as a central chiplet 420. For example, each time the context of the OS scheduler 414 switches from a first program to a second program, the OS scheduler 414 determines at least some values ​​from a set of health indicators for the first program and sends those values ​​to the central chiplet 420. The health indicators are selected metrics that, when compared to predetermined calibration values ​​or input into a trained machine learning model, can indicate the normal or abnormal operation of software and hardware components of an external system 410. These health indicators may include the amount of memory allocated to the first program, program runtime, stack pointer changes, and heap values.

[0049] In one embodiment, the central chiplet 420 stores the values ​​of a set of health indicators in an external OS status buffer 435 in shared memory 430. In addition to the values ​​of each set of health indicators, the buffer may include identifying metadata such as the source of the value (i.e., an external system 410), the program or process associated with the value (i.e., a process ID), and a timestamp when the value was recorded.

[0050] In one example, the health monitor 440 periodically reads health indicator values ​​from the external OS status buffer 435. The health monitor 440 then compares each value to its corresponding calibration value to determine whether the software and hardware of the external system 410 are functioning correctly. If the comparison indicates that each health indicator value is within the error threshold, the health monitor 440 confirms that the external system 410 is operating normally. However, if the comparison indicates that one or more of the health indicator values ​​deviate from their calibration value by more than the acceptable error threshold, the health monitor 440 can determine that the external system 410 is not functioning correctly. For example, the health monitor 440 may detect that a program on the external system 410 is using more memory than normal, taking longer to execute than normal, or that its stack or heap pointer is behaving irregularly compared to its calibration value.

[0051] In some embodiments, the calibration value and error threshold are predetermined values ​​loaded and stored in the shared memory 430. The calibration value can be determined by rigorous testing of the hardware and software of the external system 410 (e.g., at the factory before the system is sold to the customer). The error threshold can similarly be determined through testing and manufacturer guidelines (e.g., temperature and voltage guidelines for the processor 418, such as not exceeding 100 degrees Celsius or 1.5 volts at the location where the probe is positioned).

[0052] In one implementation, the calibration value is determined by a machine learning model trained through a machine learning process. For example, a labeled dataset can be prepared that includes examples of both normal and abnormal behavior, covering a wide range of potential anomalies that may occur in the external system 410. A machine learning model (e.g., a support vector machine, autoencoder, Bayesian network) can be trained using the labeled dataset to learn the patterns and characteristics of the normal behavior of the components of the external system 410. Thus, the calibration value includes the patterns and characteristics of the normal behavior learned by the model. Before deployment, the performance of the trained model can be evaluated using evaluation metrics such as precision, recall, F1 score, and area under the receiver operating characteristic curve.

[0053] When deployed as part of the health monitor 440, the machine learning model uses the latest values ​​in combination with patterns and characteristics of normal behavior (i.e., calibration values) determined during the training process to verify the normal (normal) behavior of the external system. For example, the latest values ​​may be input into the model to detect anomalies indicating abnormal behavior (i.e., abnormal values ​​or events that deviate from normal (normal) behavior). In some embodiments, an analysis of the model's predicted score or the distribution of normal and abnormal instances is used to determine an error threshold that describes the difference between normal and abnormal behavior of the external system 410.

[0054] Additionally or alternatively, the operating system 412 of the external system 410 includes a test driver 415 for running a test program on the processor 418. In one example, the test program stresses (loads) the processor 418 and performs demanding test tasks on the processor 418 to determine whether the processor 418 is capable of operating correctly under heavy load. The test program can read voltage and temperature values ​​from probes placed on the processor 418. The test driver 415 reports these values ​​to the health monitor 440 (which may store the values ​​in the external OS status buffer 435), and the health monitor 440 can then determine the status of the processor 418 by comparing the values ​​to calibration values ​​and error thresholds. The status may include information such as whether the processor 418 is operating normally and an estimate of the remaining service life before the processor 418 degrades to a state that could pose a safety concern.

[0055] In some cases, the health monitor 440 can detect software failures or bugs, but comparing health indicator values ​​with calibration values ​​can also determine whether a program on the external system 410 has been tampered with. For example, malicious code injected into one of the programs running on the operating system 412 of the external system 410 may result in the program's runtime execution being longer than its runtime calibration value, using more memory, or manipulating the stack pointer and register values ​​in unexpected ways, and the health monitor 440 can detect these.

[0056] In some embodiments, if the health monitor 440 determines that the comparison between the latest value and the calibration value falls outside a predetermined error threshold, it alerts the functional safety program 455 in the shared memory 430 of the central chiplet 420. The health monitor 440 can send an error detection alert to the functional safety program 455 via the scheduling program 450, and the functional safety program 455 can trigger one or more corrective actions in response. In one example, the corrective action is to switch the primary system-on-chip to a backup system-on-chip in a multi-system-on-chip architecture (see Figure 2) to control the autonomous vehicle.

[0057] Figure 5 is a flowchart illustrating how to verify the normal operation of an external system.

[0058] While the external system's operating system is running, the scheduler receives a preemptive switch (510). The external system's operating system sends a snapshot of the latest values ​​of the external system's system health indicators (520). The operating system scheduler then switches tasks (525). A health monitor running on a separate system (e.g., a separate chiplet in a system-on-chip architecture) receives a snapshot of the latest values ​​of the system health indicators sent from the external system (530). In some embodiments, the health monitor compares each latest value of the system health indicators to a stored calibration value (540). The health monitor verifies the normal operation of the external system by determining whether each latest value is within the error threshold of the calibration value (550). If the comparison of one of the latest values ​​with its calibration value is determined to be outside the given error threshold, the health monitor may alert the functional safety program (560).

[0059] It is intended that the examples described herein be extended to the individual elements and concepts described herein, independently of other concepts, ideas, or systems, and that combinations of elements described in any part of this application be included as examples. While examples are described in detail herein with reference to the accompanying drawings, it should be understood that the concepts are not limited to those exact examples. Therefore, many modifications and variations will be apparent to those skilled in the art. Accordingly, the scope of the concepts is intended to be defined by the following claims and their equivalents. Furthermore, specific features described individually or as part of an example are intended to be combined with other features or parts of other examples described individually, even if other features and examples do not refer to those specific features.

Claims

1. A method for verifying a runtime program, wherein the method is performed by one or more processors. The latest values ​​of the set of system health indicators of the external system are received through the preemptive mechanism of the operating system running on the external system. The latest value is compared with the calibration value of the set of system health indicators. The normal operation of the external system is verified based on whether the comparison between the latest value and the calibration value falls within a predetermined error threshold. method.

2. Furthermore, if it is determined that the comparison between the latest value and the calibration value is outside the predetermined error threshold, the functional safety program will issue a warning. The method according to claim 1.

3. The method according to claim 2, wherein the functional safety program switches to the backup system of the external system.

4. The method according to claim 1, wherein the calibration value is determined by a machine learning model trained through a machine learning process.

5. The method according to claim 4, wherein the machine learning model uses the latest values ​​to verify the normal operation of the external system.

6. Furthermore, a test program is executed on the processor of the external system to determine at least some of the latest values ​​of the set of system health indicators. The aforementioned test program is executed according to a predetermined schedule. The method according to claim 1.

7. The method according to claim 1, wherein the system health indicator includes the amount of allocated memory, program runtime, stack pointer changes, and heap values.

8. The method according to claim 1, wherein the one or more processors are part of a first chiplet, and the external system is a second chiplet connected to the first chiplet in a system-on-chip arrangement.

9. The method according to claim 8, wherein the system-on-chip is configured for autonomous driving.

10. The method according to claim 1, wherein the preemptive mechanism is a scheduler that runs on the operating system.

11. The method according to claim 1, wherein the external system does not meet the required level of safety requirements without runtime program verification.

12. A computer monitoring system, Memory resources for storing instructions, The system comprises one or more processors that perform an operation using the instructions stored in the memory resources, and the operation is, The latest values ​​of the set of system health indicators of the external system are received through the preemptive mechanism of the operating system running on the external system. The latest value is compared with the calibration value of the set of system health indicators. The normal operation of the external system is verified based on whether the comparison between the latest value and the calibration value falls within a predetermined error threshold. A computer monitoring system that includes the following.

13. The one or more processors perform further operations, the operations being: If the comparison between the latest value and the calibration value is determined to be outside the predetermined error threshold, the functional safety program is alerted. The computer monitoring system according to claim 12, including the following:

14. The computer monitoring system according to claim 13, wherein the functional safety program switches to the backup system of the external system.

15. The one or more processors perform further operations, the operations being: The process includes running a test program on the processor of the external system to determine at least some of the latest values ​​of the set of system health indicators, the test program being run according to a predetermined schedule. The computer monitoring system according to claim 12.

16. The computer monitoring system according to claim 12, wherein the system health indicator includes the amount of allocated memory, program runtime, stack pointer changes, and heap values.

17. The computer monitoring system according to claim 12, wherein the one or more processors are part of a first chiplet, and the external system is a second chiplet connected to the first chiplet in a system-on-chip arrangement.

18. The computer monitoring system according to claim 12, wherein the external system does not meet the required level of safety requirements without runtime program verification.

19. A non-temporary computer-readable medium for storing instructions, wherein when an instruction is executed by one or more processors of a computing system, the computing system... The latest values ​​of the set of system health indicators of the external system are received through the preemptive mechanism of the operating system running on the external system. The latest value is compared with the calibration value of the set of system health indicators. The normal operation of the external system is verified based on whether the comparison between the latest value and the calibration value falls within a predetermined error threshold. Non-temporary computer-readable media.

20. If it is determined that the comparison between the latest value and the calibration value is outside the predetermined error threshold, the functional safety program is warned. A non-temporary computer-readable medium according to claim 19, further storing instructions.

Images