IN-SYSTEM TEST ARCHITECTURE THAT SUPPORTS RECONFIGURED FIELD TESTING FOR AUTONOMOUS SYSTEMS AND APPLICATIONS

Abstract

Embodiments of the present disclosure relate to applications, platforms, architectures, etc., for using a master test image that can be used for several different tests. For example, a test system may contain a register that can be loaded with test configurations corresponding to one or more tests. The test configurations may each correspond to sets of control packages contained in the master test image and which can be used or executed for the respective tests. The test configurations may specify execution orders for their respective sets of control packages, and the execution order of one or more of the control packages contained in the sets of control packages may differ from a default execution order for such control packages as specified in the master test image.Such a configuration can therefore provide the flexibility to perform many different tests using a single master test image.

Description

BACKGROUND

[0001] Computing systems (e.g., systems-on-a-chip, SoCs) can be subjected to various tests relating to different operations and functions. For example, in-system testing (IST) is a test scheme that can be used to perform tests such as structural scans and / or memory tests on computing systems. Furthermore, IST can include performing tests on a complete computing system (e.g., a complete SoC, multiple SoCs, etc.), such as the complete functionality of the computing system, all operations performed by the computing system, all components of the computing system (including hardware and / or software), and so on.Additionally or alternatively, IST may include performing tests with respect to one or more subsets of the computing system, including some of the functionality and / or operations that can be performed by the computing system and / or one or more subsets of the components of the computing system.

[0002] In the context of vehicle systems, IST can be performed either only during Key-ON (e.g., when the vehicle is running or the systems are in an "ON" state), only during Key-OFF (e.g., when the vehicle is in an "OFF" state), or both during Key-ON and OFF. It may be desirable to have the flexibility to choose a shorter test during Key-ON (e.g., testing only critical components in a corresponding SoC) versus a full test (e.g., testing an entire SoC) during Key-OFF, in order to keep the test latency within an acceptable timeframe while making a trade-off for coverage.

[0003] With the existing IST architecture (e.g., existing IST hardware (IST-HW)) and existing IST configurations, separate test images are generated for different tests to support the desired testing flexibility. For example, test images typically contain control packages that specify which operations can be performed with respect to a particular test. Furthermore, the control packages are typically included in the test images as a linked list, where the execution sequence from one control package to the next is fixed and predefined by the linked list. Such a configuration therefore typically requires different test images to be used for different tests. However, the logistics of generating, characterizing, and producing multiple test images are unsustainable.Furthermore, this approach is difficult to scale, as the requirements for the test image can vary depending on the users. SUMMARY

[0004] Embodiments of the present disclosure relate to applications, platforms, architectures, etc., for using a master test image that can be used for several different tests. For example, a test system may contain a register that can be loaded with test configurations corresponding to different tests. The test configurations may each correspond to sets of control packages contained in the master test image and which can be used or executed for the respective tests. The test configurations may specify execution orders for their respective sets of control packages, wherein the execution order of one or more of the control packages contained in the sets of control packages may differ from a default execution order for such control packages as specified in the master test image.Such a configuration can therefore provide the flexibility to perform many different tests using a single master test image. This flexibility creates the ability to improve in-system (IST) testing by allowing specific tests to be tailored to specific requirements without having to create individual test images for each test. This improvement in testing can also contribute to improving the systems under test by providing greater flexibility in identifying opportunities for improvement.

[0005] The invention is defined by the claims. To illustrate the invention, aspects and embodiments that may or may not be within the scope of protection of the claims are described herein.

[0006] The revelation extends to all novel aspects or features described and / or illustrated herein.

[0007] Further features of the disclosure are characterized by the independent and dependent claims.

[0008] Any feature of one aspect of the disclosure can be applied in any suitable combination to other aspects of the disclosure. In particular, procedural aspects can be applied to apparatus or system aspects, and vice versa.

[0009] Furthermore, features implemented in hardware can be implemented in software and vice versa. Any reference to software and hardware features herein should be interpreted accordingly.

[0010] Each system or device feature described herein can also be provided as a process feature, and vice versa. System and / or device aspects that are functionally described (including means plus functional features) can alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and allocated working memory.

[0011] It is also understood that certain combinations of the various features described and defined in each aspect of the revelation can be implemented and / or provided and / or used independently of one another.

[0012] The disclosure also provides computer programs and computer program products comprising software code designed to perform one of the methods described herein when executed on a data processing device and / or to embody one of the device and system features described herein, including one or all component steps of a method.

[0013] The disclosure also includes a computer or computing system (including networked or distributed systems) with an operating system that supports a computer program for carrying out the procedures described herein and / or for embodying the device or system features described herein.

[0014] The revelation also provides a computer-readable medium on which one or more of the aforementioned computer programs are stored.

[0015] The revelation also provides a signal that carries one or more of the aforementioned computer programs.

[0016] The disclosure extends to methods and / or devices and / or systems as described herein with reference to the accompanying drawings.

[0017] Aspects and embodiments of the disclosure will now be described purely by way of example with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The systems and procedures presented here for in-system testing of autonomous and semi-autonomous systems and applications are described in detail below with reference to the accompanying diagrams. These diagrams show: Fig. 1A an exemplary system configured to perform an in-system test according to one or more embodiments of the present disclosure; Fig.1B an exemplary linked list of control packages that may be included in a test image according to one or more embodiments of the present disclosure; Fig. 2 a flowchart illustrating a method for performing in-system tests according to one or more embodiments of the present disclosure; Fig. 3A an illustration of an exemplary autonomous vehicle, according to one or more embodiments of the present disclosure; Fig. 3B is an example of camera locations and fields of view for the exemplary autonomous vehicle from Fig. 3A, according to one or more embodiments of the present disclosure; Fig. 3C presents a block diagram of an exemplary system architecture for the exemplary autonomous vehicle. Fig. 3A, according to one or more embodiments of the present disclosure; Fig.3D system diagram for communication between one or more cloud-based servers and the example autonomous vehicle Fig. 3A, according to one or more embodiments of the present disclosure; Fig. 4 a block diagram of an exemplary computing device suitable for use in the implementation of one or more embodiments of the present disclosure; and Fig. 5 a block diagram of an exemplary data center suitable for use in the implementation of one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] The systems and methods disclosed herein may relate to in-system tests (ISTs) that can be used by machines. In general, ISTs can use test images for executing tests. The test images may contain information related to instructions for executing the corresponding tests, stimuli or other information that can be used to perform the tests, and / or the corresponding expected behavior of the computing system in response to the respective stimuli.

[0020] According to one or more embodiments of the present disclosure, system architectures and methods can be configured to provide the use of a master test image for multiple different tests. In contrast, current IST architectures and techniques typically require specific and individual test images for each test to be performed.

[0021] The ability to use a master test image, instead of having to use different test images for different tests, can allow for greater testing flexibility by simplifying the elements required for multiple tests. Furthermore, using a single master test image that can replace several different test images can reduce the resources used for IST (Integrated Testing), such as the amount of memory used to store test images.

[0022] One or more embodiments of the present disclosure may relate to ISTs that can be associated with Ego-machines and / or components of one or more Ego-machines, which may include any applicable machine or system capable of performing one or more autonomous or semi-autonomous operations. Exemplary Ego-machines may, without limitation, include vehicles (land, sea, space, and / or air vehicles), robots, robotic platforms, etc. For example, the computing applications of the Ego-machine may include one or more applications that can be executed by an autonomous or semi-autonomous vehicle, such as an exemplary Autonomous Vehicle 300 (hereafter referred to alternatively as "Vehicle 300" or "Ego-Machine 300"), as described in relation to Fig.Described in Figures 3A-3D. In the present disclosure, the reference to an “autonomous vehicle” or a “semi-autonomous vehicle” can include any vehicle that can be configured to perform one or more autonomous or semi-autonomous navigation or driving operations. Therefore, such vehicles can also include vehicles in which an operator is required or in which an operator can also perform such operations.

[0023] Additionally or alternatively, the systems and procedures described herein may be used without restriction by non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more adaptive driver assistance systems (ADAS)), autonomous vehicles or machines, steered and unsteered robots or robotic platforms, warehouse vehicles, off-road vehicles, vehicles coupled to one or more trailers, hydrofoils, boats, shuttles, emergency vehicles, motorcycles, electric or motorized bicycles, aircraft, construction vehicles, underwater vehicles, drones, and / or other types of vehicles.Furthermore, the systems and methods described herein can be used for a variety of purposes, including but not limited to machine control, machine locomotion, machine propulsion, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twinning, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twinning, generative AI, data center processing, conversational AI (such as by employing one or more language models, such as one or more large language models (LLMs)), light transport simulations (such as ray tracing, path tracing, etc.), collaborative content creation for 3D assets, cloud computing, and / or other suitable applications.

[0024] The disclosed embodiments can include a variety of different systems, such as automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine), systems implemented with a robot, aviation systems, media systems, boat systems, intelligent area surveillance systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using an edge device, systems incorporating one or more virtual machines (VMs), systems for performing operations to generate synthetic data, systems implemented at least partially in a data center, systems for performing conversational AI operations (e.g.,Systems implementing one or more LLMs, systems implementing one or more Vision Language Models (VLMs), systems implementing one or more multimodal language models, systems for performing one or more operations with generative AI, systems for hosting real-time streaming applications, systems for presenting one or more virtual reality, augmented reality, or mixed reality content, systems for performing light transport simulations, systems for performing collaborative content creation for 3D assets, systems that are implemented at least partially using cloud computing resources, and / or other types of systems.

[0025] The embodiments of the present disclosure are explained with reference to the accompanying figures. It is understood that the figures are diagrammatic and schematic representations of such exemplary embodiments and are neither limiting in nature nor necessarily drawn to scale. In the figures, features with the same reference numerals denote the same structure and function, unless otherwise described.

[0026] In relation to Fig. 1A illustrates Fig.1A An exemplary system 100 configured to perform an in-system test according to one or more embodiments of this disclosure. In general, the system 100 may include a memory 102, a test module 110, a test controller 104, a register bank 106, and one or more test objects 108 (“test units 108”). It is understood that these and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, arrays, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components, in any suitable combination and at any suitable location.Various functions described herein as being performed by entities can be executed using hardware, firmware, and / or software. For example, various functions can be performed using a processor that executes instructions stored in memory.

[0027] The test units 108 can contain one or more elements of a computing system that can be tested using IST. For example, one or more of the test units 108 can individually contain hardware and / or software components configured to perform one or more tasks or operations and / or configured to contribute to or facilitate the performance of one or more tasks or operations by the computing system.

[0028] For example, the test units 108 can contain one or more processing devices, storage devices, data storage devices, communication devices, software modules, etc., wherein a single or a group of one or more test units 108 can be used in the execution of operations by the computing system. In some embodiments, the test units 108 can perform operations in various environments, which may include, but are not limited to, autonomous vehicles or machines, vehicle or machine performance, and / or vehicle or machine safety.

[0029] The working memory 102 can contain any suitable computer-readable storage medium for transmitting or containing computer-executable instructions or data structures stored thereon. Such computer-readable storage media can include any available media to which the controller 104 can access. For example, and without limitation, such computer-readable storage media can include physical or non-volatile computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash storage devices (e.g.,Solid-state storage devices) or any other storage medium that can be used to store specific program code in the form of computer-executable instructions or data structures, and which can be accessed by a general-purpose or special-purpose computer. Combinations of the above elements may be included in the scope of protection of computer-readable storage media.

[0030] Additionally or alternatively, in some embodiments, the memory 102 can be part of or comprise a multimedia card (MMC) that may contain flash memory (e.g., NAND flash memory) and a corresponding memory controller. In these and other embodiments, the MMC may be configured as an embedded MMC (eMMC). Additionally or alternatively, the memory 102 can be off-chip memory, in which the memory 102 may be separate from a chip that may contain the test module 110, the test controller 104, the register bank 106, and / or the test units 108.

[0031] In some embodiments, the memory 102 can contain a master test image 112 ("image 112") stored therein. Generally, image 112 can contain information that can be used to perform multiple tests and evaluate the results of such tests. Additionally or alternatively, image 112 can be configured to allow the execution of several different types of tests. For example, in some embodiments, image 112 can contain information that enables the execution of an embedded core test, a memory test, a scan test (also called a "logic test"), or any other suitable test. In these and other embodiments, the tests can be implemented according to various protocols or guidelines, such as a Joint Test Action Group (JTAG)-based test or a test based on the IEEE 1500 standard.

[0032] In these and other embodiments, image 112 can contain control packages, data packages, and / or status packages that correspond to the execution of tests. The control packages can contain instructions for decoding and executing the test. Additionally or alternatively, the control packages in test image 112 can be arranged sequentially, for example, in the form of a linked list. In these and other embodiments, the sequential order of the control packages can specify a default order for the execution of the control packages during test execution.

[0033] Fig.Figure 1B illustrates, for example, a sample linked list 150 of "n" control packages that may be included in Figure 112. In the illustrated example, the linked list 150 can specify the default execution order of the control packages it contains. For example, the linked list 150 can specify that the default execution order is such that a first control package (Strpkt-1) is executed first, followed by a second control package (Strpkt-2), then a third control package (Strpkt-3), and so on, until the nth control package (Strpkt-n) has been executed.

[0034] The data packets can contain inputs and / or stimuli that can be used when performing tests. For example, in some embodiments, one or more data packets can contain test vectors (e.g., scan test vectors) and the data for programming registers to control the clock generators, resetters, Dynamic Function eXchange (DFX) controls, and / or pad controls corresponding to test units 108.

[0035] The result packets can contain the behavior of the test units 108 during the test, as performed based on the control packets and / or the data packets. In some embodiments, for example, the result packets contain a scan test response or a memory test response.

[0036] In these and other embodiments, the status packages can contain a summary of the corresponding test execution. For example, in some embodiments, one or more status packages can indicate whether the test units 108 passed or failed a test, errors that may have been identified during a test, etc.

[0037] The register bank 106 can contain any suitable working memory that the controller 104 can access. In some embodiments, for example, the register bank 106 can be part of a working memory that is separate from the controller 104. Additionally or alternatively, the register bank 106 can be a set of registers contained within the controller 104. Although the term "register" is used, it should further be understood that any suitable working memory that can be used in the manner described in the present disclosure is included within the scope of protection of the present disclosure.

[0038] In some embodiments, the register bank 106 can be loaded with one or more test configurations 114. The test configurations 114 can each correspond to different tests that can be performed on one or more of the test units 108. In some embodiments, a single test configuration 114 can correspond to a single test. Additionally or alternatively, a single test configuration 114 can correspond to a part of a specific test. In these and other embodiments, a single test configuration 114 can correspond to multiple tests.

[0039] In some embodiments, the test configurations 114 can each correspond to sets of one or more control packages shown in Figure 112, which can be used or executed with respect to tests that each correspond to the test configurations 114. A single test configuration 114 can specify an execution order of a particular set of control packages, where the execution order of the control packages included in the set of control packages differs from the standard execution order of such control packages, as shown in Figure 112.

[0040] The difference in the execution sequences specified by the test configurations 114 may include one or more differences in the standard execution sequence shown in Figure 112. In general, in this disclosure, any deviation from the execution of all control packages in Figure 112 according to the standard execution sequence may be considered a “difference” from the standard execution sequence. For example, a difference in the execution sequence may include that only a subset of the control packages included in Figure 112 is executed even in cases where the subset of control packages that is executed is executed in the standard execution sequence. Another exemplary difference in the execution sequence may include that the execution of a particular control package is repeated sequentially or at a later time.In these and other embodiments, an exemplary difference in the execution sequence may include skipping the execution of control packages compared to the standard execution sequence and / or jumping back to the execution of a control package that is later in the standard execution sequence after executing it, etc.

[0041] It should be noted that in some embodiments, and as explained in more detail in the present disclosure, the tests corresponding to test configurations 114 may involve the execution of more control packages than those included in the sets of control packages contained in test configurations 114. In these and other embodiments, the control packages belonging to such tests that are not included in the corresponding set of control packages may be executed according to the standard execution sequence shown in Figure 112.

[0042] In these and other embodiments, the register bank 106 can contain information corresponding to the individual test configurations 114 loaded onto it. This information can specify the execution orders of the control packages for the tests to which the test configurations 114 correspond. For example, individual registers of the register bank 106 can each contain a first and a second field, each populated with values ​​to specify individual execution orders of one or more control packages corresponding to individual tests. Additionally or alternatively, the number of registers with populated fields can vary depending on the number of variations from the default test execution order.

[0043] In some cases, the first field can be populated with an initial entry indicating a deviation point during test execution, specifying a point at which a subsequent deviation from the standard execution sequence is permissible. For example, the first field might contain the first specific control package of image 112. Including this first specific control package in the first entry can indicate that a deviation from the standard execution sequence may occur after the execution of this first specific control package. Additionally or alternatively, the first field can contain the entry "Initialization" or "Start," indicating that a deviation from the standard execution sequence may occur immediately after the execution of the corresponding test begins.

[0044] Additionally or alternatively, the second field can be filled with a second entry specifying a next step to be performed at the deviation point indicated by the corresponding first entry. In some cases, for example, the second field can contain a second specific control package from Figure 112. Including the second specific control package in the second entry can indicate that the second specific control package is to be executed at the deviation point corresponding to the first field (e.g., after the first specific control package in cases where the first specific control package can be the first value in the first field). In these and other embodiments, the second field can contain an entry "Finish" or "Stop" indicating that the corresponding test is to be aborted at the deviation point.

[0045] For example, with reference to the linked list 150 of Fig.As an example, in 1B, a specific test configuration 114, corresponding to a specific test, is loaded into register bank 106. In this and other embodiments, the specific test may consist of first executing the first control package (Strpkt-1), followed by executing the second control package (Strpkt-2), followed by executing the fourth control package (Strpkt-4) twice in succession, followed by executing the seventh control package (Strpkt-7), followed by executing the third control package (Strpkt-3), and then aborting the specific test. For illustration, the specific test in expression (1) can be specified as follows: START→Strpkt−1→Strpkt−2→Strpkt−4→Strpkt−4→Strpkt−5→Strpkt−7→Strpkt−3→STOP

[0046] Therefore, the respective test can deviate in various ways from the standard execution sequence of Figure 112, such as jumping from the second control package to the fourth control package, executing the fourth control package twice in succession, jumping from the execution of the fifth control package to the seventh control package, jumping from the seventh control package back to the third control package, and aborting the respective test after the execution of the third control package without executing all control packages contained in Figure 112.

[0047] In some embodiments, the registers of register bank 106 can be loaded as shown in Table 1 below to reproduce the specific test configuration 114 that corresponds to the specific test specified by expression (1). Table 1 register First field Second field Register-1 Strpkt-2 Strpkt-4 Register-2 Strpkt-4 Strpkt-4 Register-3 Strpkt-5 Strpkt-7 Register-4 Strpkt-7 Strpkt-3 Register-5 Strpkt-3 STOP

[0048] As described above, the first deviation point of the specific test from the standard execution sequence, shown in Figure 112, can occur after the execution of the second control package (Strpkt-2). Furthermore, the deviation from the standard execution sequence can jump to the execution of the fourth control package (Strpkt-4) after the execution of the second control package.

[0049] As shown in Table 1, a first register (Register 1) of register bank 106 can be populated with entries to reflect changes in the execution order. For example, the first register can contain a first field with a first entry specifying the second tax package, indicating that the first deviation point occurs after the execution of the second tax package. Additionally or alternatively, the first register can also contain a second field with a second entry specifying the fourth tax package. The second entry can indicate that the specific test should jump to the execution of the fourth tax package. Therefore, the first and second entries, respectively, in the first and second fields of the first register, can indicate that the fourth tax package is to be executed after the execution of the second tax package, which differs from the exemplary default execution order of the one shown in Fig.Figure 112 illustrates this. Registers two to five contain entries in the same manner to indicate the other deviation points and the corresponding types of deviations that are part of the specific test specified by expression 1.

[0050] Register bank 106 can contain more registers than those illustrated in Table 1. Table 1 serves only to illustrate an example of registers that can be loaded with regard to the specific test.

[0051] The controller 104 can be communicatively coupled with the register bank 106 and the main memory 102. In general, the controller 104 can be configured to perform tests on one or more of the test units 108 based on the image 112 and based on corresponding test configurations 114 that have been loaded into the register bank 106.

[0052] In some embodiments, the controller 104 may contain code and routines configured to initiate the operations described with respect to the controller 104. Additionally or alternatively, the controller 104 may be implemented using hardware that incorporates one or more processors, CPUs, graphics processing units (GPUs), data processing units (DPUs), parallel processing units (PPUs), microprocessors (e.g., to perform or control the execution of one or more operations), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), accelerators (e.g.,Deep learning accelerators (DLAs), one or more programmable vision accelerators (PVAs), which may contain one or more vector processing units (VPUs), one or more direct memory access (DMA) systems, one or more pixel processing engines (PPEs), etc., and / or other types of processors. In these and other embodiments, the controller 104 can be implemented using a combination of hardware and software. In the present disclosure, operations described as being performed by the controller 104 may include operations in which the controller 104 can instruct a corresponding computing system to perform them.In these or other embodiments, the controller 104 can be implemented by one or more computing devices, as described in relation to . Fig. 3A-3D, 4 and / or 5 are described in more detail.

[0053] In some embodiments, the controller 104 may be configured to execute control packages for tests by following the default execution sequence shown in Figure 112, unless otherwise specified by entries in register bank 106 corresponding to the respective test configurations 114 of the respective tests.

[0054] With reference to the specific test of expression (1) and Table 1, in some embodiments, for example, the controller 104 may, during the initialization of the specific test, load the information from register 1 of register bank 106 into an execution register 116. The execution register 116 may contain any memory that can be used to temporarily store information to which the controller 104 can refer as part of the current operations being performed by the controller 104. In some embodiments, the execution register 116 may be separate from register bank 106. Additionally or alternatively, the execution register 116 may be part of register bank 106. In these or other embodiments, register 1 may be used as the execution register 116, so that the loading of the execution register 116 from register 1 may occur when register 1 is loaded.

[0055] Additionally or alternatively, the entries in register bank 106 can be shifted up one register after the execution register 116 is loaded. Table 2 illustrates an example of the register entries in register bank 106 and execution register 116 after the execution register 116 is loaded during the initialization of the specific test, where the entries have been shifted up one register compared to Table 1. The example in Table 2 refers to an embodiment where execution register 116 is separate from register 1. However, it is understood that such specific implementation details are used to illustrate the general principles described herein and are not intended to be limiting. Furthermore, it should be noted that register 5 is not illustrated in Table 2.In some cases, register 5 may be empty after execution register 116 has been loaded. Additionally or alternatively, in some cases, register 5 may be loaded with other information associated with a different test configuration corresponding to a different test. Table 2 register First field Second field Execution Register Strpkt-2 Strpkt-4 Register-1 Strpkt-4 Strpkt-4 Register-2 Strpkt-5 Strpkt-7 Register-3 Strpkt-7 Strpkt-3 Register-4 Strpkt-3 STOP

[0056] Controller 104 can use the information in execution register 116 to determine how to proceed with a particular test. For example, the default execution order specifies that the first control package should be executed first. Controller 104 can first check the first entry of the first field of execution register 116 to determine if a deviation point occurs at the start of execution of the particular test. For example, Controller 104 can determine whether to execute a different control package than the first one based on whether the first field contains an entry labeled "Initialization" or "Start," indicating that the first deviation point occurs during initialization.In the illustrated example, the first field of execution register 116 indicates a deviation point after the execution of the second control package and not at the initialization of the specific test, so that controller 104 can begin the specific test by executing the first control package, as specified in the standard execution order.

[0057] After the first stage (e.g., after the execution of the first control package) and before the execution of the second stage of the specific test, controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the execution of the first control package. In the illustrated example in Table 2, the first entry in the first field of execution register 116 can still indicate that the deviation point is after the execution of the second control package, rather than after the execution of the first. Therefore, controller 104 can be configured to execute the second stage of the specific test by executing the second control package according to the specifications in the standard execution sequence.

[0058] After the second stage (e.g., after the execution of the second control package) and before the execution of the third stage of the specified test, controller 104 can again check execution register 116 to determine whether a deviation from the standard execution order should occur after the execution of the second control package. As explained earlier, in the illustrated example in Table 2, the first entry in the first field of execution register 116 may still indicate that the deviation point is after the execution of the second control package. Therefore, based on the just-completed execution of the second control package and the first field of execution register 116 indicating the second control package, controller 104 can be configured to identify that the next control package to be executed may differ from the standard execution order.In these and other embodiments, the controller 104 can read the second entry of the second field of the execution register 116 to determine which control package should be executed next. In the illustrated example, the controller 104 can execute the third stage of the specified test by executing the fourth control package, since the fourth control package is specified in the second field of the execution register 116.

[0059] In these and other embodiments, the entries of execution register 116 can be updated at a specific time before the completion of the third stage execution with the entries of register 1 of register bank 106, and the other register bank entries can shift and move up one register. Table 3 illustrates an example of the register entries of register bank 106 and execution register 116 after the update of execution register 116 with respect to the third execution stage. It should also be noted that register 4 in Table 3 is shown as empty due to the shifting of the register entries. Additionally or alternatively, in some cases, register 4 can be loaded with other information associated with a different test configuration corresponding to a different test. Table 3 Register First field Second field Execution Register Strpkt-4 Strpkt-4 Register-1 Strpkt-5 Strpkt-7 Register-2 Strpkt-7 Strpkt-3 Register-3 Strpkt-3 STOP Register-4

[0060] After the third stage (e.g., after the execution of the fourth control package) and before the execution of the fourth stage of the specified test, controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the execution of the fourth control package. In the illustrated example in Table 3, the first entry in the first field of execution register 116 can indicate that a deviation point occurs after the execution of the fourth control package. Therefore, because the fourth control package has just completed execution, and based on the first field of execution register 116, which specifies the fourth control package, controller 104 can be configured to identify that the next control package to be executed can again be the fourth control package, based on the fourth control package specification in the second field of execution register 116.In the illustrated example, controller 104 can therefore execute the fourth stage of the specific test by repeating the execution of the fourth control package.

[0061] In these and other embodiments, the entries of execution register 116 can be updated at a specific time before the completion of the fourth stage execution with the entries of register 1 of register bank 106, and the other register bank entries can shift and move up one register. Table 4 illustrates an example of the register entries of register bank 106 and execution register 116 after the update of execution register 116 with respect to the fourth execution stage. It should also be noted that registers 3 and 4 in Table 4 are shown as empty due to the shifting of the register entries. Additionally or alternatively, in some cases, one or more of registers 3 or 4 can be loaded with other information associated with a different test configuration corresponding to a different test. Table 4 register First field Second field Execution Register Strpkt-5 Strpkt-7 Register-1 Strpkt-7 Strpkt-3 Register-2 Strpkt-3 STOP Register-3 Register-4

[0062] After the fourth stage (e.g., after the fourth control package has been executed for the second time) and before the fifth stage of the specified test, controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the fourth control package has been executed. In the illustrated example in Table 4, the first entry in the first field of execution register 116 can indicate that a deviation point occurs after the fifth control package has been executed. Therefore, because the fourth control package has just been executed, controller 104 can revert to the standard execution sequence specified in the linked list 150 of Figure 112. The standard execution sequence can specify that the fifth control package is to be executed after the fourth.In the illustrated example, controller 104 can therefore execute the fifth stage of the specific test by executing the fifth control package.

[0063] After the fifth stage (e.g., after the execution of the fifth control package) and before the execution of the sixth stage of the specified test, Controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the execution of the fifth control package. In the illustrated example in Table 4, the first entry in the first field of execution register 116 can indicate that a deviation point occurs after the execution of the fifth control package. Therefore, because the fifth control package has just completed execution, Controller 104 can refer to the second entry in the second field of execution register 116. Controller 104 can accordingly determine that the next control package to be executed can be the seventh control package.In the illustrated example, the controller 104 can therefore execute the sixth stage of the specific test by executing the seventh control package.

[0064] In these and other embodiments, the entries of execution register 116 can be updated at a specific time before the completion of the sixth stage execution with the entries of register 1 of register bank 106, and the other register bank entries can shift and move up one register. Table 5 illustrates an example of the register entries of register bank 106 and execution register 116 after the update of execution register 116 with respect to the sixth execution stage. In the illustrated example in Table 5, registers 2, 3, and 4 are empty due to the shifting of the register entries. Additionally or alternatively, in some cases, one or more of registers 2, 3, or 4 can be loaded with other information associated with a different test configuration corresponding to a different test. Table 5 register First field Second field Execution Register Strpkt-7 Strpkt-3 Register-1 Strpkt-3 STOP Register-2 Register-3 Register-4

[0065] After the sixth stage (e.g., after the execution of the seventh control package) and before the execution of the seventh stage of the specified test, Controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the execution of the seventh control package. In the illustrated example in Table 5, the first entry in the first field of execution register 116 can indicate that a deviation point occurs after the execution of the seventh control package. Therefore, because the seventh control package has just completed execution, Controller 104 can refer to the second entry in the second field of execution register 116. Controller 104 can then determine that the next control package to be executed can be the third control package.In the illustrated example, the controller 104 can therefore execute the seventh stage of the specific test by executing the third control package.

[0066] In these and other embodiments, the entries of execution register 116 can be updated at a specific time before the completion of the seventh stage execution with the entries of register 1 of register bank 106, and the other register bank entries can shift and move up one register. Table 6 illustrates an example of the register entries of register bank 106 and execution register 116 after the update of execution register 116 with respect to the seventh execution stage. In the illustrated example in Table 6, registers 1, 2, 3, and 4 of register bank 106 are empty due to the shifting of the register entries. Additionally or alternatively, in some cases, one or more of registers 1, 2, 3, or 4 can be loaded with other information associated with a different test configuration corresponding to a different test. Table 6 register First field Second field Execution Register Strpkt-3 STOP Register-1 Register-2 Register-3 Register-4

[0067] After the seventh stage (e.g., after the execution of the third control package) and before the execution of the seventh stage of the specific test, controller 104 can again check execution register 116 to determine whether a deviation from the standard execution sequence should occur after the execution of the seventh control package. In the illustrated example in Table 6, the first entry in the first field of execution register 116 can indicate that a deviation point occurs after the execution of the third control package. Therefore, because the third control package has just completed its execution, controller 104 can refer to the second entry in the second field of execution register 116. Controller 104 can then determine that the test should be stopped based on the "STOP" entry in the second field.In the illustrated example, controller 104 may no longer execute any control packets and can therefore terminate the specific test.

[0068] The test module 110 can contain code and routines configured to cause the controller 104 to perform one or more test-related operations as described herein. Additionally or alternatively, the test module 110 can be implemented using hardware that incorporates one or more processors, CPUs, graphics processing units (GPUs), data processing units (DPUs), parallel processing units (PPUs), microprocessors (e.g., to perform or control the execution of one or more operations), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), accelerators (e.g., deep learning accelerators (DLAs)), one or more programmable vision accelerators (PVAs), which may contain one or more vector processing units (VPUs), one or more direct memory access (DMA) systems, one or more pixel processing engines (PPEs), etc., and / or other types of processors. In these and other embodiments, the test module 110 can be implemented using a combination of hardware and software. In the present disclosure, operations described as being performed by the test module 110 may include operations in which the test module 110 can instruct a corresponding computing system to perform them. In these or other embodiments, the test module 110 can be implemented by one or more computing devices as described in relation to . Fig. 3A-3D, 4 and / or 5 are described in more detail.

[0069] In general, the test module 110 can be configured to instruct the controller 104 to manage tests of one or more of the test units 108. For example, the test module 110 can be configured to load the register bank 106 with certain test configurations 114. In these and other embodiments, the test module 110 can be configured to receive the test configurations 114 as input. Additionally or alternatively, the test module 110 can be configured to generate one or more of the test configurations 114 based on received input (e.g., received user input regarding certain parameters to be tested for a specific test, and / or regarding which control packages are to be executed and in what order for each test).

[0070] In these and other embodiments, the test module 110 can be configured to instruct the controller 104 when to start executing tests. Additionally or alternatively, the main memory 102 can contain several different test images 112 stored therein. In these and other embodiments, the test module 110 can specify which test image 112 the controller can use for particular tests.

[0071] Modifications, additions or omissions may be made to Fig. 1A and Fig.1B and the associated descriptions may be made without departing from the scope of protection of this disclosure. In some embodiments, for example, the system 100 may contain any number of test specimens. Alternatively or additionally, some embodiments may contain implementations that differ from those described. In some embodiments, for example, the entries in the first fields of the registers may indicate whether the deviation point occurs before the control package specified in the respective entries and not, as described, after it. Alternatively or additionally, in some embodiments, the system 100 may contain any number of other components, actions, or inputs that may not be explicitly shown or described.Furthermore, although the above description focuses mainly on the execution of control packages with respect to the performance of tests, many other operations can be performed with respect to such tests (e.g., as specified by the control packages) which are not described herein.

[0072] Furthermore, although the description and explicit examples refer to loading a single test configuration 114 into register 106, in some cases multiple test configurations 114 can be loaded into register 106 simultaneously. Additionally or alternatively, different test configurations 114 can be loaded into register 106 in sequential order.

[0073] Furthermore, in some cases, the register bank 106 may not be large enough to specify all the different deviations from the standard execution order associated with a particular test. In some embodiments, multiple test configurations 114 or subtest configurations of a larger test configuration 114 corresponding to the same test can be generated and loaded sequentially into the register bank 106. In these and other embodiments, the next test configuration 114 (or next subtest configuration) after the current test configuration 114 (or current subtest configuration) that is currently in use can be loaded piecemeal as registers of the register bank 106 become available.Additionally or alternatively, the next test configuration 114 (or the next subtest configuration) can be loaded into register 106 after register 106 has been cleared of all entries corresponding to the previous test configuration 114 (or the previous subtest configuration).

[0074] Furthermore, the number of control packages executed during the tests, or which control packages can be executed during the tests, can vary. For example, in some cases, all control packages included in Figure 112 can be executed during a particular test. Additionally or alternatively, in some cases, only a subset of the control packages included in Figure 112 can be executed during a particular test.

[0075] In these and other embodiments, the control packages that may be included in the sets of control packages corresponding to the different test configurations can vary. For example, in some cases, a particular set of control packages corresponding to a specific test configuration 114 may contain all the control packages that are executed during a particular test (e.g., in cases where none of the control packages are executed according to the standard execution sequence). Additionally or alternatively, a particular set of control packages corresponding to a particular test configuration 114 may contain a subset of the control packages that are executed during a particular test (e.g., in cases where at least some of the control packages are executed according to the standard execution sequence).

[0076] Fig.Figure 2 is a flowchart illustrating a method 200 for performing in-system tests according to one or more embodiments of the present disclosure. One or more operations of the method 200 can be performed by any suitable system, device, or apparatus, such as one or more components of the system 100. Fig. 1 (e.g., the test controller 104), of one or more with respect to Fig. 3A-3D described autonomous vehicle systems, one or more of which in relation to Fig. 4 described computing devices and / or of one or more in relation to Fig. 5 data systems described in the present disclosure.

[0077] Method 200 may contain a block B202 that provides access to a test image used to test a computing system. In some embodiments, the test image may specify a default execution order for control packages used to perform a test of the computing system. For example, the default execution order may be based at least on a sequential order of the control packages as specified by a linked list of the control packages contained in the test image, such as in Fig. 1B of the present disclosure. Test pattern 112, which relates to Fig. 1A and Fig. The example described in Figure 1B can be a test image that can be accessed. In some embodiments, a test controller (e.g., the test controller 104 from [reference missing]) can be used. Fig. 1A) access the test image.

[0078] Additionally or alternatively, in some embodiments, the test image can be stored in a working memory and accessed from that memory located remotely from the test controller (e.g., on a chip separate from the chip containing the test controller). In these and other embodiments, the test image can be stored in a working memory and accessed from that memory located close to the test controller (e.g., on the same chip as the test controller).

[0079] Block B204 provides access to a test configuration corresponding to the test of the computer system. The test configuration can correspond to a set of control packages from among those contained in the test image. In some embodiments, the set of control packages can include all control packages in the test image. Additionally or alternatively, the set of control packages can contain only a subset of all control packages in the test image. The test configuration can specify an execution order for the set of control packages that differs from the default execution order. For example, the order in which the control packages are executed can differ from the default order. Fig. 1A and Fig. The test configuration described in 1B, 114, is an example of a test configuration.

[0080] In some embodiments, the test configuration can be accessed from a memory location that is the same as the memory location from which the test image can be accessed. Additionally or alternatively, in some embodiments, the test configuration can be accessed from a memory location that differs from the memory location from which the test image can be accessed. For example, in some embodiments, the test configuration can be located in a register bank, such as register bank 106, which is configured with respect to Fig. 1A and Fig. The register bank described in 1B is stored and can be accessed from there. In these and other embodiments, the register bank can be an internal register bank of the test controller. Additionally or alternatively, the register bank can also be external to the test controller.

[0081] In block B206, the test of the computing system can be performed at least based on the test image and the test configuration. In some embodiments, the test can be performed, for example, as described in the present disclosure, with respect to Fig. 1A and Fig. 1B and Tables 1-6. Therefore, Method 200 can be used to perform one or more ISTs according to one or more embodiments of the present disclosure.

[0082] Modifications, additions, or omissions may be made to Method 200 without altering the scope of protection of the present disclosure. For example, various blocks of Method 200, although presented as individual blocks, may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the specific implementations.

[0083] Furthermore, in some embodiments, method 200 can be used to perform several different tests based on the same test image and several different test configurations. In some embodiments, for example, a first test with respect to the computer system can be performed based on the test image and a first test configuration. Additionally or alternatively, a second test with respect to the computer system can be performed based on the test image and a second test configuration. EXEMPLARY AUTONOMOUS VEHICLE

[0084] Fig.Figure 3A is an illustration of an exemplary autonomous vehicle 300, according to some embodiments of the present disclosure. The autonomous vehicle 300 (here alternatively referred to as "vehicle 300") may, without limitation, include: a passenger vehicle, such as a car, truck, bus, emergency service vehicle, shuttle, electric or motorized bicycle, motorcycle, fire engine, police vehicle, ambulance, boat, construction vehicle, underwater vehicle, drone, and / or another type of vehicle (e.g., one that is unmanned and / or carries one or more passengers).Autonomous vehicles are generally described in terms of automation levels defined by the National Highway Traffic Safety Administration (NHTSA), a division of the U.S. Department of Transportation, and the Society of Automotive Engineers (SAE) standard "Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles" (Standard No. J3016-201806, published June 15, 2018, Standard No. J3016-201609, published September 30, 2016, and earlier and future versions of this standard). The Vehicle 300 can exhibit functionality corresponding to one or more of the Levels 3 through 5 of autonomous driving levels.The Vehicle 300 can exhibit functionality corresponding to one or more of the Levels 1 to 5 of autonomous driving. For example, depending on its configuration, the Vehicle 300 may be capable of driver assistance (Level 1), partial automation (Level 2), conditional automation (Level 3), high automation (Level 4), and / or full automation (Level 5). The term "autonomous," as used here, may encompass any and / or all types of autonomy for the Vehicle 300 or any other machine, such as fully autonomous, highly autonomous, conditionally autonomous, partially autonomous, assistive autonomy, semi-autonomous, primary autonomous, or any other designation.

[0085] The vehicle 300 can include components such as a chassis, a vehicle body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other vehicle components. The vehicle 300 can include a drive system 350, such as an internal combustion engine, a hybrid electric power plant, a pure electric motor, and / or another type of drive. The drive system 350 can be connected to a drivetrain of the vehicle 300, which may include a transmission to enable the propulsion of the vehicle 300. The drive system 350 can be controlled in response to signals received from the throttle valve or accelerator device 352.

[0086] A steering system 354, which may include a steering wheel, can be used to steer the vehicle 300 (e.g., along a desired path or route) when the drive system 350 is in operation (e.g., when the vehicle is in motion). The steering system 354 can receive signals from a steering actuator 356. The steering wheel can be optional for full automation (level 5).

[0087] The brake sensor system 346 can be used to actuate the vehicle brakes in response to receiving signals from the brake actuators 348 and / or the brake sensors.

[0088] The one or more controllers 336, the one or more CPUs, systems-on-chips (SoCs) 304 ( Fig.3C) and / or GPUs, can provide signals (e.g., representing instructions) to one or more components and / or systems of the vehicle 300. For example, the one or more controllers can send signals to actuate the vehicle brakes via one or more brake actuators 348, to actuate the steering system 354 via one or more steering actuators 356, and / or to actuate the propulsion system 350 via one or more throttle / accelerator devices 352. The one or more controllers 336 can include one or more built-in (e.g., integrated) computing devices (e.g., supercomputers) that process sensor signals and issue operating commands (e.g., signals representing commands) to enable autonomous driving and / or to assist a human driver in driving the vehicle 300.The one or more Controller 336s can include a first Controller 336 for autonomous driving functions, a second Controller 336 for functional safety functions, a third Controller 336 for artificial intelligence functions (e.g., computer vision), a fourth Controller 336 for infotainment functions, a fifth Controller 336 for emergency redundancy, and / or other controllers. In some examples, a single Controller 336 can perform two or more of the above-mentioned functionalities, two or more Controller 336s can perform a single functionality, and / or any combination thereof.

[0089] The one or more controllers 336 can provide the signals for controlling one or more components and / or systems of the vehicle 300 in response to sensor data received from one or more sensors (e.g., sensor inputs). The sensor data can be received, for example, without limitation, from one or more of the following: one or more global navigation satellite system sensors 358 (e.g., one or more global positioning system sensors), one or more radar sensors 360, one or more ultrasonic sensors 362, one or more lidar sensors 364, one or more inertial measurement unit (IMU) sensors 366 (e.g., one or more accelerometers, gyroscopes, magnetic compasses, magnetometers, etc.), one or more microphones 396, one or more stereo cameras 368, one or more wide-angle cameras 370 (e.g.,fisheye cameras), one or more infrared cameras 372, one or more surround view cameras 374 (e.g. 360-degree cameras), one or more long-range and / or medium-range cameras 398, one or more speed sensors 344 (e.g. for measuring the speed of the vehicle 300), one or more vibration sensors 342, one or more steering sensors 340, one or more brake sensors (e.g. as part of the brake sensor system 346), and / or other sensor types.

[0090] One or more of the controllers 336 can receive inputs (e.g., in the form of input data) from an instrument cluster 332 of the vehicle 300 and provide outputs (e.g., in the form of output data, display data, etc.) via a human-machine interface (HMI) display 334, an acoustic alarm, a loudspeaker, and / or via other components of the vehicle 300. The outputs can include information such as vehicle speed, engine speed, time, map data (e.g., the HD map 322 of Fig.3C), location data (e.g., the location of vehicle 300, e.g., on a map), direction, location of other vehicles (e.g., an occupancy grid), information about objects and the status of objects as perceived by the one or more controllers 336, etc. For example, the HMI display 334 can show information about the presence of one or more objects (e.g., a road sign, a warning sign, a changing traffic signal, etc.) and / or information about driving maneuvers that the vehicle has performed, is currently performing, or will perform (e.g., changing lanes now, taking exit 34B in two miles, etc.).

[0091] The vehicle 300 also includes a network interface 324, which can use one or more wireless antennas 326 and / or modems for communication over one or more networks. The network interface 324 can be suitable, for example, for communication via LTE, WCDMA, UMTS, GSM, CDMA2000, etc. The one or more wireless antennas 326 can also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.) using local area networks such as Bluetooth, Bluetooth LE, Z-Wave, ZigBee, etc., and / or low-power wide area networks (LPWANs) such as LoRaWAN, SigFox, etc.

[0092] Fig. 3B is an example of camera locations and fields of view for the exemplary autonomous vehicle 300 from Fig. 3A, according to some embodiments of the present disclosure. The cameras and respective fields of view are an exemplary embodiment and are not intended to be limiting. For example, additional and / or alternative cameras may be included and / or the cameras may be located at different locations on the vehicle 300.

[0093] The camera types may include, but are not limited to, digital cameras designed for use with the components and / or systems of the vehicle 300. The one or more cameras may operate at Automotive Safety Integrity Level (ASIL) B and / or another ASIL. Depending on the configuration, the camera types may be capable of any frame rate, such as 60 frames per second (fps), 120 fps, 240 fps, etc. The cameras may use roller shutters, global shutters, another type of shutter, or a combination thereof.In some examples, the color filter array may include a red-clear-clear-clear (RCCC) color filter array, a red-clear-clear-blue (RCCB) color filter array, a red-blue-green (RBGC) color filter array, a Foveon X3 color filter array, a Bayer sensor color filter array (RGGB), a monochrome sensor color filter array, and / or another type of color filter array. In some embodiments, cameras with clear pixels, such as cameras with an RCCC, RCCB, and / or RBGC color filter array, may be used to increase light sensitivity.

[0094] In some examples, one or more of the cameras can be used to perform advanced driver assistance system (ADAS) functions (e.g., as part of a redundant or fail-safe design). For instance, a multi-function monocular camera can be installed to provide features including lane departure warning, traffic sign recognition, and intelligent headlight control. One or more of the cameras (e.g., all cameras) can simultaneously record and provide image data (e.g., video).

[0095] One or more cameras can be mounted in a bracket, such as a specially designed (3D-printed) one, to eliminate stray light and reflections from inside the vehicle (e.g., reflections of the dashboard in the windshield) that could interfere with the camera's image acquisition. Regarding the mounting of exterior mirrors, the mirrors themselves can be individually 3D-printed so that the camera mounting plate is shaped to fit the mirror. In some cases, the one or more cameras can be integrated into the exterior mirror. For side cameras, the one or more cameras can also be integrated into the four pillars at each corner of the cabin.

[0096] Cameras with a field of view that includes portions of the environment in front of the vehicle (e.g., forward-facing cameras) can be used for surround view to help identify forward paths and obstacles and, with the aid of one or more controllers and / or control SoCs, to provide information critical for creating an occupancy grid and / or determining preferred vehicle paths. Forward-facing cameras can be used to perform many of the same ADAS functions as LiDAR, including emergency braking, pedestrian detection, and collision avoidance. Forward-facing cameras can also be used for ADAS functions and systems that include lane departure warnings (LDW), autonomous cruise control (ACC), and / or other functions such as traffic sign recognition.

[0097] A variety of cameras can be used in a forward-facing configuration, including, for example, a monocular camera platform containing a CMOS (complementary metal oxide semiconductor) color imager. Another example is the 370 wide-angle camera, which can be used to capture objects entering the field of view from the periphery (e.g., pedestrians, crossing vehicles, or bicycles). Although in Fig. While Figure 3B illustrates only one wide-angle camera, the vehicle 300 can have any number of wide-angle cameras 370. Furthermore, one or more long-range cameras 398 (e.g., a pair of long-range stereo cameras) can be used for depth-based object detection, especially for objects for which a neural network has not yet been trained. The one or more long-range cameras 398 can also be used for object detection and classification, as well as basic object tracking.

[0098] One or more Stereo Cameras 368 can also be included in a forward-facing configuration. The one or more Stereo Cameras 368 contain an integrated control unit that includes a scalable processing unit capable of providing programmable logic (FPGA) and a multicore microprocessor with an integrated CAN or Ethernet interface on a single chip. Such a unit can be used to create a 3D map of the vehicle's surroundings, including a distance estimate for all points in the image. Alternatively, one or more Stereo Cameras 368 can include a compact stereo vision sensor that may contain two camera lenses (one left and one right) and an image processing chip. This chip can measure the distance between the vehicle and the target object and use the generated information (e.g., metadata) to activate the autonomous emergency braking and lane departure warning functions.Other types of stereo cameras 368 can be used in addition to or as an alternative to those described here.

[0099] Cameras with a field of view that includes sections of the environment to the sides of the vehicle 300 (e.g., side cameras) can be used for the surround view and provide information that is used to create and update the occupancy grid and to generate side-impact collision warnings. For example, one or more surround cameras 374 (e.g., four surround cameras 374, as in Fig. (Figure 3B illustrates) are positioned on the vehicle 300. The one or more surround-view cameras 374 can include one or more wide-angle cameras 370, one or more fisheye cameras, one or more 360-degree cameras, and / or the like. For example, four fisheye cameras can be mounted at the front, rear, and sides of the vehicle. In an alternative arrangement, the vehicle can use three surround-view cameras 374 (e.g., left, right, and rear) and one or more other cameras (e.g., a forward-facing camera) as a fourth surround-view camera.

[0100] Cameras with a field of view that includes sections of the area behind the vehicle 300 (e.g., reversing cameras) can be used for parking assistance, surround view, rear-impact warnings, and creating and updating the occupancy grid. A variety of cameras can be used, including cameras that are also suitable as one or more forward-facing cameras (e.g., one or more long-range and / or medium-range cameras 398, one or more stereo cameras 368, one or more infrared cameras 372, etc.), as described herein.

[0101] Fig. 3C is a block diagram of an exemplary system architecture for the exemplary autonomous vehicle 300 from Fig. 3A, according to some embodiments of the present disclosure. It is understood that these and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, arrays, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components, in any suitable combination and at any suitable location. Various functions described herein, which are performed by entities, may be executed by hardware, firmware, and / or software. Various functions may, for example, be performed by a processor executing instructions stored in memory.

[0102] Each of the components, features and systems of the 300 vehicle in Fig. 3C is illustrated as being connected via bus 302. Bus 302 may contain a Controller Area Network (CAN) data interface (here alternatively referred to as a "CAN bus"). A CAN can be a network within the vehicle 300 that serves to support the control of various features and functions of the vehicle 300, such as the operation of brakes, acceleration, braking, steering, windshield wipers, etc. A CAN bus may be configured to have dozens or even hundreds of nodes, each with its own unique identifier (e.g., a CAN ID). The CAN bus can be read to determine the steering wheel angle, vehicle speed, engine speed (rpm), button positions, and / or other vehicle status indicators. The CAN bus may be ASIL B compliant.

[0103] Although bus 302 is described here as a CAN bus, this is not intended as a limitation. For example, FlexRay and / or Ethernet can be used in addition to or as an alternative to the CAN bus. Furthermore, while a single line is used to represent bus 302, this is not meant as a restriction. There can be any number of buses 302, which may contain one or more CAN buses, one or more FlexRay buses, one or more Ethernet buses, and / or one or more other types of buses using a different protocol. In some examples, two or more buses 302 may be used to perform different functions and / or for redundancy. For example, a first bus 302 may be used for collision avoidance functionality, and a second bus 302 may be used for actuation control.In each example, each bus 302 can communicate with one of the vehicle 300's components, and two or more buses 302 can communicate with the same components. In some examples, each SoC 304, each controller 336, and / or each computer within the vehicle can have access to the same input data (e.g., inputs from vehicle 300 sensors) and be connected to a common bus, such as the CAN bus.

[0104] The vehicle 300 can contain one or more controllers 336, as described herein with reference to Fig. 3A are described. The one or more controllers 336 can be used for a variety of functions. The one or more controllers 336 can be coupled with one or more of the various other components and systems of the vehicle 300 and can be used for controlling the vehicle 300, for the artificial intelligence of the vehicle 300, for infotainment for the vehicle 300, and / or the like.

[0105] The vehicle 300 can contain one or more systems-on-a-chip (SoC) 304. The SoC 304 can contain one or more CPUs 306, one or more GPUs 308, one or more processors 310, one or more caches 312, one or more accelerators 314, one or more data storage devices 316, and / or other components and features not illustrated. The one or more SoCs 304 can be used to control the vehicle 300 in a variety of platforms and systems. For example, the one or more SoCs 304 in a system (e.g., the system of the vehicle 300) can be combined with an HD card 322, which is accessed via a network interface 324 by one or more servers (e.g., the one or more servers 378). Fig. 3D) map refreshes and / or updates.

[0106] The one or more CPUs 306 can contain a CPU cluster or CPU complex (hereafter referred to as "CCPLEX"). The one or more CPUs 306 can contain multiple cores and / or L2 caches. In some embodiments, the one or more CPUs 306 can, for example, contain eight cores in a coherent multiprocessor configuration. In some embodiments, the one or more CPUs 306 can contain four dual-core clusters, each cluster having a dedicated L2 cache (e.g., a 2 MB L2 cache). The one or more CPUs 306 (e.g., the CCPLEX) can be configured to support the concurrent operation of clusters, so that any combination of clusters of the one or more CPUs 306 can be active at any given time.

[0107] The one or more CPUs 306 can implement power management features that include one or more of the following: individual hardware blocks can be automatically clocked when idle to dynamically conserve power; each core clock can be controlled when the core is not actively executing instructions due to the execution of WFI / WFE instructions; each core can be independently power-controlled; each core cluster can be independently clock-controlled when all cores are clock-controlled or power-controlled; and / or each core cluster can be independently power-controlled when all cores are power-controlled. The one or more CPUs 306 can also implement an enhanced power state management algorithm that establishes acceptable power states and expected wake-up times, and the hardware / microcode determines the best power state to input for the core, cluster, and CCPLEX.The processing kernels can support simplified sequences for inputting the energy state into the software, thereby offloading the work to the microcode.

[0108] The one or more GPUs 308 can include an integrated GPU (referred to herein alternatively as an "iGPU"). The one or more GPUs 308 can be programmable and can be efficient for parallel workloads. The one or more GPUs 308 can use an extended Tensor instruction set in some examples. The one or more GPUs 308 can include one or more streaming microprocessors, each of which can contain an L1 cache (for example, an L1 cache with a minimum of 96 KB of memory), and two or more of the streaming microprocessors can share an L2 cache (for example, an L2 cache with a minimum of 512 KB of memory). In some embodiments, the one or more GPUs 308 can contain at least eight streaming microprocessors. The one or more GPUs 308 can use one or more application programming interfaces (APIs) for computation.Furthermore, the one or more GPUs 308 can use one or more parallel computing platforms and / or programming models (e.g., NVIDIA's CUDA).

[0109] The one or more GPUs 308 can be power-optimized for best performance in automotive and embedded applications. The one or more GPUs 308 can be manufactured, for example, on a FinFET field-effect transistor. However, this is not a limitation, and the one or more GPUs 308 can also be manufactured using other semiconductor manufacturing processes. Each streaming microprocessor can contain an array of mixed-precision processing cores, divided into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 PF64 cores can be divided into four processing blocks. In such an example, each processing block can be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed-precision NVIDIA TENSOR COREs for deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit and / or a 64 KB register file.Furthermore, streaming microprocessors can include independent parallel integer and floating-point data paths to enable efficient execution of workloads with a mix of computations and addressing operations. Streaming microprocessors can include an independent thread scheduling function to allow fine-grained synchronization and cooperation between parallel threads. Streaming microprocessors can also include a combined L1 data cache and a shared memory unit to improve performance while simplifying programming.

[0110] The one or more GPUs 308 can include high-bandwidth memory (HBM) and / or a 16 GB HBM2 memory subsystem to provide a peak memory bandwidth of approximately 900 GB / second in some examples. In some examples, synchronous graphics random access memory (SGRAM), such as double-data-rate type five synchronous graphics random access memory (GDDR5), can be used in addition to or as an alternative to HBM memory.

[0111] The one or more GPUs 308 can incorporate a unified memory technology that includes access counters to enable more accurate migration of memory pages to the processor that accesses them most frequently, thereby improving the efficiency of memory areas shared by processors. In some examples, support for Address Translation Services (ATS) can be used so that the one or more GPUs 308 can directly access the page tables of the one or more CPUs 306. In such examples, if the Memory Management Unit (MMU) of the one or more GPUs 308 fails, an address translation request can be sent to the one or more CPUs 306.In response, the one or more CPUs 306 can search their page tables for the virtual-physical mapping for the address and send the translation back to the one or more GPUs 308. Thus, the unified memory technology enables a single, unified virtual address space for the memory of both the one or more CPUs 306 and the one or more GPUs 308, thereby simplifying the programming of the one or more GPUs 308 and the porting of applications to the one or more GPUs 308.

[0112] Additionally, the one or more GPUs 308 can contain an access counter that tracks the frequency of accesses by the one or more GPUs 308 to the memory of other processors. The access counter can help ensure that memory pages are moved into the physical memory of the processor that accesses them most frequently.

[0113] The one or more SoCs 304 can contain any number of caches 312, including those described here. The one or more caches 312 can, for example, contain an L3 cache that is available to both the one or more CPUs 306 and the one or more GPUs 308 (e.g., one that is connected to both the one or more CPUs 306 and the one or more GPUs 308). The one or more caches 312 can contain a write-back cache that can track the states of the rows, for example, by using a cache coherence protocol (e.g., MEI, MESI, MSI, etc.). The L3 cache can be 4 MB or larger, depending on the implementation, although smaller cache sizes can also be used.

[0114] The one or more SoCs 304 can contain one or more arithmetic logic units (ALUs) that can be used to perform processing related to one of the many tasks or operations of the vehicle 300—such as processing deep neural networks (DNNs). Additionally, the one or more SoCs 304 can contain one or more floating-point units (FPUs)—or other mathematical or numerical coprocessors—for performing mathematical operations within the system. For example, the one or more SoCs 304 can contain one or more FPUs integrated as execution units into one or more CPUs 306 and / or one or more GPUs 308.

[0115] The one or more SoCs 304 can contain one or more Accelerators 314 (e.g., hardware accelerators, software accelerators, or a combination thereof). For example, the one or more SoCs 304 can contain a hardware acceleration cluster, which may include optimized hardware accelerators and / or a large amount of on-chip memory. The large on-chip memory (e.g., 4 MB SRAM) can enable the hardware acceleration cluster to accelerate neural networks and other computations. The hardware acceleration cluster can be used in conjunction with the one or more GPUs 308 and offload some of the tasks performed by the one or more GPUs 308 (e.g., to free up more cycles of the one or more GPUs 308 for other tasks). The one or more Accelerators 314 can, for example, be used for specific workloads (e.g.,Perception, convolutional neural networks (CNNs), etc., are used that are stable enough to be suitable for acceleration. The term "CNN" as used here can include all types of CNNs, including region-based or regional convolutional neural networks (RCNNs) and fast RCNNs (e.g., for object detection).

[0116] The one or more Accelerators 314 (e.g., the Hardware Acceleration Cluster) can include a Deep Learning Accelerator (DLA). The one or more DLAs can include one or more Tensor Processing Units (TPUs) configured to provide an additional ten trillion operations per second for deep learning applications and inference. The TPUs can be accelerators configured and optimized to perform image processing functions (e.g., for CNNs, RCNNs, etc.). The one or more DLAs can also be optimized for a specific set of neural network types and floating-point operations, as well as for inference. The design of the one or more DLAs can deliver more performance per millimeter than a general-purpose GPU and far surpasses the performance of a CPU.The one or more TPUs can perform multiple functions, including a convolution function for a single instance that supports, for example, INT8, INT16 and FP16 data types for both features and weights, as well as post-processor functions.

[0117] One or more DLAs can quickly and efficiently run neural networks, especially CNNs, on processed or unprocessed data for a variety of functions, including, but not limited to: a CNN for object identification and detection using camera sensor data; a CNN for distance estimation using camera sensor data; a CNN for emergency vehicle detection and identification using microphone data; a CNN for facial recognition and vehicle owner identification using camera sensor data; and / or a CNN for security and / or protection-related events.

[0118] The one or more DLAs can perform any function of the one or more GPUs 308, and by using an inference accelerator, a developer can, for example, allocate either the one or more DLAs or the one or more GPUs 308 to each function. For example, the developer can concentrate the processing of CNNs and floating-point operations on the one or more DLAs and leave other functions to the one or more GPUs 308 and / or other accelerators 314.

[0119] The one or more Accelerators 314 (e.g., the Hardware Acceleration Cluster) can contain a Programmable Vision Accelerator (PVA), which may also be referred to here as a Computer Vision Accelerator. The one or more PVAs can be designed and configured to accelerate computer vision algorithms for Advanced Driver Assistance Systems (ADAS), autonomous driving, and / or augmented reality (AR) and / or virtual reality (VR) applications. The one or more PVAs can offer a balance between performance and flexibility. For example, each PVA can contain any number of Reduced Instruction Set Computer (RISC) cores, Direct Memory Access (DMA) cores, and / or any number of vector processors, without limitation.

[0120] The RISC cores can interact with image sensors (e.g., the image sensors of one of the cameras described here), image signal processors, and / or the like. Each RISC core can contain any amount of memory. Depending on the implementation, the RISC cores can use any number of protocols. In some examples, the RISC cores can run a real-time operating system (RTOS). The RISC cores can be implemented with one or more integrated circuits, application-specific integrated circuits (ASICs), and / or memory devices. For example, the RISC cores can include an instruction cache and / or tightly coupled RAM.

[0121] The DMA can allow components of the PVA(s) to access the system's memory independently of the single or multiple CPUs. The DMA can support any number of features that optimize the PVA, including, but not limited to, support for multidimensional and / or circular addressing. In some examples, the DMA can support up to six or more dimensions of addressing, which may include block width, block height, block depth, horizontal block stepping, vertical block stepping, and / or depth stepping.

[0122] Vector processors can be programmable processors designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing capabilities. In some examples, the PVA may contain a PVA core and two vector processing subsystem partitions. The PVA core may include a processor subsystem, one or more DMA machines (e.g., two DMA machines), and / or other peripheral devices. The vector processing subsystem may operate as the primary processing machine of the PVA and may include a vector processing unit (VPU), an instruction cache, and / or working memory (e.g., VMEM).A VPU core can contain a digital signal processor, such as a single instruction, multiple data (SIMD) or a very long instruction word (VLIW). The combination of SIMD and VLIW can increase throughput and speed.

[0123] Each vector processor can contain an instruction cache and can be coupled to dedicated memory. Therefore, in some examples, each vector processor can be configured to operate independently of the others. In other examples, the vector processors contained in a particular PVA can be configured to use data parallelism. For example, in some embodiments, the multiple vector processors contained in a single PVA can execute the same computer vision algorithm, but on different regions of an image. In other examples, the vector processors contained in a particular PVA can simultaneously execute different computer vision algorithms on the same image, or even different algorithms on successive images or sections of an image.Among other things, any number of PVAs can be included in the hardware acceleration cluster, and any number of vector processors can be contained in each of the PVAs. Furthermore, one or more PVAs can include additional memory for error-correcting code (ECC) to increase the overall security of the system.

[0124] The one or more Accelerator 314 units (e.g., the hardware acceleration cluster) can include an on-chip computer vision network and SRAM to provide high-bandwidth, low-latency SRAM for the Accelerator 314. In some examples, the on-chip memory can include at least 4 MB of SRAM, consisting, for example, and without limitation, of eight field-configurable memory blocks accessible to both the PVA and the DLA. Each pair of memory blocks can include an Advanced Peripheral Bus (APB) interface, configuration circuitry, a controller, and a multiplexer. Any type of memory can be used. The PVA and the DLA can access the memory via a backbone, enabling high-speed memory access for both the PVA and the DLA.The backbone can include an on-chip computer vision network that connects the PVA and DLA to the main memory (e.g., using the APB).

[0125] The on-chip computer vision network can include an interface that, prior to the transmission of control signals / addresses / data, ensures that both the PVA and the DLA provide ready-to-use and valid signals. Such an interface can provide separate phases and channels for the transmission of control signals / addresses / data, as well as burst communication for continuous data transmission. This type of interface can conform to ISO 26262 or IEC 61508 standards, although other standards and protocols can also be used.

[0126] In some examples, one or more SoCs 304 can include a real-time ray tracing hardware accelerator as described in US patent application no. 16 / 101,232, filed on August 10, 2018. The real-time ray tracing hardware accelerator can be used to quickly and efficiently determine the positions and extents of objects (e.g., within a world model), to generate real-time visualization simulations, for radar signal interpretation, for sound propagation synthesis and / or analysis, for simulating SONAR systems, for general wave propagation simulation, for comparison with lidar data for localization purposes, and / or for other functions and / or purposes. In some embodiments, one or more Tree Traversal Units (TTUs) can be used to perform one or more operations related to ray tracing.

[0127] The single or multiple Accelerator 314 (e.g., the hardware accelerator cluster) have a wide range of applications for autonomous driving. The PVA can be a programmable vision accelerator used for critical processing steps in ADAS and autonomous vehicles. The PVA's capabilities are well-suited to algorithmic domains requiring predictable processing with low power consumption and low latency. In other words, the PVA is well-suited for semi-dense or dense regular computations, even with small datasets, that require predictable runtimes with low latency and low power consumption. Therefore, in the context of autonomous vehicle platforms, PVAs are designed to execute classic computer vision algorithms because they are efficient at object detection and operate with integer mathematics.

[0128] According to one embodiment of the technology, the PVA is used, for example, to perform computer stereovision. In some examples, a semi-global matching-based algorithm can be used, although this is not intended as a limitation. Many applications for Level 3-5 autonomous driving require spontaneous motion estimation or stereo matching (e.g., structure of motion, pedestrian detection, lane detection, etc.). The PVA can perform computer stereovision on input from two monocular cameras.

[0129] In some examples, the PVA can be used to perform a dense optical flow, such as processing raw radar data (e.g., using a 4D Fast Fourier Transform) to produce processed radar data. In other examples, the PVA is used for time-of-flight depth processing, for example, by processing raw time-of-flight data to deliver processed time-of-flight data.

[0130] The DLA can be used to power any type of network to improve control and driving safety; this includes, for example, a neural network that outputs a confidence score for each object detection. Such a confidence score can be interpreted as a probability or as providing a relative "weighting" of each detection compared to other detections. This confidence score allows the system to make further decisions about which detections should be considered true positives and not false positives. For example, the system can set a confidence threshold and consider only those detections that exceed the threshold as true positives.In an automatic emergency braking (AEB) system, false positive detections would cause the vehicle to automatically initiate emergency braking, which is obviously undesirable. Therefore, only the safest detections should be considered as triggers for AEB. The DLA can employ a neural network for confidence regression. The neural network can use as input at least a subset of parameters, such as the dimensions of the boundary frame, the ground plane estimate (obtained, for example, from another subsystem), the output of the inertial measurement unit (IMU) sensor 366, which correlates with the vehicle's orientation 300, distance, and 3D position estimates of the object obtained from the neural network and / or other sensors (e.g., one or more LiDAR sensors 364 or one or more radar sensors 360).

[0131] The one or more SoCs 304 can contain the one or more data stores 316 (e.g., main memory). The one or more data stores 316 can be on-chip main memory on the one or more SoCs 304, where neural networks can be stored to run on the GPU and / or the DLA. In some examples, the one or more data stores 316 can be large enough to store multiple instances of neural networks for redundancy and security. The one or more data stores 316 can include one or more L2 or L3 caches 312. The reference to the one or more data stores 316 can include a reference to the main memory allocated to the PVA, the DLA, and / or one or more other accelerators 314, as described here.

[0132] The one or more SoCs 304 can contain one or more Processors 310 (e.g., embedded processors). The one or more Processors 310 can contain a Boot and Power Management Processor, which can be a dedicated processor and subsystem to handle boot power and management functions and the associated security enforcement. The Boot and Power Management Processor can be part of the boot sequence of the one or more SoCs 304 and can provide runtime power management services. The Boot and Power Management Processor can provide clock and voltage programming, support for system transitions to a low-power state, management of the thermals and temperature sensors of the one or more SoCs 304, and / or management of the one or more SoCs 304 power states.Each temperature sensor can be implemented as a ring oscillator whose output frequency is proportional to the temperature, and the one or more SoCs 304 can use the ring oscillators to detect the temperatures of the one or more CPUs 306, the one or more GPUs 308, and / or the one or more accelerators 314. If it is determined that the temperatures exceed a threshold, the boot and power management processor can enter a temperature fault routine and put the one or more SoCs 304 into a reduced-power state and / or put the vehicle 300 into a chauffeur-to-safe-stop mode (e.g., bring the vehicle 300 to a safe stop).

[0133] The one or more 310 processors can also include a number of embedded processors that can serve as an audio processing engine. The audio processing engine can be an audio subsystem that provides full hardware support for multi-channel audio across multiple interfaces and a wide and flexible range of audio I / O interfaces. In some examples, the audio processing engine is a dedicated processor core with a digital signal processor and dedicated RAM.

[0134] The one or more 310 processors can also include an always-on processor machine, which provides the necessary hardware functions to support low-power sensor management and wake-up of use cases. The always-on processor machine can include a processor core, tightly coupled RAM, supporting peripherals (such as timers and interrupt controllers), various I / O controller peripherals, and routing logic.

[0135] The one or more 310 processors can also include a security cluster machine, which contains a dedicated processor subsystem for the security management of automotive applications. The security cluster machine can include two or more processor cores, tightly coupled RAM, supporting peripherals (such as timers, an interrupt controller, etc.), and / or routing logic. In a security mode, the two or more cores can operate in lockstep mode, functioning as a single core with comparison logic that detects any differences between their operations.

[0136] The one or more 310 processors can also contain a real-time camera machine, which may include a dedicated processor subsystem for managing the real-time camera.

[0137] The one or more 310 processors may also include a high dynamic range signal processor, which may include an image signal processor, which is a hardware machine that is part of the camera processing pipeline.

[0138] The one or more 310 processors can contain a video image compositor, which can be a processing block (e.g., implemented on a microprocessor) that implements video post-processing functions required by a video playback application to generate the final image for the player window. The video image compositor can perform lens distortion correction on the one or more 370 wide-angle cameras, the one or more 374 ambient lighting cameras, and / or on the sensors of the in-cabin surveillance camera. A sensor of the in-cabin surveillance camera is preferably monitored by a neural network running on a separate instance of the extended SoC and configured to detect events in the cabin and respond accordingly.The in-cabin system can lip-read to activate mobile service and make calls, dictate emails, change the destination, activate or change the infotainment system and vehicle settings, or enable voice-controlled internet browsing. Certain functions are only available to the driver when the vehicle is operating in autonomous mode and are otherwise deactivated.

[0139] The video image compositor can include enhanced temporal noise reduction for both spatial and temporal noise reduction. For example, if there is motion in a video, the noise reduction weights the spatial information accordingly and reduces the impact of information provided by adjacent frames. If a frame or portion of a frame does not contain motion, the temporal noise reduction performed by the video image compositor can use information from the previous frame to reduce noise in the current frame.

[0140] The video image compositor can also be configured to perform stereo equalization of the input stereo lens images. Furthermore, the video image compositor can be used for user interface design when the operating system desktop is in use and the one or more GPUs 308 do not need to constantly render new surfaces. Even when the one or more GPUs 308 are powered on and actively performing 3D rendering, the video image compositor can be used to offload the workload from the GPUs 308, thus improving performance and responsiveness.

[0141] The one or more SoCs 304 can also include a serial camera interface with a Mobile Industry Processor Interface (MIPI) for receiving video and camera input, a high-speed interface, and / or a video input block that can be used for camera and related pixel input functions. The one or more SoCs 304 can also include one or more input / output controllers, one or more of which can be software-controlled and used for receiving I / O signals that are not assigned to a specific role.

[0142] The one or more SoCs 304 can also include a wide range of peripheral interfaces to enable communication with peripheral devices, audio codecs, power management, and / or other devices. The one or more SoCs 304 can be used to process data from cameras (e.g., via Gigabit Multimedia Serial Link and Ethernet), sensors (e.g., one or more LiDAR sensors 364, one or more RADAR sensors 360, etc., which can be connected via Ethernet), data from the bus 302 (e.g., vehicle speed 300, steering wheel position, etc.), and data from one or more GNSS sensors 358 (e.g., connected via Ethernet or CAN bus).Furthermore, the one or more SoCs 304 can contain dedicated high-performance mass storage controllers, which can contain their own DMA machines and can be used to offload routine data management tasks from the one or more CPUs 306.

[0143] The single or multiple SoCs 304 can form an end-to-end platform with a flexible architecture spanning automation levels 3-5, thereby providing a comprehensive functional safety architecture that supports and efficiently utilizes computer vision and ADAS techniques for diversity and redundancy, and provides a platform for a flexible, reliable driving software stack along with deep learning tools. The single or multiple SoCs 304 can be faster, more reliable, and even more energy- and space-efficient than conventional systems. For example, the single or multiple accelerators 314, in combination with the single or multiple CPUs 306, the single or multiple GPUs 308, and the single or multiple data stores 316, can form a fast, efficient platform for autonomous vehicles of levels 3-5.

[0144] This technology thus offers capabilities and functions that cannot be achieved by conventional systems. For example, computer vision algorithms can be run on CPUs that can be configured using a high-level programming language, such as C, to execute a variety of processing algorithms on a wide range of visual data. However, CPUs are often unable to meet the performance requirements of many computer vision applications, such as execution time and power consumption. In particular, many CPUs are unable to execute complex object detection algorithms in real time, which is a requirement for in-vehicle ADAS applications and a prerequisite for practical Level 3-5 autonomous vehicles.

[0145] Unlike conventional systems, the technology described herein, by providing a CPU complex, a GPU complex, and a hardware acceleration cluster, enables the simultaneous and / or sequential execution of multiple neural networks and the combination of their results to enable Level 3-5 autonomous driving functionality. For example, a CNN running on the DLA or the dGPU (e.g., one or more GPUs 320) can include text and word recognition, allowing the supercomputer to read and understand traffic signs, including signs for which the neural network has not been specifically trained. The DLA can further include a neural network capable of identifying and interpreting the sign, providing a semantic understanding, and passing this semantic understanding to the path planning modules running on the CPU complex.

[0146] Another example is that multiple neural networks can run simultaneously, as required for driving at levels 3, 4, or 5. For instance, a warning sign reading "Caution: Flashing lights indicate black ice" accompanied by an electric light can be interpreted independently or jointly by several neural networks. The sign itself can be identified as a traffic sign by a first neural network (e.g., a trained one), while the text "Flashing lights indicate black ice" can be interpreted by a second neural network, which then informs the vehicle's path planning software (preferably running on the CPU) that the presence of black ice indicates the presence of flashing lights.The turn signal can be identified across multiple images by a third neural network, which informs the vehicle's path planning software about the presence (or absence) of turn signals. All three neural networks can run simultaneously, e.g., within the DLA and / or on one or more GPUs 308.

[0147] In some examples, a CNN for facial recognition and vehicle owner identification can use data from camera sensors to identify the presence of an authorized driver and / or owner of the vehicle. The always-on sensor processing unit can be used to unlock the vehicle and turn on the lights when the owner approaches the driver's door, and to disable the vehicle in security mode when the owner leaves. In this way, the one or more SoCs provide security against theft and / or carjacking.

[0148] In another example, a CNN for detecting and identifying emergency vehicles can use data from microphones 396 to detect and identify emergency vehicle sirens. Unlike conventional systems that use general classifiers to detect sirens and manually extract features, the one or more SoCs 304 use the CNN to classify environmental and urban sounds as well as visual data. In a preferred embodiment, the CNN running on the DLA is trained to detect the relative approach speed of the emergency vehicle (e.g., by using the Doppler effect). The CNN can also be trained to identify emergency vehicles specific to the local area in which the vehicle is operating, as identified by one or more GNSS sensors 358.For example, the CNN will attempt to detect European sirens when it is operating in Europe, and when it is operating in the United States, the CNN will attempt to identify only North American sirens. Once an emergency vehicle is detected, a controller can be used to execute an emergency vehicle safety routine, slowing the vehicle down, pulling over to the side of the road, parking the vehicle, and / or letting the vehicle idle, using the 362 ultrasonic sensors, until one or more emergency vehicles pass.

[0149] The vehicle can contain one or more CPUs 318 (e.g., one or more discrete CPUs or one or more dCPUs) that can be coupled to the one or more SoCs 304 via a high-speed connection (e.g., PCIe). The CPUs 318 can, for example, contain an x86 processor. The CPUs 318 can be used, for example, to perform a variety of functions, including reconciling potentially inconsistent results between ADAS sensors and the one or more SoCs 304 and / or monitoring the status and health of the one or more Controllers 336 and / or the Infotainment SoC 330.

[0150] The Vehicle 300 can contain one or more GPUs 320 (e.g., one or more discrete GPUs or one or more dGPUs) that can be coupled to the one or more SoCs 304 via a high-speed connection (e.g., NVIDIA's NVLINK). The one or more GPUs 320 can provide additional artificial intelligence capabilities, such as running redundant and / or different neural networks, and can be used to train and / or update neural networks based on inputs (e.g., sensor data) from the Vehicle 300's sensors.

[0151] The vehicle 300 may also include the network interface 324, which may contain one or more wireless antennas 326 (e.g., one or more wireless antennas for different communication protocols, such as a cellular antenna, a Bluetooth antenna, etc.). The network interface 324 can be used to enable a wireless connection via the internet to the cloud (e.g., to the one or more servers 378 and / or other network devices), to other vehicles, and / or to computing devices (e.g., client devices of passengers). To communicate with other vehicles, a direct connection between the two vehicles and / or an indirect connection (e.g., via networks and the internet) can be established. Direct connections can be established via vehicle-to-vehicle communication.Vehicle-to-vehicle communication can provide the vehicle 300 with information about vehicles in its vicinity (e.g., vehicles in front of, beside, and / or behind the vehicle 300). This functionality can be part of a cooperative adaptive cruise control function of the vehicle 300.

[0152] The network interface 324 can include a system-on-a-chip (SoC) that provides modulation and demodulation functions, enabling one or more controllers 336 to communicate over wireless networks. The network interface 324 can include a high-frequency (RF) front end for up-conversion from baseband to RF and down-conversion from RF to baseband. The frequency conversions can be performed using known methods and / or superheterodyne methods. In some examples, the RF front-end functionality can be provided by a separate chip. The network interface can include wireless functionality for communication over LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and / or other wireless protocols.

[0153] The vehicle 300 may further include one or more data storage devices 328, which may be located outside the chip (e.g., outside the SoCs 304). The one or more data storage devices 328 may contain one or more memory elements, including RAM, SRAM, DRAM, VRAM, flash, hard disks, and / or other components and / or devices capable of storing at least one bit of data.

[0154] The vehicle 300 can also include one or more GNSS sensors 358. The one or more GNSS sensors 358 (e.g., GPS, supported GPS sensors, differential GPS (DGPS) sensors, etc.) assist with mapping, perception, occupancy grid creation, and / or path planning. Any number of GNSS sensors 358 can be used, including, for example, and without limitation, a GPS unit that uses a USB connection with an Ethernet-to-serial (RS-232) bridge.

[0155] The vehicle 300 can also include one or more RADAR sensors 360. The vehicle 300 can use the one or more RADAR sensors 360 to detect vehicles at long range, even in darkness and / or adverse weather conditions. The functional safety level of the RADAR can be ASIL B. The one or more RADAR sensors 360 can use the CAN bus and / or the 302 bus (e.g., for transmitting the data generated by the one or more RADAR sensors 360) for control and access to object tracking data, with some examples using Ethernet for access to the raw data. A variety of RADAR sensor types can be used. The one or more RADAR sensors 360 can be suitable for front, rear, and side RADAR applications without restriction. In some examples, one or more pulse-Doppler RADAR sensors are used.

[0156] The single or multiple 360° radar sensors can incorporate various configurations, such as long-range with a narrow field of view, short-range with a wide field of view, short-range with side coverage, and so on. In some cases, long-range radar can be used for adaptive cruise control. Long-range radar systems can provide a wide field of view, achieved through two or more independent scans, for example, at a range of 250 m. The single or multiple 360° radar sensors can assist in distinguishing between stationary and moving objects and can be used by ADAS systems for emergency braking and forward collision warning. Long-range radar sensors can include a monostatic multimodal radar with multiple (e.g., six or more) fixed radar antennas and a high-speed CAN and FlexRay interface.In an example with six antennas, the four central antennas can generate a focused beam pattern designed to detect the area around vehicle 300 at higher speeds with minimal interference from traffic in adjacent lanes. The other two antennas can expand the field of view, enabling the rapid detection of vehicles entering or exiting vehicle 300's lane.

[0157] Medium-range radar systems can, for example, have a range of up to 160 m (front) or 80 m (rear) and a field of view of up to 42 degrees (front) or 150 degrees (rear). Short-range radar systems can include, among other things, radar sensors designed for installation at both ends of the rear bumper. When such a radar sensor system is installed at both ends of the rear bumper, it can generate two beams that continuously monitor the blind spot behind and to the sides of the vehicle.

[0158] Short-range radar systems can be used in an ADAS system for blind spot detection and / or as a lane change assistant.

[0159] The vehicle 300 can also include one or more ultrasonic sensors 362. The one or more ultrasonic sensors 362, which can be mounted on the front, rear, and / or sides of the vehicle 300, can be used for parking assistance and / or for creating and updating an occupancy grid. A variety of ultrasonic sensors 362 can be used, and different ultrasonic sensors 362 can be used for different detection ranges (e.g., 2.5 m, 4 m). The one or more ultrasonic sensors 362 can operate with functional safety levels of ASIL B.

[0160] The vehicle 300 can contain one or more LiDAR sensors 364. The one or more LiDAR sensors 364 can be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. The one or more LiDAR sensors 364 can meet the functional safety level ASIL B. In some examples, the vehicle 300 can contain multiple LiDAR sensors 364 (e.g., two, four, six, etc.) that can use Ethernet (e.g., to deliver data to a Gigabit Ethernet switch).

[0161] In some examples, one or more LiDAR sensors 364 may be able to provide a list of objects and their distances for a 360-degree field of view. Commercially available LiDAR sensors 364 may, for example, have a specified range of approximately 100 m, with an accuracy of 2 cm to 3 cm and support for a 100 Mbit / s Ethernet connection. In some examples, one or more non-protruding LiDAR sensors 364 may be used. In such examples, the one or more LiDAR sensors 364 may be implemented as a small device that can be embedded in the front, rear, sides, and / or corners of the vehicle 300. In such examples, one or more LIDAR 364 sensors can provide a horizontal field of view of up to 120 degrees and a vertical field of view of up to 35 degrees, with a range of 200 m, even with objects of low reflectivity.The one or more front-mounted LIDAR 364 sensors can be configured for a horizontal field of view between 45 degrees and 135 degrees.

[0162] In some examples, LiDAR technologies, such as 3D flash LiDAR, can also be used. 3D flash LiDAR uses a laser pulse as a transmission source to illuminate the vehicle's surroundings up to approximately 200 m. A flash LiDAR unit contains a sensor that records the travel time of the laser pulse and the reflected light at each pixel, which in turn corresponds to the distance between the vehicle and the objects. Flash LiDAR can enable the generation of highly accurate and distortion-free images of the surroundings with each laser pulse. In some examples, four flash LiDAR sensors can be used, one on each side of the vehicle. Available 3D flash LiDAR systems include a solid-state 3D focal plane array LiDAR camera that contains no moving parts other than a fan (e.g., a non-scanning LiDAR device).The flash LIDAR device can use a 5-nanosecond pulse of a Class I (eye-safe) laser per frame and capture the reflected laser light in the form of 3D distance point clouds and co-registered intensity data. By using flash LIDAR, and because flash LIDAR is a solid-state device with no moving parts, the single or multiple LIDAR sensors can be less susceptible to motion blur, vibration, and / or shock.

[0163] The vehicle may also contain one or more IMU sensors 366. In some examples, the one or more IMU sensors 366 may be located in the center of the rear axle of the vehicle 300. The one or more IMU sensors 366 may, for example, and without limitation, contain one or more accelerometers, one or more magnetometers, one or more gyroscopes, one or more magnetic compasses, and / or other sensor types. In some examples, such as six-axis applications, the one or more IMU sensors 366 may contain accelerometers and gyroscopes, while in nine-axis applications, the one or more IMU sensors 366 may contain accelerometers, gyroscopes, and magnetometers.

[0164] In some embodiments, the one or more IMU sensors 366 can be implemented as a miniaturized, high-performance GPS-aided inertial navigation system (GPS / INS) that combines inertial sensors of a microelectromechanical system (MEMS), a highly sensitive GPS receiver, and advanced Kalman filter algorithms to provide estimates of position, velocity, and orientation. Thus, in some examples, the one or more IMU sensors 366 can enable the vehicle 300 to estimate its course without requiring input from a magnetic sensor by directly observing and correlating velocity changes from the GPS with the one or more IMU sensors 366. In some examples, the one or more IMU sensors 366 and the one or more GNSS sensors 358 can be combined in a single integrated unit.

[0165] The vehicle may contain one or more microphones 396, which are mounted in and / or around the vehicle 300. The one or more microphones 396 may be used, among other things, for the detection and identification of emergency vehicles.

[0166] The vehicle can furthermore include any number of camera types, including one or more stereo cameras 368, one or more wide-angle cameras 370, one or more infrared cameras 372, one or more surround-view cameras 374, one or more long-range and / or medium-range cameras 398, and / or other camera types. The cameras can be used to capture image data around the entire periphery of the vehicle 300. The types of cameras used depend on the embodiment and requirements of the vehicle 300, and any combination of camera types can be used to ensure the necessary coverage around the vehicle 300. Furthermore, the number of cameras can vary depending on the embodiment. For example, the vehicle can include six cameras, seven cameras, ten cameras, twelve cameras, and / or another number of cameras.The cameras can, for example and without limitation, support Gigabit Multimedia Serial Link (GMSL) and / or Gigabit Ethernet. Each of the one or more cameras is described here with reference to... Fig. 3A and Fig. 3B is described in more detail.

[0167] The vehicle 300 may also include one or more vibration sensors 342. The one or more vibration sensors 342 can measure vibrations of vehicle components, such as one or more axles. For example, changes in vibrations may indicate a change in the road surface. In another example, if two or more vibration sensors 342 are used, the differences between the vibrations can be used to determine the friction or slip on the road surface (e.g., if the difference in vibration is between a driven axle and a freely rotating axle).

[0168] The vehicle 300 may include an ADAS system 338. The ADAS system 338 may include a SoC in some examples. The ADAS system 338 may include autonomous / adaptive / automatic cruise control (ACC), cooperative adaptive cruise control (CACC), forward crash warning (FCW), automatic emergency braking (AEB), lane departure warning (LDW), lane keep assist (LKA), blind spot warning (BSW), rear cross-traffic warning (RCTW), collision warning systems (CWS), lane centering (LC), and / or other features and functions.

[0169] The ACC systems can use one or more 360° radar sensors, one or more 364° lidar sensors, and / or one or more cameras. The ACC systems can include longitudinal and / or lateral ACC. Longitudinal ACC monitors and controls the distance to the vehicle immediately in front and automatically adjusts the vehicle speed to maintain a safe distance from vehicles ahead. Lateral ACC performs distance control and advises the vehicle to change lanes if necessary. Lateral ACC interacts with other ADAS applications, such as LCA and CWS.

[0170] The CACC uses information from other vehicles, which can be received via the network interface 324 and / or the one or more wireless antennas 326 from other vehicles via a wireless connection or indirectly via a network connection (e.g., via the Internet). Direct connections can be provided via a vehicle-to-vehicle (V2V) communication link, while indirect connections can be an infrastructure-to-vehicle (I2V) communication link. In general, the V2V communication concept provides information about the vehicles immediately ahead (e.g., vehicles that are directly in front of the vehicle 300 and in the same lane), while the I2V communication concept provides information about traffic further ahead. CACC systems can incorporate both I2V and V2V information sources.Given the information about the vehicles ahead of vehicle 300, the CACC can be more reliable and has the potential to improve traffic flow and reduce congestion on the road.

[0171] FCW systems are designed to warn the driver of a hazard, allowing them to take corrective action. FCW systems use a forward-facing camera and / or one or more 360° radar sensors coupled with a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to the driver feedback system, such as a display, speaker, and / or vibrating component. FCW systems can provide a warning in the form of an audible signal, a visual warning, a vibration, and / or a rapid braking pulse.

[0172] AEB systems detect an impending forward collision with another vehicle or object and can automatically apply the brakes if the driver does not take corrective action within a specific time or distance parameter. AEB systems can use one or more forward-facing cameras and / or one or more 360° radar sensors coupled with a dedicated processor, DSP, FPGA, and / or ASIC. When the AEB system detects a hazard, it typically first warns the driver so they can take corrective action to avoid the collision; if the driver fails to take corrective action, the AEB system can automatically apply the brakes to prevent or at least mitigate the effects of the predicted collision. AEB systems may incorporate techniques such as dynamic brake assist and / or emergency braking for an impending collision.

[0173] Lane Departure Warning (LDW) systems provide visual, audible, and / or tactile warnings, such as steering wheel or seat vibrations, to alert the driver if the vehicle crosses lane markings. An LDW system will not activate if the driver indicates an intentional lane departure by using a turn signal. LDW systems may utilize forward-facing cameras coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to the feedback signal for the driver, such as a display, speaker, and / or vibrating component.

[0174] Lane Keeping Assist (LKA) systems are a variant of Lane Departure Warning (LDW) systems. LKA systems provide steering or braking inputs to correct the vehicle if it begins to drift out of its lane. Blind Spot Warning (BSW) systems detect and warn the driver of vehicles in the car's blind spot. BSW systems can provide visual, audible, and / or tactile warnings to indicate that merging into or changing lanes is unsafe. The system can issue an additional warning if the driver activates a turn signal. BSW systems can use one or more rear-facing cameras and / or radar sensors.

[0175] Rear Cross Traffic Alert (RCTW) systems can provide visual, audible, and / or tactile alerts when an object is detected outside the reversing camera's field of view while the vehicle is reversing at 300 degrees. Some RCTW systems incorporate Advanced Emergency Braking (AEB) to ensure the vehicle's brakes are applied to prevent a collision. RCTW systems can utilize one or more rear-facing radar sensors coupled with a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to the driver for feedback, such as a display, speaker, and / or vibrating component.

[0176] Conventional ADAS systems can produce false positives, which, while annoying and distracting for the driver, are generally not catastrophic because the ADAS systems warn the driver and give them the opportunity to decide whether a safety issue truly exists and to act accordingly. However, in an autonomous vehicle 300, the vehicle 300 itself must decide, in the event of conflicting results, whether to follow the result from a primary computer or a secondary computer (e.g., a first controller 336 or a second controller 336). In some embodiments, the ADAS system 338 can, for example, be a backup and / or secondary computer that provides information about perception to a rationality module of the backup computer.The backup computer rationality monitor can run redundant, diverse software on hardware components to detect errors in perception and dynamic driving tasks. The outputs of the ADAS system 338 can be provided to a monitoring MCU. If the outputs of the primary and secondary computers conflict, the monitoring MCU must determine how to resolve the conflict to ensure safe operation.

[0177] In some examples, the primary computer can be configured to provide the monitoring MCU with a confidence score indicating its confidence in the chosen outcome. If the confidence score exceeds a threshold, the monitoring MCU can follow the primary computer's instruction, regardless of whether the secondary computer returns a conflicting or inconsistent result. If the confidence score does not reach the threshold and the primary and secondary computers display different results (e.g., conflicting results), the monitoring MCU can mediate between the computers to determine the appropriate outcome.

[0178] The monitoring MCU can be configured to run one or more neural networks that are trained and configured to determine, based on the output of the primary and secondary computers, the conditions under which the secondary computer will trigger false alarms. This allows the one or more neural networks in the monitoring MCU to learn when the output of the secondary computer can be trusted and when it cannot. For example, if the secondary computer is a radar-based FCW system, a neural network in the monitoring MCU can learn when the FCW system identifies metallic objects that do not actually pose a threat, such as a drain grate or manhole cover, triggering an alarm.Similarly, if the secondary computer is a camera-based lane departure warning (LDW) system, a neural network in the monitoring MCU can learn to override the LDW system when cyclists or pedestrians are present and leaving the lane is indeed the safest maneuver. In embodiments containing one or more neural networks running on the monitoring MCU, the monitoring MCU can include at least one DLA or GPU suitable for executing the one or more neural networks with associated memory. In preferred embodiments, the monitoring MCU can include and / or be contained as a component of one or more SoCs 304.

[0179] In other examples, the ADAS system 338 can include a secondary computer that executes the ADAS functionality according to the classical rules of computer vision. Thus, the secondary computer can use classical computer vision rules (if-then), and the presence of one or more neural networks in the monitoring MCU can improve reliability, safety, and performance. For example, the diverse implementation and intentional non-identity make the overall system more fault-tolerant, especially to errors caused by software (or software-hardware interfaces).For example, if a software bug or error occurs in the software on the primary computer and the non-identical software code on the secondary computer produces the same overall result, the monitoring MCU can have greater confidence that the overall result is correct and that the bug in the software or hardware on the primary computer does not cause a significant error.

[0180] In some examples, the output of the ADAS system 338 can be fed into the perception block of the primary computer and / or into the dynamic driving task block of the primary computer. For example, if the ADAS system 338 displays a forward collision warning due to an object immediately in front of the vehicle, the perception block can use this information in object identification. In other examples, the secondary computer may have its own trained neural network, thus reducing the risk of false positives, as described herein.

[0181] The Vehicle 300 may also include the Infotainment SoC 330 (e.g., an in-vehicle infotainment system (IVI)). Although illustrated and described as an SoC, the infotainment system may not actually be an SoC and may contain two or more discrete components. The Infotainment SoC 330 may include a combination of hardware and software that can be used to provide the Vehicle 300 with audio (e.g., music, a personal digital assistant, navigation instructions, news, radio, etc.), video (e.g., TV, movies, streaming, etc.), telephone (e.g., hands-free calling), network connectivity (e.g., LTE, Wi-Fi, etc.), and / or information services (e.g., navigation systems, rear parking sensors, a radio data system, vehicle-related information such as fuel level, total distance traveled, brake fluid level, oil level, door open / close status, air filter information, etc.).The Infotainment SoC 330 can, for example, include radios, record players, navigation systems, video players, USB and Bluetooth connectivity, car computers, in-car entertainment, Wi-Fi, steering wheel audio controls, hands-free systems, a head-up display (HUD), an HMI display 334, a telematics device, a control panel (e.g., for controlling and / or interacting with various components, functions, and / or systems), and / or other components. The Infotainment SoC 330 can also be used to provide information (e.g., visual and / or audible) to one or more vehicle users, such as information from the ADAS system 338, autonomous driving information such as planned vehicle maneuvers, road layouts, environmental information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.

[0182] The Infotainment SoC 330 may include GPU functionality. The Infotainment SoC 330 can communicate with other devices, systems, and / or components of the vehicle 300 via bus 302 (e.g., CAN bus, Ethernet, etc.). In some examples, the Infotainment SoC 330 may be coupled with a monitoring MCU so that the Infotainment System's GPU can perform some self-driving functions if one or more primary controllers 336 (e.g., the primary and / or backup computers of the vehicle 300) fail. In such an example, the Infotainment SoC 330 can put the vehicle 300 into a chauffeur-to-safe-stop mode, as described here.

[0183] The vehicle 300 may also include an instrument cluster 332 (e.g., a digital instrument cluster, an electronic instrument cluster, a digital instrument panel, etc.). The instrument cluster 332 may include a controller and / or supercomputer (e.g., a discrete controller or supercomputer). The instrument cluster 332 may contain a number of instruments, such as a speedometer, fuel gauge, oil pressure gauge, tachometer, odometer, turn signals, shift position indicator, seat belt warning light(s), parking brake warning light(s), engine malfunction light(s), airbag system (SRS) information, lighting controls, safety system controls, navigation information, etc. In some examples, information from the infotainment SoC 330 and the instrument cluster 332 may be displayed and / or shared. In other words, the instrument cluster 332 may be included as part of the infotainment SoC 330, or vice versa.

[0184] Fig. 3D is a system diagram for the communication between the one or more cloud-based servers and the exemplary autonomous vehicle 300. Fig. 3A, according to some embodiments of the present disclosure. The system 376 may include one or more servers 378, one or more networks 390, and the vehicles, including the vehicle 300. The server(s) 378 may include multiple GPUs 384(A)-384(H) (here collectively referred to as GPUs 384), PCIe switches 382(A)-382(H) (here collectively referred to as PCIe switches 382), and / or CPUs 380(A)-380(B) (here collectively referred to as CPUs 380). The GPUs 384, the CPUs 380, and the PCIe switches may be interconnected by high-speed links, such as, but not limited to, NVIDIA's NVLink interfaces 388 and / or PCIe links 386. In some examples, the GPUs 384 are connected via NVLink and / or NVSwitch SoC, and the GPUs 384 and the PCIe switches 382 are connected via PCIe links.Although eight GPUs 384, two CPUs 380, and two PCIe switches are illustrated, this should not be interpreted as a limitation. Depending on the configuration, each Server 378 can contain any number of GPUs 384, CPUs 380, and / or PCIe switches. For example, one or more Server 378s can each contain eight, sixteen, thirty-two, and / or more GPUs 384.

[0185] The one or more servers 378 can receive image data from the vehicles via the one or more networks 390. This image data is representative of images showing unexpected or changed road conditions, such as recently started roadworks. The one or more servers 378 can transmit neural networks 392, updated neural networks 392, and / or map information 394 to the vehicles via the one or more networks 390. This map information contains information about traffic and road conditions. The map information updates 394 can include updates to the HD map 322, such as information about construction sites, potholes, detours, flooding, and / or other obstacles.In some examples, the neural networks 392, the updated neural networks 392 and / or the map information 394 may result from new training and / or experience represented in the data received from any number of vehicles in the environment, and / or may be based on training performed in a data center (e.g. using one or more servers 378 and / or other servers).

[0186] One or more Server 378s can be used to train machine learning models (e.g., neural networks) based on training data. The training data can be generated by the vehicles and / or in a simulation (e.g., using a game machine). In some examples, the training data is tagged (e.g., if the neural network benefits from supervised learning) and / or subjected to other preprocessing, while in other examples, the training data is not tagged and / or preprocessed (e.g., if the neural network does not require supervised learning).Training can be performed using one or more classes of machine learning techniques, including, but not limited to, classes such as: supervised training, semi-supervised training, unsupervised training, self-learning, reinforcement learning, federated learning, transfer learning, feature learning (including principal component and cluster analysis), multilinear subspace learning, diverse learning, representational learning (including substitute dictionary learning), rule-based machine learning, anomaly detection, and all variants or combinations thereof. Once the machine learning models are trained, they can be used by the vehicles (e.g., transferred to the vehicles via one or more networks 390) and / or the machine learning models can be used by one or more servers 378 for remote monitoring of the vehicles.

[0187] In some examples, one or more Server 378s can receive data from the vehicles and apply that data to advanced neural networks in real time for intelligent, real-time inference. The one or more Server 378s can include deep learning supercomputers and / or dedicated AI computers powered by GPUs 384, such as NVIDIA's DGX and DGX Station machines. However, in some examples, the one or more Server 378s can include a deep learning infrastructure that uses only CPU-powered data centers.

[0188] The deep learning infrastructure of one or more servers 378 can perform fast, real-time inference and can use this capability to assess and verify the state of the processors, software, and / or associated hardware in the vehicle 300. For example, the deep learning infrastructure can receive periodic updates from the vehicle 300, such as a sequence of images and / or objects that the vehicle 300 has located within that sequence of images (e.g., via computer vision and / or other machine learning object classification techniques).The deep learning infrastructure can run its own neural network to identify the objects and compare them with the objects identified by the vehicle 300, and if the results do not match and the infrastructure concludes that the AI ​​in the vehicle 300 is not working correctly, one or more servers 378 can send a signal to the vehicle 300, instructing a fail-safe computer in the vehicle 300 to take control, notify the passengers and perform a safe parking maneuver.

[0189] For inference, one or more Server 378s can include GPUs 384s and one or more programmable inference accelerators (e.g., NVIDIA's TensorRT). The combination of GPU-driven servers and inference accelerators can enable real-time responsiveness. In other scenarios, such as when performance is less critical, servers powered by CPUs, FPGAs, and other processors can be used for inference. EXAMPLE CALCULATION DEVICE

[0190] Fig. Figure 4 is a block diagram of an exemplary computing device 400 suitable for use in implementing some embodiments of the present disclosure. The computing device 400 may include a connection system 402 that directly or indirectly couples the following devices: main memory 404, one or more central processing units (CPUs) 406, one or more graphics processing units (GPUs) 408, a communication interface 410, input / output (I / O) ports 412, input / output components 414, a power supply 416, one or more presentation components 418 (e.g., display(s)), and one or more logic units 420. In at least one embodiment, the one or more computing devices 400 may comprise one or more virtual machines (VMs), and / or each of the components thereof may comprise virtual components (e.g., virtual hardware components).As non-restrictive examples, one or more of the GPUs 408 can comprise one or more vGPUs, one or more of the CPUs 406 can comprise one or more vCPUs, and / or one or more of the logic units 420 can comprise one or more virtual logic units. Thus, a compute device 400 can contain discrete components (e.g., a complete GPU allocated to the compute device 400), virtual components (e.g., a portion of a GPU allocated to the compute device 400), or a combination thereof.

[0191] Although the various blocks of Fig. Where components 4 are shown as being connected via the connection system 402, this is not intended as a limitation and is for clarity only. In some embodiments, for example, a presentation component 418, such as a display device, can be considered an I / O component 414 (e.g., if the display is a touchscreen). As another example, the CPUs 406 and / or GPUs 408 can contain memory (e.g., the memory 404 can represent a storage device in addition to the memory of the GPUs 408, the CPUs 406, and / or other components). In other words, the computing device of Fig. Section 4 is for illustrative purposes only. No distinction is made between categories such as "workstation", "server", "laptop", "desktop", "tablet", "client device", "mobile device", "handheld device", "game console", "electronic control unit (ECU)", "virtual reality system" and / or other device or system types, as all are within the scope of protection of the computing device of Fig. 4 are possible.

[0192] The 402 interconnect system can represent one or more connections or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The 402 interconnect system can include one or more bus or connection types, such as an Industry Standard Architecture (ISA) bus, an Extended ISA bus, a Video Electronics Standards Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI Express (PCIe) bus, and / or another type of bus or connection. In some embodiments, there are direct connections between components. For example, the CPU 406 can be directly connected to the memory 404. Furthermore, the CPU 406 can be directly connected to the GPU 408.In a direct or point-to-point connection between components, the 402 connection system can include a PCIe link to establish the connection. In these examples, a PCI bus does not need to be included in the 400 computing device.

[0193] The 404 main memory can contain a variety of computer-readable media. Computer-readable media can be any available media that the 400 computing device can access. Computer-readable media can include both volatile and non-volatile media, as well as removable and non-removable media. For example, and without limitation, computer-readable media can include computer storage media and communication media.

[0194] Computer storage media can include both volatile and non-volatile media, and / or removable and non-removable media, implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, and / or other data types. For example, main memory can store computer-readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system). Computer storage media can include, but are not limited to, RAM, ROM, EEPROM, flash memory, or other storage technologies; CD-ROM, Digital Versatile Discs (DVDs), or other optical disk storage; magnetic cartridges, magnetic tapes, magnetic disk storage, or other magnetic storage devices; or any other medium that can be used to store the desired information and that the computing device can access.As used here, computer storage media do not inherently contain signals.

[0195] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal, such as a carrier wave or other transport mechanism, and include any media for transmitting information. The term "modulated data signal" can refer to a signal in which one or more of its properties are set or modified to encode information within the signal. Computer storage media can include, but are not limited to, wired media, such as a wired network or a direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of the above should also be included in the scope of protection of the computer-readable media.

[0196] The one or more CPUs 406 can be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 400 to perform one or more of the procedures and / or processes described herein. The one or more CPUs 406 can each contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a plurality of software threads simultaneously. The one or more CPUs 406 can contain any type of processor and can contain different types of processors depending on the type of computing device 400 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers).Depending on the type of computing device 400, the processor can be, for example, an Advanced RISC Machine (ARM) processor implemented with Reduced Instruction Set Computing (RISC), or an x86 processor implemented with Complex Instruction Set Computing (CISC). The computing device 400 can contain one or more CPUs 406, in addition to one or more microprocessors or additional coprocessors, such as mathematical coprocessors.

[0197] In addition to or as an alternative to the one or more CPUs 406, the one or more GPUs 408 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 400 to perform one or more of the procedures and / or processes described herein. One or more of the GPUs 408 may be an integrated GPU (e.g., with one or more of the CPUs 406) and / or one or more of the GPUs 408 may be a discrete GPU. In embodiments, one or more of the GPUs 408 may be a coprocessor of one or more of the CPUs 406. The one or more GPUs 408 may be used by the computing device 400 to render graphics (e.g., 3D graphics) or to perform general-purpose computing. For example, the one or more GPUs 408 may be used for general-purpose computing on GPUs (GPGPU).The one or more GPUs 408 can contain hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The one or more GPUs 408 can generate pixel data for output images in response to rendering commands (e.g., rendering commands from the one or more CPUs 406 received through a host interface). The one or more GPUs 408 can include graphics memory, such as display memory, for storing pixel data or other suitable data, such as GPGPU data. The display memory can be included as part of the 404 main memory. The one or more GPUs 408 can contain two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK) or connect them via a switch (e.g., using NVSwitch).When combined, each GPU can generate 408 pixel data or GPGPU data for different sections of an output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can have its own dedicated memory or share memory with other GPUs.

[0198] In addition to or as an alternative to the one or more CPUs 406 and / or the one or more GPUs 408, the one or more logic units 420 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 400 to perform one or more of the methods and / or processes described herein. In embodiments, the one or more CPUs 406, the GPUs 408, and / or the one or more logic units 420 may discretely or jointly execute any combination of the methods, processes, and / or sections thereof. One or more of the logic units 420 may be part of and / or integrated into one or more of the CPUs 406 and / or one or more of the GPUs 408, and / or one or more of the logic units 420 may be discrete components or otherwise separate from the CPUs 406 and / or the GPUs 408.In embodiments, one or more of the logic units 420 can be a co-processor of one or more of the CPUs 406 and / or one or more of the GPUs 408.

[0199] Examples of one or more logic units (420) include one or more processing cores and / or components thereof, such as data processing units (DPUs), tensor cores (TCs), tensor processing units (TPUs), pixel visual cores (PVCs), vision processing units (VPUs), graphics processing clusters (GPCs), texture processing clusters (TPCs), streaming multiprocessors (SMs), tree traversal units (TTUs), artificial intelligence accelerators (AIAs), deep learning accelerators (DLAs), arithmetic logic units (ALUs), and application-specific integrated circuits. (Application-Specific Integrated Circuits, ASICs), Floating Point Units (FPUs),Input / output (I / O) elements, peripheral component interconnect (PCI) or PCI Express (PCIe) elements, and / or similar.

[0200] The Communication Interface 410 can include one or more receivers, transmitters, and / or transceivers that enable the Computing Device 400 to communicate with other computers over an electronic network, including wired and / or wireless communication. The Communication Interface 410 can include components and functions that enable communication over a variety of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., Ethernet or InfiniBand communication), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.In one or more embodiments, the one or more logic units 420 and / or the communication interface 410 may contain one or more data processing units (DPUs) to directly transfer data received via a network and / or via the connection system 402 to one or more GPUs 408 (e.g., a memory thereof).

[0201] The I / O ports 412 enable the computing device 400 to be logically coupled with other devices, including the I / O components 414, one or more presentation components 418, and / or other components, some of which may be built into (e.g., integrated with) the computing device 400. Illustrative I / O components 414 include a microphone, mouse, keyboard, joystick, gamepad, game controller, satellite dish, scanner, printer, wireless device, etc. The I / O components 414 can provide a natural user interface (NUI) that processes air gestures, speech, or other physiological inputs generated by a user. In some cases, the inputs can be transmitted to a suitable network element for further processing.A NUI can implement any combination of speech capture, stylus capture, face capture, biometric capture, gesture capture both on-screen and off-screen, air gestures, head and eye tracking, and touch capture (as further described in the present disclosure) associated with a display of the computing device 400. The computing device 400 can include depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof, for gesture capture and recognition. Additionally, the computing device 400 can include accelerometers or gyroscopes (e.g., as part of an inertial measurement unit (IMU)) that enable motion detection. In some examples, the output of the accelerometers or gyroscopes can be used by the computing device 400 to display immersive augmented reality or virtual reality.

[0202] The power supply 416 can include a hardwired power supply, a battery power supply, or a combination thereof. The power supply 416 can supply power to the computing device 400 to enable the operation of the computing device 400's components.

[0203] The one or more presentation components 418 can include a display (e.g., a monitor, a touchscreen, a television screen, a head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The one or more presentation components 418 can receive data from other components (e.g., the one or more GPUs 408, the one or more CPUs 406, etc.) and output the data (e.g., as an image, video, sound, etc.). EXEMPLARY DATA CENTER

[0204] Fig. Figure 5 illustrates an exemplary data center 500 that can be used in at least one embodiment of the present disclosure. The data center 500 can include an infrastructure layer 510, a framework layer 520, a software layer 530, and / or an application layer 540.

[0205] As in Fig. As shown in Figure 5, the infrastructure layer 510 of the data center can contain a resource orchestrator 512, clustered compute resources 514 and node compute resources (“node RRs”) 516(1)516(N), where “N” is any positive integer. In at least one embodiment, the Node-RRs 516(1)516(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic solid-state memory), storage devices (e.g., solid-state or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power supply modules, and / or cooling modules, etc. In some embodiments, one or more Node-RRs mayThe node RRs 516(1)516(N) correspond to a server that has one or more of the computing resources mentioned above. Furthermore, in some embodiments, the node RRs 516(1)516(N) may contain one or more virtual components, such as vGPUs, vCPUs, and / or the like, and / or one or more of the node RRs 516(1)516(N) may correspond to a virtual machine (VM).

[0206] In at least one embodiment, the grouped compute resources 514 can contain separate groupings of node RRs 516, which are housed in one or more racks (not shown) or in many racks in data centers at different geographic locations (also not shown). Separate groupings of node RRs 516 within grouped compute resources 514 can contain grouped compute, network, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node RRs 516, including the CPUs, GPUs, DPUs, and / or other processors, can be grouped in one or more racks to provide compute resources to support one or more workloads.The one or more racks can also contain any number of power supply modules, cooling modules and / or network switches in any combination.

[0207] The resource orchestrator 512 can configure or otherwise control one or more node RRs 516(1)516(N) and / or grouped compute resources 514. In at least one embodiment, the resource orchestrator 512 can include an entity for managing the software design infrastructure (SDI) for the data center 500. The resource orchestrator 512 can include hardware, software, or a combination thereof.

[0208] In at least one embodiment, as in Fig. As shown in Figure 5, the framework layer 520 can contain a job scheduler 532, a configuration manager 534, a resource manager 536, and / or a distributed file system 538. The framework layer 520 can contain a framework that supports the software 532 of the software layer 530 and / or one or more applications 542 of the application layer 540. The software 532 or the one or more applications 542 can each contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 520 can be a type of free and open-source software web application framework, such as Apache Spark™ (hereinafter "Spark"), which can utilize a distributed file system 538 for processing large amounts of data (e.g., "Big Data"), without being limited to it.In at least one embodiment, the job scheduler 532 can include a Spark driver to facilitate the scheduling of workloads supported by different layers of the data center 500. The configuration manager 534 can be capable of configuring different layers, such as the software layer 530 and the framework layer 520, which contains Spark and the distributed file system 538, to support the processing of large amounts of data. The resource manager 536 can be capable of managing clustered or grouped compute resources allocated or assigned to support the distributed file system 538 and the job scheduler 532. In at least one embodiment, the clustered or grouped compute resources can include the grouped compute resource 514 on the infrastructure layer 510 of the data center.The Resource Manager 536 can coordinate with the Resource Orchestrator 512 to manage these allocated or assigned computing resources.

[0209] In at least one embodiment, the software contained in software layer 530 may include software 532 that is used by at least sections of the node RRs 516(1)516(N), the grouped compute resources 514, and / or the distributed file system 538 of framework layer 520. One or more types of software may include, among others, web search software, email virus scanning software, database software, and streaming video content software.

[0210] In at least one embodiment, the applications 542 contained in the application layer 540 may include one or more types of applications used by at least sections of the node RRs 516(1)516(N), the grouped compute resources 514, and / or the distributed file system 538 of the framework layer 520. One or more types of applications may include, but are not limited to, any number of genome applications, cognitive computations, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0211] In at least one embodiment, a configuration manager 534, resource manager 536, and resource orchestrator 512 can implement any number and type of self-modifying actions based on any set and type of data acquired in any technically feasible manner. Self-modifying actions can relieve a data center operator of data center 500 of potentially making poor configuration decisions and potentially avoiding underutilized and / or poorly functioning sections of a data center.

[0212] The data center 500 may contain tools, services, software, or other resources for training one or more machine learning models or for predicting or inferring information using one or more machine learning models according to one or more embodiments described herein. For example, one or more machine learning models may be trained by calculating weighting parameters according to a neural network architecture, using software and / or computing resources described in the present disclosure with reference to the data center 500.In at least one embodiment, trained or deployed machine learning models corresponding to one or more neural networks can be used to infer or predict information using the resources described in the present disclosure with reference to the data center 500 by using weighting parameters calculated by one or more training techniques such as those described herein, without being limited thereto.

[0213] In at least one embodiment, the data center can use 500 CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, and / or other hardware (or equivalent virtual computing resources) to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described in this disclosure can be configured as a service to allow users to train or infer information, such as image capture, speech capture, or other artificial intelligence services. EXEMPLARY NETWORK ENVIRONMENTS

[0214] Network environments suitable for implementing embodiments of the disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may run on one or more instances of the one or more computing devices. Fig. 4. Implemented - e.g., each device may contain similar components, features, and / or functionality to one or more computing devices 400. If backend devices (e.g., servers, NAS, etc.) are implemented, the backend devices may also be included as part of a data center 500, an example of which is given herein with reference to Fig. 5 is described in more detail.

[0215] The components of a network environment can communicate with each other over one or more networks, which can be wired, wireless, or both. The network can contain multiple networks or a network of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the internet and / or a public switched telephone network (PSTN), and / or one or more private networks. If the network includes a wireless telecommunications network, components such as a base station, a communications tower, or even access points (as well as other components) can provide wireless connectivity.

[0216] Compatible network environments can contain one or more peer-to-peer network environments—in which case a server cannot be included in a network environment—and one or more client-server network environments—in which case one or more servers can be included in a network environment. In peer-to-peer network environments, the functionality described here can be implemented on any number of client devices with reference to one or more servers.

[0217] In at least one embodiment, a network environment can include one or more cloud-based network environments, a distributed computing environment, a combination thereof, etc. A cloud-based network environment can include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. A framework layer can include a framework to support software of a software layer and / or one or more applications of an application layer. The software or the one or more applications may each include web-based service software or applications. In embodiments, one or more of the client devices can use the web-based service software or applications (e.g.,by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer can be a type of free and open-source software web application framework that uses, for example, a distributed file system for processing large amounts of data (e.g., "Big Data"), but is not limited to this.

[0218] A cloud-based network environment can provide cloud computing and / or cloud storage, performing any combination (or parts thereof) of the computing and / or data storage functions described herein. Each of these different functions can be distributed across multiple locations of central or core servers (e.g., one or more data centers, which may be distributed across a state, region, country, the globe, etc.). If a connection to a user (e.g., a client device) is relatively close to one or more edge servers, one or more core servers can offload at least some functionality to the one or more edge servers. A cloud-based network environment can be private (e.g., restricted to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0219] The one or more client devices can have at least some of the components, features, and functions of the one or more mentioned here in relation to Fig.The exemplary computing devices described in Section 4 include 400. By way of example, and not as a limitation, a client device may be a personal computer (PC), a laptop, a mobile device, a smartphone, a tablet computer, a smartwatch, a portable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS) or global positioning device, a video player, a video camera, a surveillance device or surveillance system, a vehicle, a boat, a hydrofoil, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or gaming system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, a device, a consumer electronics device, a workstation, an edge device,any combination of these described devices or any other suitable device may be embodied.

[0220] The revelation can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions such as program modules that are executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules contain routines, programs, objects, components, data structures, etc., and refer to code that performs specific tasks or implements certain abstract data types. The revelation can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc.The revelation can also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are connected to each other via a network for communication.

[0221] As used herein, any mention of "and / or" in relation to two or more elements should be interpreted as referring to only one element or combination of elements. For example, "element A, element B and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. Furthermore, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Additionally, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Furthermore, the phrase "based on" should not be interpreted as "based only on" or "based only on."Rather, a first element “based on” a second element contains instances in which the first element is based on the second element, but can also be based on one or more additional elements.

[0222] The subject matter of this disclosure is specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of protection afforded by this disclosure. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways to include various steps or combinations of steps similar to those described in this document, in conjunction with other present or future technologies. Although the terms “step” and / or “block” may be used herein to denote various elements of the methods employed, these terms should not be interpreted as implying any particular sequence among or between the various steps disclosed herein, except where the sequence of each step is expressly described.

[0223] The technology that is the subject of this disclosure is illustrated, for example, by various aspects described below. For the sake of simplicity, several examples of aspects of this disclosure are described as numbered examples (1, 2, 3, etc.). These are provided as examples and do not limit this disclosure. The aspects of the various implementations described herein may be omitted, replaced by aspects of other implementations, or combined with aspects of other implementations, unless the context otherwise requires. For example, one or more aspects of Example 1 below may be omitted, replaced by one or more aspects of another example (e.g., Example 2), or combined with aspects of another example.

[0224] The following is a non-exhaustive presentation of some exemplary implementations listed herein.

[0225] Example 1 system, comprehensive: a first working memory for storing a test image for testing a computer system, wherein the test image contains a multitude of control packages; a second memory for storing at least: a first test configuration that specifies an initial execution sequence, which corresponds to a first set of tax packages from the multitude of tax packages; and a second test configuration that specifies a second execution order corresponding to a second set of control packages from the multitude of control packages; and a hardware controller that is communicatively coupled to the first main memory and the second main memory, wherein the hardware controller serves to instruct the execution of at least one of a first test corresponding to the first test configuration or a second test corresponding to the second test configuration, based on at least the test image and one of the first test configuration or the second test configuration.

[0226] Example 2 System according to Example 1, wherein the first test configuration specifies the execution of the first set of control packages in a first execution order that differs from a standard execution order of the first set of control packages, as specified by the test image.

[0227] Example 3 System according to one of the preceding examples, wherein the execution of the first test includes the hardware controller directing the execution of one or more control packages from the plurality of control packages based on at least one standard execution order of the one or more control packages, as specified by the test image, in response to the fact that the first set of control packages does not contain the one or more control packages.

[0228] Example 4 System according to one of the previous examples, wherein: a working memory location of the second working memory contains a first field and a second field; the first field contains a first entry of the first test configuration, which specifies a first control package of the first set of control packages; the second field contains a second entry of the first test configuration, specifying a second control package of the first set of control packages; and The execution of the first test includes the hardware controller instructing that the second control package be executed immediately after the execution of the first control package, based on at least the first entry and the second entry.

[0229] Example 5: System according to one of the previous examples, wherein the second main memory contains a register bank containing one or more registers.

[0230] Example 6. System according to one of the preceding examples, wherein one or more of the first test or the second test correspond to only a subset of the multitude of control packages.

[0231] Example 7 System according to one of the preceding examples, wherein one or more of the first test or the second test correspond to all control packages of the plurality of control packages.

[0232] Example 8 System according to one of the preceding examples, wherein one or more of the first set of control packages or the second set of control packages contain all the control packages that are executed during the first test or the second test, respectively.

[0233] Example 9 System according to one of the preceding examples, wherein one or more of the first set of control packages or the second set of control packages is a subset of the control packages that are executed during the first test or the second test, respectively.

[0234] Example 10: System according to one of the preceding examples, wherein the system comprises at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulations; a system for conducting collaborative content creation for 3D assets; a system for performing deep learning operations; a system for presenting at least one type of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more real-time streaming applications; a system that is implemented using an edge device; a system that is implemented using a robot; a system for performing operations using conversational AI; a system for performing one or more operations using generative AI; a system that implements one or more large language models (LLMs); a system that implements one or more Vision Language Models (VLMs); a system that implements one or more multimodal language models; a system for generating synthetic data; a system that includes one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources.

[0235] Example 11: System, comprehensive: a working memory for storing a test image for testing a computing system, wherein the test image contains a plurality of control packages arranged sequentially for execution in a linked list; a registry bank containing one or more registers, the registry bank serving the following purpose: Storing an initial test configuration corresponding to an initial test of the computing system and corresponding to an initial set of control packages from the plurality of control packages, wherein the initial test configuration specifies the execution of the initial set of control packages in an initial execution order that differs from the sequential order of the initial set of control packages in the linked list; and a hardware controller that is communicatively coupled to the main memory and the register bank, and which serves to instruct the execution of the first test based at least on the test image and the first test configuration.

[0236] Example 12 System according to Example 11, where: The registry bank also serves the following purposes: Storing a second test configuration corresponding to a second test of the computing system and corresponding to a second set of control packages from the plurality of control packages, wherein the second test configuration specifies the execution of the second set of control packages in a second execution order that differs from the sequential arrangement of the second set of control packages in the linked list; and The hardware controller also serves to instruct the execution of the second test based at least on the test image and the second test configuration.

[0237] Example 13 System according to one of Examples 11 or 12, where the first test corresponds to the following: only a subset of the multitude of tax packages or all tax packages of the multitude of tax packages.

[0238] Example 14 System according to one of Examples 11 to 13, where: the first set of control packages contains all control packages that are executed during the first test; or The first set of control packages is a subset of control packages that are executed during the first test.

[0239] Example 15 System according to one of Examples 11 to 14, wherein the execution of the first test includes the hardware controller directing the execution of one or more control packages from the plurality of control packages based on at least the linked list in response to the fact that the first set of control packages does not contain the one or more control packages.

[0240] Example 16 System according to one of Examples 11 to 15, where: a register of the registry bank contains a first field and a second field; the first field contains a first entry of the first test configuration, which specifies a first control package of the first set of control packages; the second field contains a second entry of the first test configuration, specifying a second control package of the first set of control packages; and The execution of the first test includes the hardware controller instructing that the second control package be executed immediately after the execution of the first control package, based on at least the first entry and the second entry.

[0241] Example 17 Procedure, comprehensive: Accessing a test image used to test a computing system, where the test image specifies a standard execution sequence for a variety of control packages while the computing system is being tested; Accessing a test configuration corresponding to a test of the computing system and corresponding to a set of control packages from the multitude of control packages, wherein the test configuration specifies the execution of the set of control packages in an execution order that differs from the standard execution order; and Performing the test of the computer system, based at least on the test image and the test configuration.

[0242] Example 18 Procedure according to Example 17, wherein the standard execution order is based at least on a sequential arrangement of the multitude of control packages as specified by a linked list contained in the test image.

[0243] Example 19 Procedure according to one of Examples 17 or 18, wherein the test configuration is accessed from a register bank which contains one or more registers and which is loaded with the test configuration.

[0244] Example 20: Procedure according to Example 19, wherein the test image is accessed from a working memory that is loaded with the test image and is separate from the register bank.

[0245] It is understood that the aspects and embodiments described above are purely exemplary and that modifications of details may be made within the scope of protection of the claims.

[0246] Each device, each method and each feature disclosed in the description, and (where applicable) the claims and drawings, may be provided independently or in any suitable combination.

[0247] Reference numerals appearing in the claims are for illustrative purposes only and do not restrict the scope of protection of the claims. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 16 / 101,232

[0126] Cited non-patent literature

[0000] Society of Automotive Engineers, SAE) (Standard No. J3016-201806, published on June 15, 2018, Standard No. J3016-201609, published on September 30, 2016

[0084] Radios, record players, navigation systems, video players, USB

[0181] Node-RRs 516(1)516(N)

[0209] Node-RRs 516(1)516(N

[0210]

Claims

[1] System, encompassing: a first working memory for storing a test image for testing a computer system, wherein the test image contains a multitude of control packages; a second memory for storing at least: a first test configuration that specifies a first execution order corresponding to a first set of control packages from the multitude of control packages; and a second test configuration that specifies a second execution order corresponding to a second set of control packages from the multitude of control packages; and a hardware controller that is communicatively coupled to the first main memory and the second main memory, wherein the hardware controller serves to instruct the execution of at least one of a first test corresponding to the first test configuration or a second test corresponding to the second test configuration, based on at least the test image and one of the first test configuration or the second test configuration. [2] System according to claim 1, wherein the first test configuration specifies the execution of the first set of control packages in a first execution sequence that differs from a standard execution sequence of the first set of control packages as specified by the test image. [3] System according to one of the preceding claims, wherein the execution of the first test includes the hardware controller directing the execution of one or more control packages from the plurality of control packages based on at least one standard execution sequence of the one or more control packages as specified by the test image, in response to the fact that the first set of control packages does not contain the one or more control packages. [4] System according to any one of the preceding claims, wherein: a working memory location of the second working memory contains a first field and a second field; the first field contains a first entry of the first test configuration, which specifies a first control package of the first set of control packages; the second field contains a second entry of the first test configuration, specifying a second control package of the first set of control packages; and The execution of the first test includes the hardware controller instructing that the second control package be executed immediately after the execution of the first control package, based on at least the first entry and the second entry. [5] System according to any of the preceding claims, wherein the second working memory contains a register bank containing one or more registers. [6] System according to any of the preceding claims, wherein one or more of the first test or the second test correspond to only a subset of the plurality of control packages. [7] System according to any of the preceding claims, wherein one or more of the first test or the second test correspond to all control packages of the plurality of control packages. [8] System according to any of the preceding claims, wherein one or more of the first set of control packages or the second set of control packages contain all the control packages that are executed during the first test or the second test, respectively. [9] System according to any of the preceding claims, wherein one or more of the first set of control packages or the second set of control packages is a subset of the control packages that are executed during the first test or the second test, respectively. [10] System according to any one of the preceding claims, wherein the system comprises at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulations; a system for conducting collaborative content creation for 3D assets; a system for performing deep learning operations; a system for presenting at least one type of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more real-time streaming applications; a system that is implemented using an edge device; a system that is implemented using a robot; a system for performing operations using conversational AI; a system for performing one or more operations using generative AI; a system that implements one or more large language models (LLMs); a system that implements one or more Vision Language Models (VLMs); a system that implements one or more multimodal language models; a system for generating synthetic data; a system that includes one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources. [11] System, encompassing: a working memory for storing a test image for testing a computing system, wherein the test image contains a plurality of control packages arranged sequentially for execution in a linked list; a registry bank containing one or more registers, the registry bank serving the following purpose: Storing an initial test configuration corresponding to an initial test of the computing system and corresponding to an initial set of control packages from the plurality of control packages, wherein the initial test configuration specifies the execution of the initial set of control packages in an initial execution order that differs from the sequential order of the initial set of control packages in the linked list; and a hardware controller that is communicatively coupled to the main memory and the register bank, and which serves to instruct the execution of the first test based at least on the test image and the first test configuration. [12] System according to claim 11, wherein: The registry bank also serves the following purposes: Storing a second test configuration corresponding to a second test of the computing system and corresponding to a second set of control packages from the plurality of control packages, wherein the second test configuration specifies the execution of the second set of control packages in a second execution order that differs from the sequential arrangement of the second set of control packages in the linked list; and The hardware controller also serves to instruct the execution of the second test based at least on the test image and the second test configuration. [13] System according to one of claims 11 or 12, wherein the first test corresponds to the following: only a subset of the many tax packages; or all tax packages of the multitude of tax packages. [14] System according to any one of claims 11 to 13, wherein: the first set of control packages contains all control packages that are executed during the first test; or The first set of control packages is a subset of control packages that are executed during the first test. [15] System according to any one of claims 11 to 14, wherein the execution of the first test includes the hardware controller directing the execution of one or more control packages from the plurality of control packages based on at least the linked list in response to the fact that the first set of control packages does not contain the one or more control packages. [16] System according to any one of claims 11 to 15, wherein: a register of the registry bank contains a first field and a second field; the first field contains a first entry of the first test configuration, which specifies a first control package of the first set of control packages; the second field contains a second entry of the first test configuration, specifying a second control package of the first set of control packages; and The execution of the first test includes the hardware controller instructing that the second control package be executed immediately after the execution of the first control package, based on at least the first entry and the second entry. [17] Procedures, including: Accessing a test image used to test a computing system, where the test image specifies a standard execution sequence for a variety of control packages while the computing system is being tested; Accessing a test configuration corresponding to a test of the computing system and corresponding to a set of control packages from the multitude of control packages, wherein the test configuration specifies the execution of the set of control packages in an execution order that differs from the standard execution order; and Performing the test of the computer system, based at least on the test image and the test configuration. [18] Method according to claim 17, wherein the standard execution sequence is based at least on a sequential arrangement of the plurality of control packages as specified by a linked list contained in the test image. [19] Method according to one of claims 17 or 18, wherein the test configuration is accessed from a register bank which contains one or more registers and which is loaded with the test configuration. [20] Method according to claim 19, wherein the test image is accessed from a working memory which is loaded with the test image and which is separate from the register bank.

Images