In-system test architecture for autonomous systems and applications

Abstract

An intra-system test architecture for autonomous systems and applications is disclosed. Embodiments of the present disclosure relate to applications, platforms, architectures, and the like that use a primary test image that can be used for multiple different tests. For example, a test system may include a register bank that may be loaded with a test configuration corresponding to one or more tests. The test configurations may respectively correspond to a set of control packets included in the primary test image, which may be used for or executed for a respective test. The test configuration may indicate an execution order of its respective set of control packets, where the execution order of one or more control packets included in the set of control packets may be different from a default execution order of such control packets as indicated in the primary test image. Thus, this configuration may allow many different tests to be flexibly performed using a single primary test image.

Description

BACKGROUND

[0001] A computing system (e.g., a system on a chip (SoC)) can be subjected to different tests for different operations and functions of the computing system. For example, in-system testing (IST) is a testing scheme that can be used to perform tests related to the computing system, such as structural scan and / or memory testing. Further, IST can include performing tests on the entire computing system (e.g., a complete SoC, multiple SoCs, etc.), such as all functions of the computing system, all operations performed by the computing system, all components of the computing system (including hardware and / or software), etc. Additionally or alternatively, IST can include performing tests on one or more subsets of the computing system, including some of the functions and / or operations that can be performed by the computing system, and / or one or more subsets of the components of the computing system.

[0002] For example, in the context of a vehicle system, IST can be run only during Key-ON (e.g., when the vehicle is in motion or the system is in an “on” state), only during Key-OFF (e.g., when the vehicle is in an “off’ state), or during both Key-ON and Key-OFF. It can be desirable to have the flexibility to choose to run a shorter test (e.g., to test only critical components in the respective SoC) during Key-ON or a full test (e.g., to test the entire SoC) during Key-OFF to keep the latency of the test within an acceptable window, while trading off coverage.

[0003] With existing IST architectures (e.g., existing IST hardware (IST-HW)) and configurations, separate test images are generated for different tests to support the desired flexibility in testing. For example, a test image can typically include control packets that can indicate which operations can be performed for a particular test. Further, the control packets are typically included in the test image as a linked list, where the order of execution from one control packet to another is fixed and determined by the linked list. Thus, this configuration typically requires different test images to be used for different tests. However, the logistics of generating, characterizing, and productizing multiple test images is prohibitive to implement. Further, this approach is difficult to scale as the test image requirements can vary from user to user. SUMMARY

[0004] Embodiments of this disclosure relate to applications, platforms, architectures, etc., for using a master test image that may be used for a variety of different tests. For example, the test system may include a register set that may be loaded with test configurations corresponding to different tests. These test configurations may each correspond to a set of control packets included in the master test image, which may be used for or executed against the respective test. The test configuration may indicate the execution order of its respective set of control packets, wherein the execution order of one or more control packets included in the control packet set may differ from the default execution order of such control packets as indicated in the master test image. Therefore, such a configuration may allow the flexibility to execute a variety of different tests using a single master test image. This flexibility allows for improved in-system testing (IST) capabilities by allowing specific tests to be customized to specific needs without creating a separate test image for each such test. This improvement in testing may also help improve the system under test by allowing greater flexibility in identifying methods where such a system can be improved. Attached Figure Description

[0005] The system and method described below with reference to the accompanying drawings are for in-system testing of autonomous and semi-autonomous systems and applications, wherein:

[0006] Figure 1A An example system configured to perform in-system tests according to one or more embodiments of the present disclosure is shown;

[0007] Figure 1B An example linked list of control packages that may be included in a test image according to one or more embodiments of this disclosure is shown;

[0008] Figure 2 A flowchart is shown illustrating a method for performing in-system tests according to one or more embodiments of the present disclosure;

[0009] Figure 3A This is an illustration of an example autonomous vehicle according to one or more embodiments of the present disclosure;

[0010] Figure 3B According to one or more embodiments of this disclosure Figure 3A Examples of camera positions and fields of view for autonomous vehicles;

[0011] Figure 3C According to one or more embodiments of this disclosure Figure 3A A block diagram of an example system architecture for an example autonomous vehicle;

[0012] Figure 3D A cloud-based server according to one or more embodiments of this disclosure and Figure 3AA system diagram illustrating communication between autonomous vehicles;

[0013] Figure 4 This is a block diagram of an example computing device suitable for implementing one or more embodiments of the present disclosure; and

[0014] Figure 5 This is a block diagram of an example data center applicable to implementing one or more embodiments of the present disclosure. Detailed Implementation

[0015] The systems and methods disclosed herein may involve in-system testing (IST) that may be used by a machine. Typically, IST may use test images for the execution of tests. Test images may include information related to instructions for the execution of the corresponding test, stimuli or other information that may be used for the execution of the test, and / or compute the corresponding expected behavior of the system in response to the corresponding stimuli.

[0016] According to one or more embodiments of this disclosure, the system architecture and methods may be configured to enable the use of a master test image for multiple different tests. In contrast, current IST architectures and techniques typically require specific and separate test images for each test that is expected to be performed.

[0017] The ability to use a master test image instead of different test images for different tests may allow for greater flexibility in testing by simplifying elements that multiple tests might require. Furthermore, using a master test image that may replace multiple different test images may reduce the resources used for IST, for example, potentially reducing the amount of memory used to store test images.

[0018] One or more embodiments of this disclosure may relate to an IST that may be associated with self-machines and / or components of one or more self-machines, which may include any suitable machine or system capable of performing one or more autonomous or semi-autonomous operations. Exemplary self-machines may include, but are not limited to, vehicles (land, sea, space, and / or air), robots, robotic platforms, etc. By way of example, self-machine computing applications may include one or more applications that may be executed by autonomous or semi-autonomous vehicles, such as those for... Figures 3A-3D The example autonomous vehicle 300 described herein may alternatively be referred to as "vehicle 300" or "autonomous machine 300". In this disclosure, references to "autonomous vehicle" or "semi-autonomous vehicle" may include any vehicle that may be configured to perform one or more autonomous or semi-autonomous navigation or driving operations. Therefore, such vehicles may also include those in which an operator is required or in which an operator may also perform such operations.

[0019] Additionally or alternatively, the systems and methods described herein may be used without limitation by non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more adaptive driver assistance systems (ADAS)), autonomous vehicles or machines, manned and unmanned robots or robotic platforms, warehouse vehicles, off-road vehicles, vehicles coupled to one or more trailers, aircraft, boats, shuttles, emergency response vehicles, motorcycles, electric or motorized bicycles, aircraft, engineering vehicles, underwater vehicles, unmanned aerial vehicles, and / or other vehicle types. Furthermore, the systems and methods described herein may be used for a variety of purposes, such as, but not limited to, machine control, machine motion, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or participant simulation and / or digital twins, generative AI, data center processing, conversational AI (such as by employing one or more language models, such as one or more large language models (LLM)), optical transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation for 3D assets, cloud computing and / or any other suitable application.

[0020] The disclosed embodiments may be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, marine systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations (e.g., systems implementing one or more LLMs), systems implementing one or more visual language models (VLMs), systems implementing one or more multimodal language models, systems for performing one or more generative AI operations, systems for hosting real-time streaming applications, systems for presenting one or more of virtual reality content, augmented reality content, or mixed reality content, systems for performing optical transmission simulations, systems for performing collaborative content creation for 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems.

[0021] Embodiments of this disclosure will be described with reference to the accompanying drawings. It should be understood that the drawings are merely illustrative and schematic representations of such exemplary embodiments and are not intended to be limiting, and they are not necessarily drawn to scale. In the drawings, unless otherwise stated, features with the same number represent the same structure and function.

[0022] refer to Figure 1A , Figure 1A An example system 100 configured to perform in-system testing according to one or more embodiments of the present disclosure is shown. Generally, system 100 may include a memory 102, a test module 110, a test controller 104, a register set 106, and one or more units under test 108 (“test unit 108”). It should be understood that such and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, groups of functions, etc.) may be used in addition to or in place of the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in combination with other components, and may be implemented in any suitable combination and location. The various functions performed by entities described herein may be implemented using hardware, firmware, and / or software. For example, various functions may be implemented using a processor that executes instructions stored in memory.

[0023] Test unit 108 may include one or more components of a computing system that may be tested using IST. For example, one or more test units 108 may individually include hardware and / or software components configured to perform one or more tasks or operations, and / or configured to facilitate or promote the execution of one or more tasks or operations by the computing system.

[0024] For example, test unit 108 may include one or more processing devices, one or more memory devices, one or more data storage devices, one or more communication devices, software modules, etc., wherein a single test unit 108 or a collection of one or more test units 108 may be used to compute the system's execution of operations. In some embodiments, test unit 108 may perform operations in various environments, which may include, but are not limited to, autonomous vehicles or machines, vehicle or machine performance, and / or vehicle or machine safety.

[0025] Memory 102 may include any suitable computer-readable storage medium for carrying or storing computer-executable instructions or data structures thereon. Such computer-readable storage media may include any available media that may be accessible to controller 104. For example, but not limited to, such computer-readable storage media may include tangible or non-transitory computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), optical disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices), or any other storage medium that may be used to store specific program code in the form of computer-executable instructions or data structures and may be accessible by a general-purpose or special-purpose computer. Combinations of the above storage media may also be included within the scope of computer-readable storage media.

[0026] Additionally or alternatively, in some embodiments, memory 102 may be part of or comprise of a multimedia card (MMC), which may include flash memory (e.g., NAND flash memory) and a corresponding memory controller. In these and other embodiments, the MMC may be configured as an embedded MMC (eMMC). Additionally or alternatively, memory 102 may be off-chip memory, wherein memory 102 may be decoupled from a chip that may include a test module 110, a test controller 104, a register set 106, and / or a test unit 108.

[0027] In some embodiments, memory 102 may include a master test image 112 (“image 112”) stored thereon. Typically, image 112 may include information that may be used to perform multiple tests and evaluate the results of these tests. Additionally or alternatively, image 112 may be configured to allow the performance of multiple different types of tests. For example, in some embodiments, image 112 may include information that may allow the performance of embedded core tests, memory tests, scan tests (also known as “logic” tests), or any other suitable tests. In these and other embodiments, tests may be implemented according to various protocols or standards, such as via Joint Test Action Group (JTAG) based tests or tests based on the IEEE 1500 standard.

[0028] In these and other embodiments, image 112 may include control packets, data packets, and / or status packets, which may correspond to the execution process of the test. Control packets may contain instructions for decoding and executing the test. Additionally or alternatively, control packets may be arranged sequentially in test image 112, for example, in the form of a linked list. In these and other embodiments, the order of the control packets may indicate the default execution order of the control packets during the execution of the test.

[0029] For example, Figure 1B An example linked list 150 is shown, which may contain “n” control packets as shown in image 112. In the example shown, linked list 150 may indicate the default execution order of the control packets included therein. For example, linked list 150 may indicate a default execution order such that the first control packet (cntrl pkt-1) is executed first, followed by the second control packet (cntrl pkt-2), then the third control packet (cntrl pkt-3), and so on, until the nth control packet (cntrl pkt-n) has been executed.

[0030] Data packets may include inputs and / or stimuli that may be used for the execution of tests. For example, in some embodiments, one or more data packets may include test vectors (e.g., scan test vectors) and data for programming registers to control clock, reset, dynamic function exchange (DFX) control, and / or pad control corresponding to test unit 108.

[0031] The result packet may include the actions performed by test unit 108 during testing based on control packets and / or data packets. For example, in some embodiments, the result packet includes a scan test response or a memory test response.

[0032] In these and other embodiments, the status packet may include a summary of the corresponding test execution. For example, in some embodiments, one or more status packets may indicate whether the test unit 108 passed the test or failed, errors that may have been identified from the test, etc.

[0033] Register group 106 may include any suitable memory that may be accessed by controller 104. For example, in some embodiments, register group 106 may be part of memory separate from controller 104. Alternatively, register group 106 may be a set of registers included in controller 104. Furthermore, although the term "register" is used, it is understood that any suitable memory that may be used in the manner in which register group 106 is described in this disclosure is included within the scope of this disclosure.

[0034] In some embodiments, register group 106 may be loaded with one or more test configurations 114. Each test configuration 114 may correspond to a different test that may be performed against one or more test units 108. In some embodiments, a single test configuration 114 may correspond to a single test. Alternatively, a single test configuration 114 may correspond to a portion of a particular test. In these and other embodiments, a single test configuration 114 may correspond to multiple tests.

[0035] In some embodiments, test configuration 114 may correspond to one or more control package sets included in figure 112, which may be used or executed for tests corresponding to test configuration 114. A single test configuration 114 may indicate the execution order of a particular control package set, wherein the execution order of the control packages included in the control package set differs from the default execution order of such control packages as indicated in figure 112.

[0036] The differences in the execution order indicated by test configuration 114 may include one or more differences from the default execution order indicated by image 112. Generally, in this disclosure, any deviation from the default execution order of all control packages in image 112 may be considered a “difference” in the default execution order. For example, an example execution order difference might include executing only a subset of the control packages included in image 112, even if that subset is executed in the default execution order. Another example execution order difference might include repeatedly executing a specific control package at consecutive or later times. In these and other embodiments, example execution order differences might include, for example, skipping the execution of control packages compared to the default execution order, and / or jumping back to execute earlier included control packages in the default execution order after executing later control packages in the default execution order.

[0037] Please note that in some embodiments, as discussed further in detail in this disclosure, the test corresponding to test configuration 114 may include the execution of more control packages than are included in the control package set included in test configuration 114. In these and other embodiments, control packages that are part of such tests and are not included in the corresponding control package set may be executed according to the default execution order indicated in figure 112.

[0038] In these and other embodiments, register group 106 may include information corresponding to the various test configurations 114 loaded thereon. This information may indicate the execution order of control packages for the tests corresponding to the test configurations 114. For example, each register in register group 106 may include a first field and a second field, each filled with a value indicating the execution order of one or more control packages corresponding to a particular test. Furthermore, or alternatively, the number of registers with filled fields may vary depending on the number of changes in the test execution order from the default order.

[0039] In some cases, the first field may be populated with a first entry indicating a deviation point during test execution where the default execution order may subsequently be deviated from. For example, the first field may include the first specific control package of Figure 112. Including the first specific control package in the first entry may indicate that a deviation from the default execution order may occur after the execution of the first specific control package. Additionally or alternatively, the first field may contain an "Initialize" or "Start" indication, which may indicate that a deviation from the default execution order may occur immediately upon the commencement of the corresponding test.

[0040] Alternatively, the second field may be populated with a second entry indicating the next step to be performed at the deviation point indicated by the corresponding first entry. For example, in some cases, the second field may contain a second specific control package for image 112. Including the second specific control package in the second entry may indicate that the second specific control package will be performed at the deviation point corresponding to the first field (e.g., after the first specific control package if the first specific control package is the first value contained in the first field). In these and other embodiments, the second field may contain a "terminate" or "stop" indication that may indicate the corresponding test will terminate at the deviation point.

[0041] For example, refer to Figure 1B Taking linked list 150 as an example, a specific test configuration 114 corresponding to a specific test may be loaded into register group 106. In these and other embodiments, a specific test may first execute the first control packet (Cntrl Pkt-1), then the second control packet (Cntrl Pkt-2), then two consecutive executions of the fourth control packet (Cntrl Pkt-4), then the seventh control packet (Cntrl Pkt-7), then the third control packet (Cntrl Pkt-3), and then terminate the specific test. To illustrate this, a specific test may be indicated by the following expression (1):

[0042] (1) Start → Ctrl+Pkt-1 → Ctrl+Pkt-2 → Ctrl+Pkt-4 → Ctrl+Pkt-4 → Ctrl+Pkt-5 → Ctrl+Pkt-7 → Ctrl+Pkt-3 → Stop

[0043] Therefore, a particular test may be executed in a variety of ways that differ from the default execution order of Figure 112, such as jumping from the second control pack to the fourth control pack, executing the fourth control pack twice consecutively, jumping from the execution of the fifth control pack to the seventh control pack, jumping from the seventh control pack back to the third control pack, and terminating the particular test after executing the third control pack without executing all the control packs included in Figure 112.

[0044] In some embodiments, registers in register group 106 may be loaded as shown in Table 1 below to reflect a specific test configuration 114 corresponding to a specific test indicated by expression (1).

[0045] Table 1

[0046] Register First field Second field Register-1 Cntrl Pkt-2 Cntrl Pkt-4 Register-2 Cntrl Pkt-4 Cntrl Pkt-4 Register-3 Cntrl Pkt-5 Cntrl Pkt-7 Register-4 Cntrl Pkt-7 Cntrl Pkt-3 Register-5 Cntrl Pkt-3 Stop

[0047] As described above, the first deviation of a particular test from the default execution order indicated by Figure 112 may occur after the execution of the second control package (Cntrl Pkt-2). Furthermore, a deviation from the default execution order may occur after the execution of the second control package, jumping to the execution of the fourth control package (Cntrl Pkt-4).

[0048] As indicated in Table 1, the first register (Register 1) in register set 106 may be filled with entries to reflect changes in execution order. For example, the first register may contain a first field with a first entry indicating a second control packet, indicating that the first deviation occurs after the execution of the second control packet. Additionally or alternatively, the first register may also include a second field with a second entry indicating a fourth control packet. The second entry may indicate that a particular test will jump to the execution of the fourth control packet. Thus, the first and second entries in the first and second fields of the first register, respectively, may indicate that the fourth control packet will be executed after the execution of the second control packet, which is consistent with... Figure 1B The example in Image 112 shown has a different default execution order. Registers 2 through 5 contain entries in the same manner to indicate additional deviations and corresponding deviation types as part of a specific test indicated by Expression 1.

[0049] Register group 106 may include more registers than those shown in Table 1. Table 1 is merely an example to illustrate the registers that may be loaded for a particular test.

[0050] Controller 104 may be communicatively coupled to register set 106 and memory 102. Typically, controller 104 may be configured to perform tests on one or more of test units 108 based on image 112 and a corresponding test configuration 114 loaded into register set 106.

[0051] In some embodiments, controller 104 may include code and routines configured to cause execution of processes relating to the operations described herein. Alternatively or additionally, controller 104 may be implemented using hardware including one or more processors, CPUs, graphics processing units (GPUs), data processing units (DPUs), parallel processing units (PPUs), microprocessors (e.g., for performing or controlling the execution of one or more operations), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), accelerators (e.g., deep learning accelerators (DLAs)), one or more programmable vision accelerators (PVAs) (which may include one or more vector processing units (VPUs)), one or more direct memory access (DMA) systems, one or more pixel processing engines (PPEs), and / or other processor types. In these and other embodiments, controller 104 may be implemented using a combination of hardware and software. In this disclosure, the operations performed by controller 104 as described may include operations that controller 104 may direct to be performed by a corresponding computing system. In these and other embodiments, controller 104 may be implemented by one or more computing devices, such as references Figures 3A-3D , Figure 4 and / or Figure 5 The computing device is described in further detail.

[0052] In some embodiments, controller 104 may be configured to execute control packages for testing by following the default order contained in figure 112, unless otherwise indicated by the entry in register group 106 corresponding to the corresponding test configuration 114 for the respective test.

[0053] For example, referring to expression (1) and a specific test in Table 1, in some embodiments, upon initializing a specific test, controller 104 may load information from register 1 in register set 106 into execution register 116. Execution register 116 may include any memory that might be used for temporary storage of information that controller 104 may reference as part of the current test operation being performed by controller 104. In some embodiments, execution register 116 may be separate from register set 106. Alternatively, execution register 116 may be part of register set 106. In these or other embodiments, register 1 may be used as execution register 116, such that when register 1 is loaded, execution register 116 may be loaded from register 1.

[0054] Additionally or alternatively, after the execution register 116 is loaded, the entries of the registers in register set 106 may be shifted up one register. Table 2 shows an example of the register entries of register set 106 and execution register 116 after the execution register 116 is loaded during the initialization of a particular test, where the entries have been shifted up one register compared to Table 1. The examples in Table 2 are for an embodiment where execution register 116 is separate from register 1. However, it is understood that these specific implementation details are used to illustrate the general principles described herein and are not intended to be limiting. Furthermore, note that register 5 is not described in Table 2. In some cases, after the execution register 116 is loaded, register 5 may now have no entries. Additionally or alternatively, in some cases, register 5 may be loaded with additional indications related to another test configuration corresponding to another test.

[0055] Table 2

[0056] Register First field Second field Execute register Cntrl Pkt-2 Cntrl Pkt-4 Register-1 Cntrl Pkt-4 Cntrl Pkt-4 Register-2 Cntrl Pkt-5 Cntrl Pkt-7 Register-3 Cntrl Pkt-7 Cntrl Pkt-3 Register-4 Cntrl Pkt-3 Stop

[0057] Controller 104 may use information in execution register 116 to determine how to perform a particular test. For example, the default execution order indicates that the first control package should be executed first. Controller 104 may first check the first entry of the first field of execution register 116 to determine if there is a deviation point at the start of execution of the particular test. For example, controller 104 may determine whether to execute control packages other than the first control package first based on whether the first field contains an "initialization" or "start" entry (which indicates that the first deviation point is at initialization). In the example shown, the first field of execution register 116 indicates a deviation point after the execution of the second control package, rather than at the initialization of the particular test, such that controller 104 may start the particular test by executing the first control package as indicated in the default execution order.

[0058] After the first phase (e.g., after the execution of the first control package) and before the execution of the second phase of the specific test, controller 104 may again check execution register 116 to determine whether a deviation from the default execution order will occur after the execution of the first control package. In the example shown in Table 2, the first entry in the first field of execution register 116 may still indicate that the deviation point is after the execution of the second control package, rather than after the execution of the first control package. Therefore, controller 104 may be configured to execute the second phase of the specific test by executing the second control package as indicated by the default execution order.

[0059] After the second phase (e.g., after the execution of the second control package) and before the execution of the third phase of a particular test, controller 104 may re-examine execution register 116 to determine whether a deviation from the default execution order will occur after the execution of the second control package. As previously stated, in the example shown in Table 2, the first entry in the first field of execution register 116 may still indicate a deviation point after the execution of the second control package. Therefore, since the second control package has just completed execution, and based on the first field of execution register 116 indicating the second control package, controller 104 may be configured to recognize that the next control package to be executed may differ from the default execution order. In these and other embodiments, controller 104 may read the second entry in the second field of execution register 116 to determine which control package to execute next. In the example shown, since a fourth control package is indicated in the second field of execution register 116, controller 104 may execute the third phase of a particular test by executing the fourth control package.

[0060] In these and other embodiments, at some point before the completion of the third stage of execution, the entries in execution register 116 may be updated with the entries in register 1 of register set 106, and other entries in register set 1 may be shifted and moved up. Table 3 shows an example of the register entries in register set 106 and execution register 116 after updating execution register 116 for the third execution stage. Furthermore, note that register 4 is shown as blank in Table 3 due to the shifting of register entries. Additionally, or alternatively, in some cases, register 4 may be loaded with additional indications related to another test configuration corresponding to another test.

[0061] Table 3

[0062] Register First field Second field Execute register Cntrl Pkt-4 Cntrl Pkt-4 Register-1 Cntrl Pkt-5 Cntrl Pkt-7 Register-2 Cntrl Pkt-7 Cntrl Pkt-3 Register-3 Cntrl Pkt-3 Stop Register-4

[0063] After the third phase (e.g., after the execution of the fourth control package) and before the execution of the fourth phase of a specific test, controller 104 may re-check execution register 116 to determine whether a deviation from the default execution order will occur after the execution of the fourth control package. In the example shown in Table 3, the first entry in the first field of execution register 116 may indicate a deviation point that occurred after the execution of the fourth control package. Therefore, since the fourth control package has just completed execution, and based on the indication of the fourth control package in the first field of execution register 116, controller 104 may be configured to identify, based on the indication of the fourth control package in the second field of execution register 116, that the next control package to be executed may again be the fourth control package. In the example shown, controller 104 may therefore execute the fourth phase of a specific test by repeatedly executing the fourth control package.

[0064] In these and other embodiments, at some point before the completion of the fourth stage of execution, the entry of register 116 in register set 106 may be updated using the entry of register 1, and other entries in the register set may be shifted and moved up. Table 4 shows an example of the register entries in register set 106 and execution register 116 after updating execution register 116 for the fourth execution stage. Furthermore, note that registers 3 and 4 are shown as blank in Table 4 due to shifting in the register entries. Additionally, or alternatively, in some cases, one or more of registers 3 or 4 may be loaded with additional indications related to another test configuration corresponding to another test.

[0065] Table 4

[0066]

[0067]

[0068] After the fourth phase (e.g., after the second execution of the fourth control package) and before the fifth phase of the specific test, controller 104 may recheck execution register 116 to determine if a deviation from the default execution order will occur after the execution of the fourth control package. In the example shown in Table 4, the first entry in the first field of execution register 116 may indicate a deviation point that occurred after the execution of the fifth control package. Therefore, since the fourth control package has just finished execution, controller 104 may revert to the default execution order indicated in linked list 150 of Figure 112. The default execution order may indicate that the fifth control package will be executed after the fourth control package. In the example shown, controller 104 may therefore execute the fifth phase of the specific test by executing the fifth control package.

[0069] After the fifth phase (e.g., after the execution of the fifth control package) and before the execution of the sixth phase of the specific test, controller 104 may recheck execution register 116 to determine whether a deviation from the default execution order will occur after the execution of the fifth control package. In the example shown in Table 4, the first entry in the first field of execution register 116 may indicate a deviation point that occurred after the execution of the fifth control package. Therefore, since the fifth control package has just finished executing, controller 104 may refer to the second entry in the second field of execution register 116. Thus, controller 104 may determine that the next control package to be executed is likely the seventh control package. In the example shown, controller 104 may therefore execute the sixth phase of the specific test by executing the seventh control package.

[0070] In these and other embodiments, at some point before the completion of the sixth stage of execution, the entries in execution register 116 may be updated with the entries in register 1 of register set 106, and other entries in register set 106 may be shifted and moved up. Table 5 shows an example of the register entries in register set 106 and execution register 116 after updating execution register 116 for the sixth execution stage. In the example shown in Table 5, registers 2, 3, and 4 are empty due to the shifting in the register entries. Additionally or alternatively, in some cases, one or more of registers 2, 3, or 4 may be loaded with additional indications related to another test configuration corresponding to another test.

[0071] Table 5

[0072]

[0073]

[0074] After the sixth phase (e.g., after the execution of the seventh control package) and before the execution of the seventh phase of a specific test, controller 104 may recheck execution register 116 to determine if a deviation from the default execution order will occur after the execution of the seventh control package. In the example shown in Table 5, the first entry in the first field of execution register 116 may indicate a deviation point that occurred after the execution of the seventh control package. Therefore, since the seventh control package has just finished executing, controller 104 may refer to the second entry in the second field of execution register 116. Thus, controller 104 may determine that the next control package to be executed is likely the third control package. In the example shown, controller 104 may therefore execute the seventh phase of the specific test by executing the third control package.

[0075] In these and other embodiments, at some point before the completion of the seventh stage of execution, the entry of register 116 in register set 106 may be updated using the entry of register 1, and other entries in the register set may be shifted and moved up. Table 6 shows an example of the register entries in register set 106 and execution register 116 after updating register 116 for the seventh execution stage. In the example shown in Table 6, registers 1, 2, 3, and 4 in register set 106 are empty due to the shifting in the register entries. Additionally or alternatively, in some cases, one or more of registers 1, 2, 3, or 4 may be loaded with additional indications related to another test configuration corresponding to another test.

[0076] Table 6

[0077] Register First field Second field Execute register Cntrl Pkt-3 Stop Register-1 Register-2 Register-3 Register-4

[0078] After the seventh phase (e.g., after the execution of the third control package) and before the execution of the seventh phase of a specific test, controller 104 may recheck execution register 116 to determine if a deviation from the default execution order will occur after the execution of the seventh control package. In the example shown in Table 6, the first entry in the first field of execution register 116 may indicate a deviation point that occurred after the execution of the third control package. Therefore, since the third control package has just finished executing, controller 104 may refer to the second entry in the second field of execution register 116. Thus, controller 104 may determine that the test will stop due to the "stop" indication included in the second field. In the example shown, controller 104 may not execute any more control packages, and therefore may terminate the specific test.

[0079] Test module 110 may include code and routines configured to cause controller 104 to perform execution procedures for one or more test-related operations as described herein. Alternatively or additionally, test module 110 may be implemented in hardware, including one or more processors, CPUs, graphics processing units (GPUs), data processing units (DPUs), parallel processing units (PPUs), microprocessors (e.g., for performing or controlling the execution of one or more operations), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), accelerators (e.g., deep learning accelerators (DLAs)), one or more programmable vision accelerators (PVAs) (which may include one or more vector processing units (VPUs)), one or more direct memory access (DMA) systems, one or more pixel processing engines (PPEs), and / or other processor types. In these and other embodiments, test module 110 may be implemented using a combination of hardware and software. In this disclosure, operations performed by test module 110 may include operations that test module 110 may instruct a corresponding computing system to perform. In these or other embodiments, the test module 110 may be implemented by one or more computing devices, such as those for... Figures 3A-3D , Figure 4 and / or Figure 5 A computing device described in further detail.

[0080] Typically, test module 110 may be configured to instruct controller 104 on test management of one or more test units 108. For example, test module 110 may be configured to load register set 106 with certain test configurations 114. In these and other embodiments, test module 110 may be configured to receive test configurations 114 as input. Additionally or alternatively, test module 110 may be configured to generate one or more test configurations 114 based on received input (e.g., received user input regarding certain parameters to be tested for a particular test and / or regarding which control packages to execute and the order in which they are executed for each test).

[0081] In these and other embodiments, the test module 110 may be configured to instruct the controller 104 about when to begin executing a test. Additionally or alternatively, the memory 102 may include multiple different test images 112 stored thereon. In these and other embodiments, the test module 110 may instruct the controller from which test image 112 might extract data for a specific test.

[0082] Without departing from the scope of this disclosure, it may be possible to Figure 1A and Figure 1BModifications, additions, or omissions may be made to the descriptions and related information. For example, in some embodiments, system 100 may include any number of units under test. Alternatively or additionally, some embodiments may include implementations different from those described herein. For example, in some embodiments, an entry in the first field of a register may indicate whether a deviation occurs before, rather than after, the control package indicated in a particular entry. Alternatively or additionally, in some embodiments, system 100 may include any number of other components, operations, or inputs that may not be explicitly stated or described. Furthermore, while the above description focuses primarily on the execution of the control package in the test execution process, many other operations (e.g., as instructed by the control package) may be performed for such tests, and these operations are not described herein.

[0083] Furthermore, although this specification and specific examples involve loading a single test configuration 114 in register group 106, in some cases, multiple test configurations 114 may be loaded simultaneously in register group 106. Alternatively, different test configurations 114 may be loaded in register group 106 in a sequential order.

[0084] Furthermore, in some cases, register set 106 may not be large enough to indicate all the different deviations from the default execution order associated with a particular test. In some embodiments, sub-test configurations of multiple test configurations 114 or larger corresponding to the same test may be generated and loaded sequentially into register set 106. In these and other embodiments, the next test configuration 114 (or the next sub-test configuration) following the currently used current test configuration 114 (or current sub-test configuration) may be loaded in segments as registers in register set 106 become available. Additionally or alternatively, the next test configuration 114 (or the next sub-test configuration) may be loaded into register set 106 after all entries corresponding to the previous test configuration 114 (or previous sub-test configuration) have been cleared.

[0085] Furthermore, the number of control packages executed during testing, or which control packages might be executed during testing, may vary. For example, in some cases, all control packages contained in image 112 may be executed during a particular test. Alternatively, in some cases, only a subset of the control packages contained in image 112 may be executed during a particular test.

[0086] In these and other embodiments, the control packages that may be included in the control package set corresponding to different test configurations may differ. For example, in some cases, a specific control package set corresponding to a particular test configuration 114 may include all control packages executed during a particular test (e.g., in the case where no control packages are executed according to the default execution order). Alternatively, the specific control package set corresponding to a particular test configuration 114 may include a subset of the control packages executed during a particular test (e.g., in the case where at least some control packages are executed according to the default execution order).

[0087] Figure 2 This is a flowchart illustrating a method 200 for performing in-system tests according to one or more embodiments of the present disclosure. One or more operations of method 200 may be performed by any suitable system, apparatus, or device, such as one or more components of system 100 as shown in FIG1 of the present disclosure (e.g., test controller 104), targeting... Figures 3A-3D Describing one or more autonomous vehicle systems, targeting Figure 4 Described one or more computing devices and / or targeting Figure 5 Describe one or more data systems.

[0088] Method 200 may include block B202, where a test image that may be used to test a computing system may be accessed. In some embodiments, the test image may indicate a default execution order for executing control packages used to perform tests on the computing system. For example, the default execution order may be based at least on the order of control packages as indicated by a linked list of control packages included in the test image—for example, as described in this disclosure. Figure 1B As shown. For Figure 1A and Figure 1B The described test image 112 may be an example of a test image that may be accessed. In some embodiments, the test controller (e.g., Figure 1A The test controller 104 may access the test image.

[0089] Alternatively, in some embodiments, the test image may be stored in and accessed from a memory located remotely from the test controller (e.g., on a chip separate from the chip that may include the test controller). In these and other embodiments, the test image may be stored in and accessed from a memory that is local to the test controller (e.g., on the same chip that includes the test controller).

[0090] At box B204, a test configuration corresponding to a test of the computing system may be accessed. This test configuration may correspond to a set of control packages contained in a test image. In some embodiments, this set of control packages may include all control packages included in the test image. Alternatively, the set of control packages may include only a subset of all control packages contained in the test image. The test configuration may instruct the set of control packages to be executed in an order different from the default execution order. For example, for... Figure 1A and Figure 1B The described test configuration 114 may serve as an example of a test configuration.

[0091] In some embodiments, the test configuration may be accessed from memory that is the same as the memory from which the test image is accessed. Alternatively, in some embodiments, the test configuration may be accessed from memory that is different from the memory from which the test image is accessed. For example, in some embodiments, the test configuration may be stored in a register set (such as for...). Figure 1A and Figure 1B The register set 106 described is accessed and resides on the register set. In these and other embodiments, the register set may be an internal register set of the test controller. Alternatively, the register set may be located external to the test controller.

[0092] In box B206, tests of the computing system may be performed, at least based on test images and test configurations. In some embodiments, tests such as those described in this disclosure may be performed. Figure 1A and Figure 1B And tests such as the test execution procedures described in Tables 1-6. Therefore, method 200 may be used to perform one or more ISTs according to one or more embodiments of this disclosure.

[0093] Modifications, additions, or omissions may be made to method 200 without departing from the scope of this disclosure. For example, although the individual boxes of method 200 are shown as discrete boxes, they may be split into more boxes, merged into fewer boxes, or deleted, depending on the specific implementation.

[0094] Furthermore, in some embodiments, method 200 may be used to perform multiple different tests based on the same test image and multiple different test configurations. For example, in some embodiments, a first test may be performed on the computing system based on the test image and a first test configuration. Alternatively, a second test may be performed on the computing system based on the test image and a second test configuration.

[0095] Example autonomous vehicles

[0096] Figure 3AThis is an illustration of an example autonomous vehicle 300 according to some embodiments of the present disclosure. The autonomous vehicle 300 (or, alternatively, referred to herein as “vehicle 300”) may include, but is not limited to, passenger vehicles such as automobiles, trucks, buses, ambulances, shuttles, electric or motorized bicycles, motorcycles, fire trucks, police cars, ambulances, boats, engineering vehicles, underwater vessels, drones, and / or other types of vehicles (e.g., driverless and / or capable of accommodating one or more passengers). Autonomous vehicles are generally described according to the level of automation defined by the National Highway Traffic Safety Administration (NHTSA), a division of the U.S. Department of Transportation, and the Society of Automotive Engineers (SAE) “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles” (Standard No. J3016-201806, published June 15, 2018; Standard No. J3016-201609, published September 30, 2016; and previous and future versions of this standard). Vehicle 300 is capable of performing one or more functions that meet Level 3-5 of autonomous driving standards. Vehicle 300 is capable of performing one or more functions that meet Level 1-5 of automated driving standards. For example, depending on the embodiment, vehicle 300 is capable of driver assistance (Level 1), partial automation (Level 2), conditional automation (Level 3), high automation (Level 4), and / or full automation (Level 5). The term "autonomy" as used herein can include any and / or all types of autonomy for vehicle 300 or other machines, such as full autonomy, high autonomy, conditional autonomy, partial autonomy, providing assisted autonomy, semi-autonomy, primary autonomy, or other names.

[0097] Vehicle 300 may include components such as chassis, body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other vehicle components. Vehicle 300 may include a propulsion system 350, such as an internal combustion engine, a hybrid power plant, an all-electric motor, and / or another type of propulsion system. Propulsion system 350 may be connected to the drivetrain of vehicle 300, which may include a transmission, to enable propulsion of vehicle 300. Propulsion system 350 may be controlled in response to receiving a signal from throttle / accelerator 352.

[0098] A steering system 354, which may include a steering wheel, can be used to steer the vehicle 300 (e.g., along a desired path or route) when the propulsion system 350 is operating (e.g., when the vehicle is in motion). The steering system 354 may receive signals from the steering actuator 356. For fully automatic (level 5) functionality, the steering wheel may be optional.

[0099] The brake sensor system 346 can be used to operate the vehicle brakes in response to receiving signals from the brake actuator 348 and / or the brake sensor.

[0100] It may include one or more CPUs, System-on-a-Chip (SoC) 304 ( Figure 3C One or more controllers 336, including and / or one or more GPUs, may provide signals (e.g., signals representing commands) to one or more components and / or systems of vehicle 300. For example, one or more controllers may send signals to operate vehicle brakes via one or more brake actuators 348, to operate steering system 354 via one or more steering actuators 356, and / or to operate propulsion system 350 via one or more throttles / accelerators 352. One or more controllers 336 may include one or more onboard (e.g., integrated) computing devices (e.g., supercomputers) that process sensor signals and output operating commands (e.g., signals representing commands) to enable autonomous driving and / or assist a human driver in driving vehicle 300. One or more controllers 336 may include a first controller 336 for autonomous driving functions, a second controller 336 for functional safety functions, a third controller 336 for artificial intelligence functions (e.g., computer vision), a fourth controller 336 for infotainment functions, a fifth controller 336 for redundancy in emergency situations, and / or other controllers. In some examples, a single controller 336 can handle two or more of the functions described above, and two or more controllers 336 can handle a single function, and / or any combination thereof.

[0101] One or more controllers 336 may provide signals for controlling one or more components and / or systems of vehicle 300 in response to sensor data (e.g., sensor inputs) received from one or more sensors. Sensor data may be received from, for example, but not limited to, global navigation satellite system sensors 358 (e.g., GPS sensors), RADAR sensors 360, ultrasonic sensors 362, LIDAR sensors 364, inertial measurement unit (IMU) sensors 366 (e.g., accelerometers, gyroscopes, magnetic compasses, magnetometers, etc.), microphones 396, stereo cameras 368, wide-angle cameras 370 (e.g., fisheye cameras), infrared cameras 372, surround cameras 374 (e.g., 360-degree cameras), long-range and / or medium-range cameras 398, speed sensors 344 (e.g., for measuring the rate of vehicle 300), vibration sensors 342, steering sensors 340, braking sensors (e.g., as part of braking sensor system 346), and / or other sensor types.

[0102] One or more of the controllers 336 may receive input (e.g., represented by input data) from the instrument panel 332 of the vehicle 300 and provide output (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 334, an auditory signaling device, a speaker, and / or via other components of the vehicle 300. These outputs may include information such as vehicle speed, rate, time, map data (e.g., [missing information]). Figure 3C Information such as the HD map 322, location data (e.g., the location of vehicle 300 on the map), direction, and the location of other vehicles (e.g., occupying a grid), as well as information about objects and their states perceived by controller 336, etc. For example, HMI display 334 may display information about the existence of one or more objects (e.g., street signs, warning signs, traffic light changes, etc.) and / or information about driving maneuvers that the vehicle has made, is making, or will make (e.g., changing lanes now, leaving 34B in two miles, etc.).

[0103] The vehicle 300 further includes a network interface 324, which can communicate via one or more networks using one or more wireless antennas 326 and / or a modem. For example, the network interface 324 may be able to communicate via LTE, WCDMA, UMTS, GSM, CDMA2000, etc. The one or more wireless antennas 326 may also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.) using one or more local area networks such as Bluetooth, Bluetooth LE, Z-Wave, ZigBee, etc., and / or one or more low-power wide area networks (LPWANs) such as LoRaWAN, SigFox, etc.

[0104] Figure 3B For use in accordance with some embodiments of this disclosure Figure 3A This is an example of the camera position and field of view of an example autonomous vehicle 300. The camera and its respective field of view are an example embodiment and are not intended to be limiting. For example, additional and / or replaceable cameras may be included, and / or these cameras may be located at different positions on the vehicle 300.

[0105] The camera type used for the camera may include, but is not limited to, a digital camera suitable for use with components and / or systems of vehicle 300. The camera may operate at Automotive Safety Integrity Level (ASIL) B and / or another ASIL. The camera type may have any image capture rate, such as 60 frames per second (fps), 120 fps, 240 fps, etc., depending on the embodiment. The camera may be able to use a rolling shutter, a global shutter, another type of shutter, or a combination thereof. In some examples, the color filter array may include a red-white-white-white (RCCC) color filter array, a red-white-white-blue (RCCB) color filter array, a red-blue-green-white (RBGC) color filter array, a Foveon X3 color filter array, a Bayer sensor (RGGB) color filter array, a monochrome sensor color filter array, and / or another type of color filter array. In some embodiments, a sharp-pixel camera, such as a camera with RCCC, RCCB, and / or RBGC color filter arrays, may be used in efforts to improve light sensitivity.

[0106] In some examples, one or more of the cameras can be used to perform advanced driver assistance system (ADAS) functions (e.g., as part of a redundant or fail-safe design). For example, a multi-function monocular camera can be installed to provide functions including lane departure warning, traffic sign assistance, and intelligent headlight control. One or more of the cameras (e.g., all cameras) can simultaneously record and provide image data (e.g., video).

[0107] One or more of the cameras can be mounted in mounting components such as custom-designed (3-D printed) parts to cut off stray light and reflections from inside the vehicle (e.g., reflections from the dashboard in the windshield mirror) that may interfere with the camera's image data capture capabilities. Regarding the wing mirror mounting components, the wing mirror components can be custom-3-D printed so that the camera mounting plate matches the shape of the wing mirror. In some examples, one or more cameras can be integrated into the wing mirror. For side-view cameras, one or more cameras can also be integrated into the four pillars at each corner of the cab.

[0108] A camera with a field of view that includes the environment in front of the vehicle 300 (e.g., a front-facing camera) can be used for surround view to help identify forward paths and obstacles, and, with the assistance of one or more controllers 336 and / or control SoCs, to provide information crucial for generating an occupancy grid and / or determining the preferred vehicle path. The front-facing camera can be used to perform many of the same ADAS functions as LiDAR, including emergency braking, pedestrian detection, and collision avoidance. The front-facing camera can also be used in ADAS functions and systems, including Lane Departure Warning (LDW), Autonomous Cruise Control (ACC), and / or other functions such as traffic sign recognition.

[0109] A variety of cameras can be used in front-facing configurations, including, for example, monocular camera platforms including CMOS (Complementary Metal-Oxide-Semiconductor) color imagers. Another example could be a wide-angle camera 370, which can be used to perceive objects entering the field of view from the periphery (such as pedestrians, traffic at intersections, or bicycles). Although Figure 3B The middle image shows only one wide-angle camera, but any number of wide-angle cameras 370 can be present on the vehicle 300. Furthermore, remote cameras 398 (e.g., a pair of long-view stereo cameras) can be used for depth-based object detection, especially for objects for which neural networks have not yet been trained. Remote cameras 398 can also be used for object detection and classification, as well as basic object tracking.

[0110] One or more stereo cameras 368 may also be included in a front-mounted configuration. The stereo camera 368 may include an integrated control unit comprising a scalable processing unit that can provide a multi-core microprocessor and programmable logic (FPGA) with an integrated CAN or Ethernet interface on a single chip. Such a unit can be used to generate a 3D map of the vehicle environment, including distance estimates for all points in the image. Alternative stereo cameras 368 may include a compact stereo vision sensor that may include two camera lenses (one on each side) and an image processing chip capable of measuring the distance from the vehicle to a target object and using the generated information (e.g., metadata) to activate autonomous emergency braking and lane departure warning functions. Other types of stereo cameras 368 may be used in addition to those described herein, or alternatively.

[0111] Cameras with a field of view including the side portion of the vehicle 300 (e.g., side-view cameras) can be used for surround view, providing information for creating and updating occupancy grids and generating side-impact collision warnings. For example, surround camera 374 (e.g., ... Figure 3B The four surround cameras 374 shown can be mounted on the vehicle 300. The surround cameras 374 can include wide-angle cameras 370, fisheye cameras, 360-degree cameras, and / or the like. For example, four fisheye cameras can be positioned at the front, rear, and sides of the vehicle. In an alternative arrangement, the vehicle can use three surround cameras 374 (e.g., left, right, and rear) and can utilize one or more other cameras (e.g., forward-facing cameras) as a fourth surround-view camera.

[0112] A camera with a field of view that includes the environment behind the vehicle 300 (e.g., a rear-view camera) can be used for parking assistance, surround view, rear collision warning, and creating and updating occupancy grids. A wide variety of cameras can be used, including but not limited to those also suitable as front-facing cameras as described herein (e.g., long-range and / or mid-range camera 398, stereo camera 368, infrared camera 372, etc.).

[0113] Figure 3C For use in accordance with some embodiments of this disclosure Figure 3A The example autonomous vehicle 300 is illustrated in the block diagram of an example system architecture. It should be understood that this arrangement, and other arrangements described herein, are merely illustrative. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to or in place of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities, which may be implemented as discrete or distributed components or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by these entities can be implemented via hardware, firmware, and / or software. For example, the various functions can be implemented by a processor executing instructions stored in memory.

[0114] Figure 3C Each component, feature, and system in vehicle 300 is illustrated as being connected via bus 302. Bus 302 may include a Controller Area Network (CAN) data interface (or, alternatively, referred to herein as the "CAN bus"). CAN may be a network within vehicle 300 used to assist in the control of various features and functions of vehicle 300, such as the actuation of brakes, acceleration, braking, steering, windshield wipers, etc. The CAN bus may be configured to have dozens or even hundreds of nodes, each with its own unique identifier (e.g., CAN ID). The CAN bus can be read to find steering wheel angle, ground speed, engine speed per minute (RPM), button positions, and / or other vehicle status indicators. The CAN bus may be ASIL B compliant.

[0115] Although bus 302 is described herein as a CAN bus, this is not intended to be limiting. For example, FlexRay and / or Ethernet may be used in addition to or alternatively to a CAN bus. Furthermore, although bus 302 is represented by a single line, this is not intended to be limiting. For example, any number of buses 302 may exist, which may include one or more CAN buses, one or more FlexRay buses, one or more Ethernet buses, and / or one or more other types of buses using different protocols. In some examples, two or more buses 302 may be used to perform different functions and / or may be used for redundancy. For example, a first bus 302 may be used for a collision avoidance function, and a second bus 302 may be used for drive control. In any example, each bus 302 may communicate with any component of vehicle 300, and two or more buses 302 may communicate with the same component. In some examples, each SoC 304, each controller 336, and / or each computer within the vehicle may have access to the same input data (e.g., input from sensors of vehicle 300) and may be connected to a common bus such as a CAN bus.

[0116] Vehicle 300 may include one or more controllers 336, such as those described herein. Figure 3A The controllers described herein. Controller 336 can be used for a wide variety of functions. Controller 336 can be coupled to any other different components and systems of vehicle 300 and can be used for the control of vehicle 300, artificial intelligence of vehicle 300, infotainment and / or the like for vehicle 300.

[0117] Vehicle 300 may include one or more System-on-a-Chip (SoC) 304. SoC 304 may include a CPU 306, GPU 308, processor 310, cache 312, accelerator 314, data storage 316, and / or other components and features not shown. SoC 304 can be used to control vehicle 300 across a wide variety of platforms and systems. For example, one or more SoCs 304 may be combined with an HD map 322 in a system (e.g., the system of vehicle 300), the HD map being transmitted via a network interface 324 from one or more servers (e.g., [server name missing]). Figure 3D One or more servers (378) receive map refresh and / or updates.

[0118] CPU 306 may include a CPU cluster or CPU complex (or, alternatively, referred to herein as "CCPLEX"). CPU 306 may include multiple cores and / or L2 cache. For example, in some embodiments, CPU 306 may include eight cores in a coherent multiprocessor configuration. In some embodiments, CPU 306 may include four dual-core clusters, each cluster having a dedicated L2 cache (e.g., 2MB L2 cache). CPU 306 (e.g., CCPLEX) may be configured to support simultaneous cluster operation, such that any combination of clusters of CPU 306 can be active at any given time.

[0119] CPU 306 can implement power management capabilities including one or more of the following features: automatic clock gating of hardware blocks when idle to conserve dynamic power; clock gating of each core when the core is not actively executing instructions due to the execution of WFI / WFE instructions; independent power gating of each core; independent clock gating of each core cluster when all cores are clock-gated or power-gated; and / or independent power gating of each core cluster when all cores are power-gated. CPU 306 can further implement enhanced algorithms for managing power states, wherein allowed power states and desired wake-up times are specified, and the hardware / microcode determines the optimal power state to enter for the core, cluster, and CCPLEX. The processing core can support simplified power state entry sequences in software, with this work offloaded to the microcode.

[0120] GPU 308 may include an integrated GPU (or, alternatively, referred to herein as an "iGPU"). GPU 308 may be programmable and efficient for parallel workloads. In some examples, GPU 308 may use an enhanced tensor instruction set. GPU 308 may include one or more streaming microprocessors, wherein each streaming microprocessor may include an L1 cache (e.g., an L1 cache with at least 96KB of storage capacity), and two or more of these streaming microprocessors may share an L2 cache (e.g., an L2 cache with 512KB of storage capacity). In some embodiments, GPU 308 may include at least eight streaming microprocessors. GPU 308 may use a computation application programming interface (API). Furthermore, GPU 308 may use one or more parallel computing platforms and / or programming models (e.g., NVIDIA's CUDA).

[0121] In automotive and embedded applications, the GPU 308 can be power-optimized for optimal performance. For example, the GPU 308 can be fabricated on FinFETs. However, this is not intended to be limiting, and the GPU 308 can be fabricated using other semiconductor manufacturing processes. Each streaming microprocessor can combine several mixed-precision processing cores divided into multiple blocks. For example, and without limitation, 64 FP32 cores and 32 FP64 cores can be divided into four processing blocks. In such an example, each processing block can be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed-precision NVIDIA Tensor cores for deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, dispatch units, and / or a 64KB register file. Furthermore, the streaming microprocessor can include independent parallel integer and floating-point data paths to provide efficient execution of workloads by leveraging the mixture of computation and addressing computation. The streaming microprocessor can include independent thread scheduling capabilities to allow for finer-grained synchronization and cooperation between parallel threads. Streaming microprocessors can include a combination of L1 data cache and shared memory units to improve performance while simplifying programming.

[0122] The GPU 308 may include, in some examples, a High Bandwidth Memory (HBM) and / or a 16GB HBM2 memory subsystem providing a peak memory bandwidth of approximately 900GB / s. In some examples, in addition to HBM memory or alternatively, Synchronous Graphics Random Access Memory (SGRAM), such as Generation 5 Graphics Double Data Rate Synchronous Random Access Memory (GDDR5), may be used.

[0123] The GPU 308 may include unified memory technology, which includes access counters to allow memory pages to be migrated more precisely to the processors that access them most frequently, thereby improving the efficiency of shared memory ranges between processors. In some examples, Address Translation Service (ATS) support can be used to allow the GPU 308 to directly access the CPU 306 page tables. In such examples, when the GPU 308 Memory Management Unit (MMU) experiences a miss, an address translation request can be transferred to the CPU 306. In response, the CPU 306 can look up the virtual-physical mapping for the address in its page tables and transfer the translation back to the GPU 308. Thus, unified memory technology can allow a single unified virtual address space for the memory of both the CPU 306 and the GPU 308, simplifying GPU 308 programming and porting applications to the GPU 308.

[0124] In addition, the GPU 308 may include access counters that track how frequently the GPU 308 accesses the memory of other processors. These access counters help ensure that memory pages are moved to the physical memory of the processor that accesses those pages most frequently.

[0125] SoC 304 may include any number of caches 312, including those described herein. For example, cache 312 may include an L3 cache available to both CPU 306 and GPU 308 (e.g., it is connected to both CPU 306 and GPU 308). Cache 312 may include a write-back cache, which can track the state of rows, for example, using a cache coherence protocol (e.g., MEI, MESI, MSI, etc.). Depending on the embodiment, the L3 cache may include 4 MB or more, but a smaller cache size may also be used.

[0126] SoC 304 may include one or more arithmetic logic units (ALUs) that can be used to perform processing of any of a variety of tasks or operations relating to vehicle 300, such as processing a DNN. Furthermore, SoC 304 may include a floating-point unit (FPU) or other mathematical coprocessor or digital coprocessor type for performing mathematical operations within the system. For example, SoC 304 may include one or more FPUs integrated as execution units within CPU 306 and / or GPU 308.

[0127] SoC 304 may include one or more accelerators 314 (e.g., hardware accelerators, software accelerators, or combinations thereof). For example, SoC 304 may include a hardware acceleration cluster, which may include optimized hardware accelerators and / or large on-chip memory. This large on-chip memory (e.g., 4MB SRAM) can enable the hardware acceleration cluster to accelerate neural networks and other computations. The hardware acceleration cluster can be used to supplement GPU 308 and offload some tasks from GPU 308 (e.g., freeing up more cycles of GPU 308 to perform other tasks). As an example, accelerator 314 can be used for targeted workloads (e.g., perceptrons, convolutional neural networks (CNNs), etc.) that are stable enough to be easily controlled for acceleration. When used herein, the term "CNN" can include all types of CNNs, including region-based or region convolutional neural networks (RCNNs) and fast RCNNs (e.g., for object detection).

[0128] Accelerator 314 (e.g., a hardware acceleration cluster) may include a Deep Learning Accelerator (DLA). The DLA may include one or more Tensor Processing Units (TPUs) that can be configured to provide an additional 10 trillion operations per second for deep learning applications and inference. The TPU may be an accelerator configured to perform image processing functions (e.g., for CNNs, RCNNs, etc.) and optimized for performing image processing functions. The DLA may be further optimized for a specific set of neural network types and floating-point operations as well as inference. The DLA is designed to provide higher performance per millimeter than a general-purpose GPU and significantly outperform CPUs. The TPU can perform several functions, including single-instance convolution functions, support for INT8, INT16, and FP16 data types for both features and weights, and post-processor functions.

[0129] DLA can execute neural networks, especially CNNs, quickly and efficiently on processed or unprocessed data for any function across a wide variety of applications, such as, but not limited to: CNNs for object recognition and detection using data from camera sensors; CNNs for distance estimation using data from camera sensors; CNNs for emergency vehicle detection and recognition using data from microphones; CNNs for face recognition and vehicle owner recognition using data from camera sensors; and / or CNNs for safety and / or safety-related events.

[0130] The DLA can perform any function of the GPU 308, and by using inference accelerators, for example, a designer can make either the DLA or the GPU 308 target any function. For example, a designer can focus the CNN processing and floating-point operations on the DLA and leave other functions to the GPU 308 and / or other accelerators 314.

[0131] Accelerator 314 (e.g., a hardware acceleration cluster) may include a programmable vision accelerator (PVA), which may alternatively be referred to herein as a computer vision accelerator. The PVA may be designed and configured to accelerate computer vision algorithms for advanced driver assistance systems (ADAS), autonomous driving, and / or augmented reality (AR) and / or virtual reality (VR) applications. The PVA can provide a balance between performance and flexibility. For example, each PVA may include, for example, but not limited to, any number of reduced instruction set computer (RISC) cores, direct memory access (DMA), and / or any number of vector processors.

[0132] RISC cores can interact with image sensors (such as the image sensor of any camera described herein), image signal processors, and / or the like. Each of these RISC cores may include any amount of memory. Depending on the embodiment, the RISC core may use any of several protocols. In some examples, the RISC core may execute a real-time operating system (RTOS). RISC cores may be implemented using one or more integrated circuit devices, application-specific integrated circuits (ASICs), and / or memory devices. For example, a RISC core may include an instruction cache and / or tightly coupled RAM.

[0133] DMA enables PVA components to access system memory independently of the CPU 306. DMA can support any number of features to provide optimizations to the PVA, including but not limited to support for multidimensional addressing and / or circular addressing. In some examples, DMA can support addressing in up to six or more dimensions, which can include block width, block height, block depth, horizontal block step, vertical block step, and / or depth step.

[0134] A vector processor can be a programmable processor designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing capabilities. In some examples, a PVA may include a PVA core and two vector processing subsystem partitions. The PVA core may include a processor subsystem, one or more DMA engines (e.g., two DMA engines), and / or other peripherals. The vector processing subsystem may operate as the main processing engine of the PVA and may include a vector processing unit (VPU), an instruction cache, and / or vector memory (e.g., VMEM). The VPU core may include a digital signal processor, such as, for example, a Single Instruction Multiple Data (SIMD) or Very Long Instruction Word (VLIW) digital signal processor. The combination of SIMD and VLIW can enhance throughput and speed.

[0135] Each of the vector processors may include an instruction cache and may be coupled to dedicated memory. Consequently, in some examples, each of the vector processors may be configured to execute independently of other vector processors. In other examples, the vector processors included in a particular PVA may be configured to employ data parallelism. For example, in some embodiments, multiple vector processors included in a single PVA may execute the same computer vision algorithm, but on different regions of an image. In other examples, vector processors included in a particular PVA may execute different computer vision algorithms simultaneously on the same image, or even different algorithms on a sequence of images or portions of an image. Among other things, any number of PVAs may be included in a hardware-accelerated cluster, and any number of vector processors may be included in each of these PVAs. Furthermore, the PVA may include additional error-correcting code (ECC) memory to enhance overall system security.

[0136] Accelerator 314 (e.g., a hardware acceleration cluster) may include an on-chip computer vision network and SRAM to provide high-bandwidth, low-latency SRAM for accelerator 314. In some examples, on-chip memory may include at least 4MB of SRAM consisting of, for example, but not limited to, eight field-configurable memory blocks, accessible by both the PVA and DLA. Each pair of memory blocks may include an Advanced Peripheral Bus (APB) interface, configuration circuitry, a controller, and a multiplexer. Any type of memory may be used. The PVA and DLA may access memory via a backbone that provides high-speed memory access to the PVA and DLA. The backbone may include (e.g., using an APB) an on-chip computer vision network that interconnects the PVA and DLA to memory.

[0137] On-chip computer vision networks can include interfaces that ensure both the PVA and DLA provide ready and valid signals before transmitting any control signals / addresses / data. Such interfaces can provide separate phases and channels for transmitting control signals / addresses / data, as well as burst communication for continuous data transmission. This type of interface can conform to ISO 26262 or IEC 61508 standards, but other standards and protocols can also be used.

[0138] In some examples, SoC 304 may include, for example, a real-time ray tracing hardware accelerator as described in U.S. Patent Application No. 16 / 101,232, filed August 10, 2018. This real-time ray tracing hardware accelerator can be used to quickly and efficiently determine the location and extent of objects (e.g., within a world model) to generate real-time visualization simulations for RADAR signal interpretation, sound propagation synthesis and / or analysis, SONAR system simulation, general wave propagation simulation, comparison with LiDAR data for localization and / or other functional purposes, and / or other uses. In some embodiments, one or more Tree Traversal Units (TTUs) may be used to perform one or more ray tracing-related operations.

[0139] Accelerators 314 (e.g., hardware accelerator clusters) have broad applications in autonomous driving. PVAs can be programmable vision accelerators used in critical processing stages of ADAS and autonomous vehicles. The capabilities of PVAs are a good match for algorithmic domains requiring predictable processing, low power, and low latency. In other words, PVAs perform well in semi-dense or dense rule computation, even on small datasets requiring predictable runtimes with low latency and low power. Therefore, in the context of platforms for autonomous vehicles, PVAs are designed to run classical computer vision algorithms because they are efficient in object detection and integer mathematical operations.

[0140] For example, according to one embodiment of this technology, PVA is used to perform computer stereo vision. In some examples, semi-global matching-based algorithms may be used, but this is not intended to be limiting. Many applications for Level 3–5 autonomous driving require instantaneous motion estimation / stereo matching (e.g., from moving structures, pedestrian recognition, lane detection, etc.). PVA can perform computer stereo vision functions on input from two monocular cameras.

[0141] In some examples, PVA can be used to perform intensive optical flow, processing raw RADAR data (e.g., using 4D Fast Fourier Transform) to provide processed RADAR. In other examples, PVA is used for time-of-flight depth processing, which, for example, involves processing raw time-of-flight data to provide processed time-of-flight data.

[0142] DLA can be used to run any type of network to enhance control and driving safety, including, for example, neural networks that output a confidence metric for each object detection. Such a confidence value can be interpreted as a probability or as providing a relative “weight” for each detection compared to other detections. This confidence value allows the system to make further decisions about which detections should be considered true positives rather than false positives. For example, the system can set a threshold for the confidence and only consider detections exceeding the threshold as true positives. In an Automatic Emergency Braking (AEB) system, false positives can cause the vehicle to automatically perform emergency braking, which is clearly undesirable. Therefore, only the most confident detections should be considered as triggers for AEB. DLA can run a neural network to regress the confidence value. This neural network can take at least a subset of parameters as input, such as bounding box dimensions, ground plane estimates (e.g., from another subsystem), inertial measurement unit (IMU) sensor 366 outputs related to vehicle orientation and distance, 3D position estimates of objects obtained from the neural network and / or other sensors (e.g., LiDAR sensor 364 or RADAR sensor 360), etc.

[0143] SoC 304 may include one or more data storage units 316 (e.g., memory). The data storage unit 316 may be on-chip memory of SoC 304, which may store neural networks to be executed on the GPU and / or DLA. In some examples, for redundancy and security, the data storage unit 316 may be large enough to store multiple instances of the neural network. The data storage unit 316 may include L2 or L3 cache 312. References to the data storage unit 316 may include references to memory associated with PVA, DLA, and / or other accelerators 314 as described herein.

[0144] SoC 304 may include one or more processors 310 (e.g., embedded processors). Processor 310 may include a startup and power management processor, which may be a dedicated processor and subsystem for handling startup power and management functions, as well as safety implementation. The startup and power management processor may be part of the SoC 304 startup sequence and may provide runtime power management services. The startup power and management processor may provide clock and voltage programming, auxiliary system low-power state transitions, SoC 304 thermal and temperature sensor management, and / or SoC 304 power state management. Each temperature sensor may be implemented as a ring oscillator whose output frequency is proportional to the temperature, and SoC 304 may use the ring oscillator to detect the temperature of CPU 306, GPU 308, and / or accelerator 314. If it is determined that the temperature exceeds a threshold, the startup and power management processor may enter a temperature fault routine and place SoC 304 into a lower power state and / or place vehicle 300 into a driver-safe parking mode (e.g., safely stop vehicle 300).

[0145] Processor 310 may further include a set of embedded processors that can be used as an audio processing engine. The audio processing engine can be an audio subsystem that allows for full hardware support for multi-channel audio via multiple interfaces, as well as a wide and flexible range of audio I / O interfaces. In some examples, the audio processing engine is a dedicated processor core with a digital signal processor and dedicated RAM.

[0146] The processor 310 may further include an always-on-processor engine that can provide the necessary hardware features to support low-power sensor management and wake-up use cases. This always-on-processor engine may include a processor core, tightly coupled RAM, support for peripherals (such as timers and interrupt controllers), various I / O controller peripherals, and routing logic.

[0147] Processor 310 may further include a secure cluster engine, which includes a dedicated processor subsystem for handling security management of automotive applications. The secure cluster engine may include two or more processor cores, tightly coupled RAM, support for peripheral devices (e.g., timers, interrupt controllers, etc.), and / or routing logic. In secure mode, the two or more cores may operate in lockstep mode and function as a single core with comparison logic that detects any differences between their operations.

[0148] The processor 310 may further include a real-time camera engine, which may include a dedicated processor subsystem for handling real-time camera management.

[0149] The processor 310 may further include a high dynamic range signal processor, which may include an image signal processor, which is a hardware engine that is part of the camera processing pipeline.

[0150] Processor 310 may include a video image compositer, which may be (e.g., implemented on a microprocessor) a processing block, implementing video post-processing functions required by the video playback application to generate the final image for the player window. The video image compositer may perform lens distortion correction on the wide-angle camera 370, the surround camera 374, and / or the in-cabin monitoring camera sensor. The in-cabin monitoring camera sensor is preferably monitored by a neural network running on another instance of an advanced SoC, configured to recognize in-cabin events and respond accordingly. The in-cabin system may perform lip reading to activate mobile phone services and make calls, dictate emails, change vehicle destinations, activate or change the vehicle's infotainment system and settings, or provide voice-activated web browsing. Some functions are only available to the driver when the vehicle is operating in autonomous mode and are disabled in other situations.

[0151] Video image compositers can include enhanced temporal denoising for both spatial and temporal noise reduction. For example, in the case of motion in the video, denoising appropriately weights spatial information, reducing the weight of information provided by neighboring frames. In cases where the image or part of the image does not contain motion, the temporal denoising performed by the video image compositer can use information from previous images to reduce noise in the current image.

[0152] The video image compositer can also be configured to perform stereo correction on input stereo camera frames. When the operating system desktop is in use and the GPU 308 does not need to continuously render new surfaces, the video image compositer can be further used for user interface components. Even when the GPU 308 is powered on and activated, performing 3D rendering, the video image compositer can be used to offload the GPU 308 to improve performance and responsiveness.

[0153] SoC 304 may further include a Mobile Industry Processor Interface (MIPI) camera serial interface, a high-speed interface, and / or a video input block that can be used for camera and related pixel input functions for receiving video and input from a camera. SoC 304 may further include an input / output controller that can be software-controlled and can be used to receive I / O signals not assigned to a specific role.

[0154] SoC 304 may further include a wide range of peripheral interfaces to enable communication with peripherals, audio codecs, power management and / or other devices. SoC 304 can be used to process data from cameras and sensors (e.g., LIDAR sensor 364, RADAR sensor 360, etc., which can be connected via Gigabit Multimedia Serial Link and Ethernet), data from bus 302 (e.g., vehicle 300 speed, steering wheel position, etc.), and data from GNSS sensor 358 (connected via Ethernet or CAN bus). SoC 304 may further include a dedicated high-performance, high-capacity memory controller, which may include its own DMA engine, and which can be used to free up CPU 306 from routine data management tasks.

[0155] SoC 304 can be an end-to-end platform with a flexible architecture spanning Automation Levels 3-5, providing a comprehensive functional safety architecture that leverages and efficiently utilizes computer vision and ADAS technologies for diversity and redundancy, along with deep learning tools to deliver a flexible and reliable driving software stack. SoC 304 can be faster, more reliable, and even more energy- and space-efficient than conventional systems. For example, when combined with CPU 306, GPU 308, and data storage 316, accelerator 314 can provide a fast and efficient platform for Level 3-5 autonomous vehicles.

[0156] Therefore, this technology offers capabilities and functionalities that cannot be achieved through conventional systems. For example, computer vision algorithms can be executed on CPUs, which can be configured using high-level programming languages ​​such as C to execute a wide variety of processing algorithms across a diverse range of visual data. However, CPUs often cannot meet the performance requirements of many computer vision applications, such as those related to execution time and power consumption. In particular, many CPUs cannot execute complex object detection algorithms in real time, which is a requirement for automotive ADAS applications and practical Level 3-5 autonomous vehicles.

[0157] In contrast to conventional systems, the techniques described in this paper, by providing CPU complexes, GPU complexes, and hardware acceleration clusters, allow multiple neural networks to be executed simultaneously and / or sequentially, and the results combined to achieve Level 3–5 autonomous driving capabilities. For example, a CNN executed on a DLA or dGPU (e.g., GPU 320) could include text and word recognition, allowing a supercomputer to read and understand traffic signs, including those for which neural networks have not yet been specifically trained. The DLA could further include a neural network capable of recognizing, interpreting, and providing semantic understanding of the signs, and passing that semantic understanding to a path planning module running on a CPU complex.

[0158] As another example, multiple neural networks can operate simultaneously, as required for Level 3, 4, or 5 driving. For instance, a warning sign consisting of "Caution: Flashing lights indicate icy conditions," along with a light, can be interpreted independently or jointly by several neural networks. The sign itself can be recognized as a traffic sign by a deployed first neural network (e.g., a trained neural network), and the text "Flashing lights indicate icy conditions" can be interpreted by a deployed second neural network, which informs the vehicle's path planning software (preferably executing on a CPU complex) that icy conditions exist when the flashing lights are detected. The flashing lights can be identified by a deployed third neural network operating across multiple frames, which informs the vehicle's path planning software of the presence (or absence) of the flashing lights. All three neural networks can operate simultaneously, for example, within a DLA and / or on a GPU 308.

[0159] In some examples, the CNN used for facial recognition and owner identification can use data from camera sensors to identify the presence of an authorized driver and / or owner of vehicle 300. A processing engine always on the sensors can be used to unlock the vehicle and turn on the lights when the owner approaches the driver's door, and in safe mode, to disable the vehicle when the owner leaves. In this way, SoC 304 provides security against theft and / or carjacking.

[0160] In another example, the CNN used for emergency vehicle detection and identification can use data from microphone 396 to detect and identify emergency vehicle siren. In contrast to conventional systems that use a general classifier to detect siren and manually extract features, SoC 304 uses a CNN to classify environmental and urban sounds as well as visual data. In a preferred embodiment, the CNN running on the DLA is trained to identify the relative shut-off rate of emergency vehicles (e.g., by using the Doppler effect). The CNN can also be trained to identify emergency vehicles specific to the localized area in which the vehicle operates, as identified by GNSS sensor 358. Thus, for example, when operating in Europe, the CNN will seek to detect European siren, and when operating in the United States, the CNN will seek to identify siren only in North America. Once an emergency vehicle is detected, with the assistance of ultrasonic sensor 362, the control program can be used to execute emergency vehicle safety routines, causing the vehicle to slow down, pull over to the side of the road, stop, and / or idle until the emergency vehicle passes.

[0161] The vehicle may include a CPU 318 (e.g., a discrete CPU or dCPU) that can be coupled to the SoC 304 via a high-speed interconnect (e.g., PCIe). The CPU 318 may include, for example, an x86 processor. The CPU 318 can be used to perform any of a wide variety of functions, including, for example, arbitrating the results of potential inconsistencies between ADAS sensors and the SoC 304, and / or monitoring the status and health of the controller 336 and / or the infotainment SoC 330.

[0162] Vehicle 300 may include a GPU 320 (e.g., a discrete GPU or dGPU) that can be coupled to SoC 304 via a high-speed interconnect (e.g., NVIDIA's NVLINK). GPU 320 may provide additional artificial intelligence capabilities, for example by executing redundant and / or different neural networks, and can be used to train and / or update neural networks based on inputs from sensors of vehicle 300 (e.g., sensor data).

[0163] Vehicle 300 may further include a network interface 324, which may include one or more wireless antennas 326 (e.g., one or more wireless antennas for different communication protocols, such as cellular antennas, Bluetooth antennas, etc.). Network interface 324 can be used to enable wireless connectivity via the Internet to the cloud (e.g., with server 378 and / or other network devices), with other vehicles, and / or with computing devices (e.g., a passenger's client device). For communication with other vehicles, a direct link can be established between the two vehicles, and / or an indirect link can be established (e.g., across a network and via the Internet). A direct link can be provided using a vehicle-to-vehicle communication link. The vehicle-to-vehicle communication link can provide vehicle 300 with information about vehicles approaching vehicle 300 (e.g., vehicles in front, to the side, and / or behind vehicle 300). This functionality can be part of vehicle 300's cooperative adaptive cruise control function.

[0164] Network interface 324 may include a SoC that provides modulation and demodulation functions and enables controller 336 to communicate via a wireless network. Network interface 324 may include an RF front-end for up-conversion from baseband to RF and down-conversion from RF to baseband. Frequency conversion can be performed using known processes and / or using a superheterodyne process. In some examples, the RF front-end functionality may be provided by a separate chip. The network interface may include wireless functions for communication via LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and / or other wireless protocols.

[0165] Vehicle 300 may further include data storage 328, which may include off-chip (e.g., outside of SoC 304) storage devices. Data storage 328 may include one or more storage elements, including RAM, SRAM, DRAM, VRAM, flash memory, hard disk, and / or other components and / or devices capable of storing at least one bit of data.

[0166] Vehicle 300 may further include a GNSS sensor 358. The GNSS sensor 358 (e.g., GPS, assisted GPS sensor, differential GPD (DGPS) sensor, etc.) is used for auxiliary mapping, sensing, occupancy grid generation, and / or path planning functions. Any number of GNSS sensors 358 can be used, including, for example, but not limited to, GPS using a USB connector with an Ethernet-to-serial (RS-232) bridge.

[0167] Vehicle 300 may further include a RADAR sensor 360. The RADAR sensor 360 can be used by vehicle 300 for remote vehicle detection even in dark and / or inclement weather conditions. The RADAR functional safety level may be ASIL B. The RADAR sensor 360 can use CAN and / or bus 302 (e.g., to transmit data generated by the RADAR sensor 360) for control and access to object tracking data, and in some examples, Ethernet access for accessing raw data. A wide variety of RADAR sensor types can be used. For example, and without limitation, the RADAR sensor 360 can be adapted for front, rear, and side RADAR use. In some examples, a pulse Doppler RADAR sensor is used.

[0168] RADAR sensor 360 can include different configurations, such as long-range with a narrow field of view, short-range with a wide field of view, short-range side coverage, etc. In some examples, long-range RADAR can be used for adaptive cruise control functions. A long-range RADAR system can provide a wide field of view (e.g., within 250m) achieved through two or more independent scans. RADAR sensor 360 can help distinguish between stationary and moving objects and can be used by ADAS systems for emergency braking assist and forward collision warning. Long-range RADAR sensors can include a single-site multi-mode RADAR with multiple (e.g., six or more) fixed RADAR antennas and high-speed CAN and FlexRay interfaces. In an example with six antennas, the four central antennas can create a focused beam pattern designed to record the vehicle 300's surroundings at higher rates with minimal traffic interference from adjacent lanes. The other two antennas can extend the field of view, enabling rapid detection of vehicles entering or leaving the vehicle 300's lane.

[0169] As an example, a mid-range RADAR system can include a range of up to 160m (front) or 80m (rear) and a field of view of up to 42 degrees (front) or 150 degrees (rear). Short-range RADAR systems can include, but are not limited to, RADAR sensors designed to be mounted at both ends of the rear bumper. When mounted at both ends of the rear bumper, such a RADAR sensor system can create two beams that continuously monitor blind spots behind and beside the vehicle.

[0170] Short-range RADAR systems can be used in ADAS systems for blind spot detection and / or lane change assistance.

[0171] Vehicle 300 may further include ultrasonic sensors 362. Ultrasonic sensors 362, which may be positioned at the front, rear, and / or sides of vehicle 300, can be used for parking assistance and / or creating and updating occupancy grids. A wide variety of ultrasonic sensors 362 can be used, and different ultrasonic sensors 362 can be used for different detection ranges (e.g., 2.5m, 4m). Ultrasonic sensors 362 can operate at functional safety level ASIL B.

[0172] Vehicle 300 may include a LIDAR sensor 364. The LIDAR sensor 364 may be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. The LIDAR sensor 364 may be of functional safety level ASIL B. In some examples, vehicle 300 may include multiple LIDAR sensors 364 (e.g., two, four, six, etc.) that can use Ethernet (e.g., to provide data to a Gigabit Ethernet switch).

[0173] In some examples, the LiDAR sensor 364 may be able to provide a list of objects and their distances within a 360-degree field of view. Commercially available LiDAR sensors 364 may have an advertising range of, for example, approximately 100m, with an accuracy of 2cm-3cm, and support for 100Mbps Ethernet connectivity. In some examples, one or more non-protruding LiDAR sensors 364 may be used. In such examples, the LiDAR sensor 364 may be implemented as a small device that can be embedded in the front, rear, sides, and / or corners of a vehicle 300. In such examples, the LiDAR sensor 364 may provide a horizontal field of view of up to 120 degrees and a vertical field of view of 35 degrees, even for low-reflectivity objects, with a range of 200m. Front-mounted LiDAR sensors 364 may be configured for a horizontal field of view between 45 degrees and 135 degrees.

[0174] In some examples, LiDAR technologies such as 3D flash LiDAR can also be used. 3D flash LiDAR uses a flash of laser light as the emission source to illuminate the vehicle's surroundings up to approximately 200 meters. A flash LiDAR unit includes a receiver that records the laser pulse propagation time and reflected light on each pixel, which in turn corresponds to the range from the vehicle to the object. Flash LiDAR allows for the generation of highly accurate and distortion-free images of the surrounding environment using each laser flash. In some examples, four flash LiDAR sensors can be deployed, one on each side of the vehicle. Available 3D flash LiDAR systems include solid-state 3D staring array LiDAR cameras (e.g., non-browsing LiDAR devices) without moving parts other than a fan. Flash LiDAR devices can use 5 nanosecond Class I (eye-safe) laser pulses per frame and can capture reflected laser light in the form of a 3D range point cloud and co-registered intensity data. By using flash LiDAR, and because flash LiDAR is a solid-state device with no moving parts, LiDAR sensor 364 is less susceptible to motion blur, vibration, and / or shock.

[0175] The vehicle may further include an IMU sensor 366. In some examples, the IMU sensor 366 may be located at the center of the rear axle of the vehicle 300. The IMU sensor 366 may include, for example, but not limited to, an accelerometer, a magnetometer, a gyroscope, a magnetic compass, and / or other sensor types. In some examples, such as in a six-axis application, the IMU sensor 366 may include an accelerometer and a gyroscope, while in a nine-axis application, the IMU sensor 366 may include an accelerometer, a gyroscope, and a magnetometer.

[0176] In some embodiments, the IMU sensor 366 can be implemented as a miniature, high-performance GPS-assisted inertial navigation system (GPS / INS) that combines a microelectromechanical system (MEMS) inertial sensor, a high-sensitivity GPS receiver, and an advanced Kalman filter algorithm to provide estimates of position, velocity, and attitude. Thus, in some examples, the IMU sensor 366 can enable the vehicle 300 to estimate heading by directly observing and correlating velocity changes from GPS to the IMU sensor 366 without input from a magnetic sensor. In some examples, the IMU sensor 366 and the GNSS sensor 358 can be combined into a single integrated unit.

[0177] The vehicle may include a microphone 396 placed in and / or around the vehicle 300. Among other things, the microphone 396 may be used for emergency vehicle detection and identification.

[0178] The vehicle may further include any number of camera types, including stereo cameras 368, wide-angle cameras 370, infrared cameras 372, surround cameras 374, long-range and / or mid-range cameras 398, and / or other camera types. These cameras can be used to capture image data around the entire perimeter of the vehicle 300. The types of cameras used depend on the embodiment and the requirements of the vehicle 300, and any combination of camera types can be used to provide the necessary coverage around the vehicle 300. Furthermore, the number of cameras may vary depending on the embodiment. For example, the vehicle may include six cameras, seven cameras, ten cameras, twelve cameras, and / or another number of cameras. As an example and without limitation, these cameras may support Gigabit Multimedia Serial Link (GMSL) and / or Gigabit Ethernet. Each of the cameras is described herein with respect to... Figure 3A and Figure 3B It was described in more detail.

[0179] Vehicle 300 may further include vibration sensor 342. Vibration sensor 342 can measure vibrations of vehicle components such as axles. For example, changes in vibration can indicate changes in the road surface. In another example, when two or more vibration sensors 342 are used, differences between vibrations can be used to determine friction or slippage on the road surface (e.g., when there is a vibration difference between a power drive shaft and a freely rotating shaft).

[0180] Vehicle 300 may include ADAS system 338. In some examples, ADAS system 338 may include SoC. ADAS system 338 may include autonomous / adaptive / automatic cruise control (ACC), cooperative adaptive cruise control (CACC), forward collision warning (FCW), automatic emergency braking (AEB), lane departure warning (LDW), lane keeping assist (LKA), blind spot warning (BSW), rear cross traffic warning (RCTW), collision warning system (CWS), lane centering (LC) and / or other features and functions.

[0181] ACC systems can utilize RADAR sensors 360, LIDAR sensors 364, and / or cameras. ACC systems can include longitudinal ACC and / or lateral ACC. Longitudinal ACC monitors and controls the distance to vehicles immediately in front of vehicle 300 and automatically adjusts the vehicle speed to maintain a safe distance. Lateral ACC performs distance holding and, if necessary, advises vehicle 300 to change lanes. Lateral ACC is associated with other ADAS applications such as LCA and CWS.

[0182] CACC uses information from other vehicles, which can be received indirectly from other vehicles via a wireless link or network connection (e.g., via the Internet) through network interface 324 and / or wireless antenna 326. Direct links can be provided by vehicle-to-vehicle (V2V) communication links, while indirect links can be infrastructure-to-vehicle (I2V) communication links. Typically, the V2V communication concept provides information about vehicles immediately ahead (e.g., vehicles immediately in front of vehicle 300 and in the same lane), while the I2V communication concept provides information about traffic further ahead. A CACC system can include either or both of these I2V and V2V information sources. Given information about vehicles ahead of vehicle 300, CACC can be more reliable, and it has the potential to improve traffic flow and reduce road congestion.

[0183] The Forward-Looking Warning (FCW) system is designed to alert the driver to hazards, enabling the driver to take corrective action. The FCW system uses a front-facing camera and / or RADAR sensor 360 coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically coupled to driver feedback such as a display, speaker, and / or vibrating components. The FCW system can provide warnings in the form of, for example, audible, visual, haptic, and / or rapid braking pulses.

[0184] An AEB (Autonomous Emergency Braking) system detects an impending forward collision with another vehicle or other object and can automatically apply the brakes if the driver does not take corrective action within a specified time or distance parameter. The AEB system can use a front-facing camera and / or RADAR sensor 360 coupled to a dedicated processor, DSP, FPGA, and / or ASIC. When the AEB system detects a hazard, it typically first alerts the driver to take corrective action to avoid a collision, and if the driver does not take corrective action, the AEB system can automatically apply the brakes to attempt to prevent or at least mitigate the effects of the predicted collision. The AEB system may include technologies such as dynamic brake support and / or collision proximity braking.

[0185] The Lane Departure Warning (LDW) system provides visual, auditory, and / or tactile warnings, such as steering wheel or seat vibrations, to alert the driver when the vehicle crosses a lane marking. When the driver indicates intentional lane departure, the LDW system is deactivated by activating a turn signal. The LDW system can utilize a front-facing camera coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically coupled to driver feedback such as a display, speaker, and / or vibrating components.

[0186] The Lane Keeping Assist (LKA) system is a variant of the Lane Departure Warning (LDW) system. If vehicle 300 begins to leave its lane, the LKA system provides corrective steering input or braking to vehicle 300. The Blind Spot Warning (BSW) system detects and warns the driver of vehicles in the vehicle's blind spot. The BSW system can provide visual, audible, and / or tactile warnings to indicate that merging or changing lanes is unsafe. The system can provide additional warnings when the driver uses a turn signal. The BSW system can use one or more rear-facing cameras and / or one or more RADAR sensors.

[0187] RCTW systems can provide visual, auditory, and / or tactile notifications when an object is detected outside the range of a rear-view camera while the vehicle is reversing. Some RCTW systems include AEB (Autonomous Emergency Braking) to ensure the application of the vehicle's brakes to avoid a collision. RCTW systems can use one or more rear-view RADAR sensors 360 coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically coupled to driver feedback such as displays, speakers, and / or vibrating components.

[0188] Conventional ADAS systems can be prone to false positives, which can be annoying and distracting for the driver, but typically not catastrophic, as they alert the driver and allow them to determine whether a safe condition truly exists and take appropriate action. However, in an autonomous vehicle 300, in the event of conflicting results, the vehicle 300 itself must decide whether to heed the results from the main computer or auxiliary computer (e.g., the first controller 336 or the second controller 336). For example, in some embodiments, the ADAS system 338 may be a backup and / or auxiliary computer for providing perception information to a backup computer rationality module. The backup computer rationality monitor may run redundant and varied software on hardware components to detect faults in perception and dynamic driving tasks. Outputs from the ADAS system 338 may be provided to a supervisory MCU. If outputs from the main computer and the auxiliary computer conflict, the supervisory MCU must determine how to reconcile the conflict to ensure safe operation.

[0189] In some examples, the master computer can be configured to provide a confidence score to the supervisory MCU, indicating the master computer's confidence level in the selected result. If the confidence score exceeds a threshold, the supervisory MCU can follow the master computer's direction regardless of whether the auxiliary computer provides conflicting or inconsistent results. If the confidence score does not meet the threshold and the master and auxiliary computers indicate different results (e.g., conflict), the supervisory MCU can arbitrate between these computers to determine the appropriate result.

[0190] The supervisory MCU can be configured to run a neural network trained and configured to determine the conditions under which the auxiliary computer provides a false alarm based on outputs from both the host and auxiliary computers. Thus, the neural network in the supervisory MCU can learn when the output of the auxiliary computer can be trusted and when it cannot. For example, when the auxiliary computer is a RADAR-based FCW system, the neural network in the supervisory MCU can learn when the FCW system is identifying a metallic object that is not actually dangerous, such as a drain grid or manhole cover that triggers an alarm. Similarly, when the auxiliary computer is a camera-based LDW system, the neural network in the supervisory MCU can learn to ignore the LDW when a cyclist or pedestrian is present and lane departure is actually the safest strategy. In embodiments that include a neural network running on the supervisory MCU, the supervisory MCU may include at least one of a DLA or GPU suitable for running the neural network using associated memory. In a preferred embodiment, the supervisory MCU may include a component of SoC 304 and / or be included as a component of SoC 304.

[0191] In other examples, ADAS system 338 may include an auxiliary computer that performs ADAS functions using conventional computer vision rules. This allows the auxiliary computer to use classic computer vision rules (if-then), and the presence of neural networks in the supervising MCU can improve reliability, safety, and performance. For example, diverse implementations and intentional non-identity make the entire system more fault-tolerant, especially for failures caused by software (or software-hardware interface) functionality. For instance, if a software vulnerability or bug exists in the software running on the host computer and non-identical software code running on the auxiliary computer provides the same overall result, the supervising MCU can be more confident that the overall result is correct and that the vulnerability in the software or hardware on the host computer does not cause a substantial error.

[0192] In some examples, the output of ADAS system 338 can be fed to the perception block and / or the dynamic driving task block of the main computer. For example, if ADAS system 338 issues a forward collision warning because an object is immediately in front, the perception block can use this information when identifying the object. In other examples, the assistance computer can have its own neural network, which is trained and thus reduces the risk of false positives as described herein.

[0193] Vehicle 300 may further include an infotainment SoC 330 (e.g., an in-vehicle infotainment system (IVI)). Although illustrated and described as an SoC, the infotainment system may not be an SoC and may include two or more discrete components. The infotainment SoC 330 may include a combination of hardware and software that can be used to provide vehicle 300 with audio (e.g., music, personal digital assistant, navigation instructions, news, radio, etc.), video (e.g., TV, movies, streaming media, etc.), telephone (e.g., hands-free calling), network connectivity (e.g., LTE, Wi-Fi, etc.) and / or information services (e.g., navigation system, rear parking assistance, radio data system, vehicle-related information such as fuel level, total coverage distance, brake fuel level, fuel level, door opening / closing, air filter information, etc.). For example, the infotainment SoC 330 may include a radio, disc player, navigation system, video player, USB and Bluetooth connectivity, in-vehicle computer, in-vehicle entertainment, Wi-Fi, steering wheel audio controls, hands-free voice controls, head-up display (HUD), HMI display 334, telematics device, control panel (e.g., for controlling and / or interacting with various components, features, and / or systems) and / or other components. The infotainment SoC 330 may further be used to provide information (e.g., visual and / or auditory) to the vehicle's users, such as information from ADAS system 338, autonomous driving information such as planned vehicle maneuvers, trajectories, surrounding environment information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.

[0194] The infotainment SoC 330 may include GPU functionality. The infotainment SoC 330 can communicate with other devices, systems, and / or components of the vehicle 300 via bus 302 (e.g., CAN bus, Ethernet, etc.). In some examples, the infotainment SoC 330 may be coupled to a supervisory MCU, allowing the GPU of the infotainment system to perform autonomous driving functions in the event of a failure of the main controller 336 (e.g., the primary and / or backup computer of the vehicle 300). In such an example, the infotainment SoC 330 may place the vehicle 300 into a driver-safe parking mode as described herein.

[0195] Vehicle 300 may further include instrument cluster 332 (e.g., digital instrument panel, electronic instrument cluster, digital instrument panel, etc.). Instrument cluster 332 may include a controller and / or supercomputer (e.g., a discrete controller or supercomputer). Instrument cluster 332 may include a set of instruments such as speedometer, fuel level, oil pressure, tachometer, odometer, turn indicator, shift position indicator, seatbelt warning light, parking brake warning light, engine malfunction indicator, airbag (SRS) system information, lighting controls, safety system controls, navigation information, etc. In some examples, information may be displayed and / or shared between infotainment SoC 330 and instrument cluster 332. In other words, instrument cluster 332 may be included as part of infotainment SoC 330, or vice versa.

[0196] Figure 3D For cloud-based servers and according to some embodiments of this disclosure Figure 3A This is a schematic diagram of a system for communication between example autonomous vehicles 300. System 376 may include server 378, network 390, and vehicles including vehicle 300. Server 378 may include multiple GPUs 384(A)-384(H) (collectively referred to herein as GPU 384), PCIe switches 382(A)-382(H) (collectively referred to herein as PCIe switch 382), and / or CPUs 380(A)-380(B) (collectively referred to herein as CPU 380). GPUs 384, CPUs 380, and PCIe switches may be interconnected with high-speed interconnects and / or PCIe connections 386, such as, but not limited to, NVLink interface 388 developed by NVIDIA. In some examples, GPUs 384 are connected via NVLink and / or NVSwitch SoCs, and GPUs 384 and PCIe switches 382 are connected via PCIe interconnects. Although eight GPUs 384, two CPUs 380, and two PCIe switches are shown in the diagram, this is not intended to be limiting. Depending on the embodiment, each of the servers 378 may include any number of GPUs 384, CPUs 380, and / or PCIe switches. For example, each of the servers 378 may include eight, sixteen, thirty-two, and / or more GPUs 384.

[0197] Server 378 can receive image data from vehicles via network 390, representing images of unexpected or changed road conditions such as recently commenced roadworks. Server 378 can also transmit neural network 392, updated neural network 392, and / or map information 394, including information about traffic and road conditions, to vehicles via network 390. Updates to map information 394 may include updates to HD map 322, such as information about construction sites, potholes, bends, floods, or other obstacles. In some examples, neural network 392, updated neural network 392, and / or map information 394 may have been generated from new training and / or data received from any number of vehicles in the environment, and / or based on experience gained from training performed at a data center (e.g., using server 378 and / or other servers).

[0198] Server 378 can be used to train machine learning models (e.g., neural networks) based on training data. Training data can be generated by the vehicle and / or generated in a simulation (e.g., using a game engine). In some examples, the training data is labeled (e.g., where the neural network benefits from supervised learning) and / or undergoes other preprocessing, while in other examples, the training data is not labeled and / or preprocessed (e.g., where the neural network does not require supervised learning). Training can be performed according to any one or more classes of machine learning techniques, including but not limited to: supervised training, semi-supervised training, unsupervised training, self-learning, reinforcement learning, joint learning, transfer learning, feature learning (including principal component analysis and cluster analysis), multilinear subspace learning, manifold learning, representation learning (including alternative dictionary learning), rule-based machine learning, anomaly detection, and any variations or combinations thereof. Once the machine learning model is trained, it can be used by the vehicle (e.g., transmitted to the vehicle via network 390), and / or the machine learning model can be used by server 378 to remotely monitor the vehicle.

[0199] In some examples, server 378 can receive data from vehicles and apply that data to state-of-the-art real-time neural networks for real-time intelligent inference. Server 378 may include a deep learning supercomputer powered by GPU 384 and / or a dedicated AI computer, such as the DGX and DGX Station machines developed by NVIDIA. However, in some examples, server 378 may include a deep learning infrastructure in a data center that uses only CPU power.

[0200] The deep learning infrastructure of server 378 may be capable of rapid real-time inference and can be used to assess and verify the health status of the processor, software, and / or associated hardware in vehicle 300. For example, the deep learning infrastructure may receive periodic updates from vehicle 300, such as image sequences and / or objects located in those image sequences that vehicle 300 has already located (e.g., via computer vision and / or other machine learning object classification techniques). The deep learning infrastructure may run its own neural network to identify objects and compare them with objects identified by vehicle 300. If the results do not match and the infrastructure concludes that the AI ​​in vehicle 300 has malfunctioned, then server 378 may transmit a signal to vehicle 300 instructing the vehicle 300's fail-safe computer to take control, notify passengers, and complete a safe stopping operation.

[0201] For inference, server 378 may include GPU 384 and one or more programmable inference accelerators (such as NVIDIA's TensorRT 3). The combination of GPU-powered servers and inference acceleration enables real-time response. In other examples, such as where performance is less critical, CPU, FPGA, and other processor-powered servers can be used for inference.

[0202] Example computing device

[0203] Figure 4 The diagram below is a block diagram of an example computing device 400 suitable for implementing some embodiments of this disclosure. The computing device 400 may include an interconnect system 402 directly or indirectly coupled to the following devices: memory 404, one or more central processing units (CPUs) 406, one or more graphics processing units (GPUs) 408, a communication interface 410, input / output (I / O) ports 412, input / output components 414, a power supply 416, one or more presentation components 418 (e.g., displays), and one or more logic units 420. In at least one embodiment, the computing device 400 may include one or more virtual machines (VMs), and / or any component thereof may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 408 may include one or more vGPUs, one or more CPUs 406 may include one or more vCPUs, and / or one or more logic units 420 may include one or more virtual logic units. Therefore, computing device 400 may include discrete components (e.g., a complete GPU dedicated to computing device 400), virtual components (e.g., a portion of the GPU dedicated to computing device 400), or a combination thereof.

[0204] although Figure 4The various boxes are shown connected via an interconnect system 402 with wiring, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, a presentation component 418, such as a display device, can be considered an I / O component 414 (e.g., if the display is a touchscreen). As another example, CPU 406 and / or GPU 408 may include memory (e.g., memory 404 may represent a storage device other than the memory of GPU 408, CPU 406, and / or other components). In other words, Figure 4 The computing devices mentioned are merely illustrative. No distinction is made between categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all of these are considered within the same category. Figure 4 Within the scope of computing devices.

[0205] Interconnect system 402 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 402 may include one or more link or bus types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Fast (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 406 may be directly connected to memory 404. Furthermore, CPU 406 may be directly connected to GPU 408. In cases where there is a direct or point-to-point connection between components, interconnect system 402 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required in computing device 400.

[0206] Memory 404 can include any medium of a wide variety of computer-readable media. Computer-readable media can be any available medium that can be accessed by computing device 400. Computer-readable media can include volatile and non-volatile media, as well as removable and non-removable media. For example and without limitation, computer-readable media can include computer storage media and communication media.

[0207] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media, implemented in any way or by any method or technique for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 404 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by computing device 400. As used herein, computer storage media does not include the signal itself.

[0208] Computer storage media may contain computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and include any information transport medium. The term "modulated data signal" can refer to a signal whose characteristics are set or altered in a manner that encodes information into that signal. For example and without limitation, computer storage media may include wired media such as wired networks or direct wired connections, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0209] CPU 406 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 400 to perform one or more of the methods and / or processes described herein. Each of CPUs 406 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads simultaneously. CPU 406 may include any type of processor and may include different types of processors depending on the type of computing device 400 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 400, the processor may be an advanced RISC mechanism (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors such as math coprocessors, computing device 400 may also include one or more CPUs 406.

[0210] In addition to or replacing CPU 406, GPU 408 may also be configured to execute at least some computer-readable instructions to control one or more components of computing device 400 to perform one or more of the methods and / or processes described herein. One or more GPUs 408 may be integrated GPUs (e.g., having one or more CPUs 406) and / or one or more GPUs 408 may be discrete GPUs. In embodiments, one or more GPUs 408 may be coprocessors of one or more CPUs 406. Computing device 400 may use GPU 408 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPU 408 may be used for general-purpose computing on a GPU (GPGPU). GPU 408 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. GPU 408 may generate pixel data for outputting an image in response to rendering commands (e.g., rendering commands received via a host interface from CPU 406). GPU 408 may include graphics memory, such as display memory, for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory may be included as part of memory 404. GPU 408 may include two or more GPUs operating in parallel (e.g., via a link). The link may connect the GPUs directly (e.g., using NVLINK) or via a switch (e.g., using NVSwitch). When combined, each GPU 408 may generate different portions of pixel data or GPGPU data for different outputs (e.g., the first GPU for the first image, the second GPU for the second image). Each GPU may include its own memory or may share memory with other GPUs.

[0211] In addition to or replacing CPU 406 and / or GPU 408, logic unit 420 may be configured to execute at least some computer-readable instructions to control one or more components of computing device 400 to perform one or more methods and / or processes described herein. In embodiments, CPU 406, GPU 408, and / or logic unit 420 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 420 may be part of and / or integrated into one or more CPUs 406 and / or one or more GPUs 408, and / or one or more logic units 420 may be discrete components of CPU 406 and / or GPU 408 or otherwise external thereto. In embodiments, one or more logic units 420 may be processors of one or more CPUs 406 and / or one or more GPUs 408.

[0212] Examples of logic unit 420 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree traversal unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), arithmetic logic unit (ALU)), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or peripheral component interconnect fast (PCIe) element, etc.

[0213] Communication interface 410 may include one or more receivers, transmitters, and / or transceivers that enable computing device 400 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 410 may include components and functions that enable communication via any of several different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication via Ethernet or InfiniBand), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 420 and / or communication interface 410 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 402 to one or more GPUs 408 (e.g., memory within GPU 408).

[0214] I / O port 412 enables computing device 400 to be logically coupled to other devices, including I / O component 414, presentation component 418, and / or other components, some of which may be built into (e.g., integrated into) computing device 400. Illustrative I / O component 414 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, browsers, printers, wireless devices, and so on. I / O component 414 can provide a Natural User Interface (NUI) for processing user-generated air gestures, voice, or other physiological input. In some instances, the input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 400 (as described in more detail in this disclosure). Computing device 400 may include depth cameras such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. In addition, computing device 400 may include an accelerometer or gyroscope that enables motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 400 to render immersive augmented reality or virtual reality.

[0215] Power supply 416 may include a hard-wired power supply, a battery power supply, or a combination thereof. Power supply 416 may supply power to computing device 400 so that components of computing device 400 can operate.

[0216] The presentation component 418 may include a display (such as a monitor, touch screen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 418 may receive data from other components (such as GPU 408, CPU 406, DPU, etc.) and output that data (e.g., as images, videos, sounds, etc.).

[0217] Example Data Center

[0218] Figure 5 An example data center 500 is shown, which can be used in at least one embodiment of this disclosure. The data center 500 may include a data center infrastructure layer 510, a framework layer 520, a software layer 530, and an application layer 540.

[0219] like Figure 5As shown, the data center infrastructure layer 510 may include a resource coordinator 512, grouped computing resources 514, and node computing resources (“nodes CR”) 516(1)-516(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 516(1)-516(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules, and cooling modules, etc. In some embodiments, one or more nodes CR 516(1)-516(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CR516(1)-516(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more of nodes CR516(1)-516(N) may correspond to virtual machines (VMs).

[0220] In at least one embodiment, the grouped computing resources 514 may include individual groups (not shown) of nodes CR516 housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographic locations. Individual groups of nodes CR516 within the grouped computing resources 514 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several nodes CR516, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0221] Resource coordinator 512 may be configured or otherwise controlled to control one or more nodes CR516(1)-516(N) and / or grouped computing resources 514. In at least one embodiment, resource coordinator 512 may include a Software Design Infrastructure (SDI) management entity for data center 500. Resource coordinator 512 may include hardware, software, or some combination thereof.

[0222] In at least one embodiment, such as Figure 5As shown, framework layer 520 may include a job scheduler 533, a configuration manager 534, a resource manager 536, and a distributed file system 538. Framework layer 520 may include a framework of software 532 supporting software layer 530 and / or one or more applications 542 of application layer 540. Software 532 or application 542 may respectively include web-based service software or applications, such as service software or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 520 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark, which can utilize distributed file system 538 for large-scale data processing (e.g., "big data"). TM (Hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 533 may include a Spark driver for facilitating the scheduling of workloads supported by various layers of data center 500. In at least one embodiment, the configuration manager 534 may be able to configure different layers, such as software layer 530 and framework layer 520 including Spark and a distributed file system 538 for supporting large-scale data processing. The resource manager 536 is able to manage cluster or group computing resources mapped to or allocated for supporting distributed file system 538 and job scheduler 533. In at least one embodiment, cluster or group computing resources may include grouped computing resources 514 at data center infrastructure layer 510. The resource manager 536 may coordinate with resource coordinator 512 to manage these mapped or allocated computing resources.

[0223] In at least one embodiment, the software 532 included in the software layer 530 may include software used by at least a portion of the nodes CR516(1)-516(N), the grouped computing resources 514, and / or the distributed file system 538 of the framework layer 520. One or more types of software may include, but are not limited to, Internet web page search software, email virus browsing software, database software, and streaming video content software.

[0224] In at least one embodiment, the application layer 540 may include one or more applications 542 that can be used by at least a portion of nodes CR516(1)-516(N), grouped computing resources 514, and / or the distributed file system 538 of the framework layer 520. The one or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0225] In at least one embodiment, any of the configuration manager 534, resource manager 536, and resource coordinator 512 can perform any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. Self-modification actions can alleviate the risk of data center operators of data center 500 making potentially poor configuration decisions and can prevent underutilization and / or skewed portions of the data center.

[0226] Data center 500 may include tools, services, software, or other resources for training one or more machine learning models or using one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, a machine learning model can be trained by calculating weight parameters based on a neural network architecture using the software and computing resources described herein with respect to data center 500. In at least one embodiment, by using weight parameters calculated through one or more training techniques, the resources described herein with respect to data center 500 can be used to infer or predict information using trained machine learning models corresponding to one or more neural networks, such as, but not limited to, those described herein.

[0227] In at least one embodiment, the data center 500 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference. Furthermore, one or more software and / or hardware resources described in this disclosure may be configured as a service to allow a user to train or perform information inference, such as image recognition, speech recognition, or other artificial intelligence services.

[0228] Example network environment

[0229] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may... Figure 4 The implementation is carried out on one or more instances of computing device 400—for example, each device may include similar components, features, and / or functions of computing device 400. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of data center 500, examples of which are described herein. Figure 5 To describe in more detail.

[0230] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks, or networks within multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. In cases where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0231] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the server functionality described herein can be implemented on any number of client devices.

[0232] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework for supporting software at the software layer and / or one or more applications at the application layer. The software or applications may respectively include network-based service software or applications. In embodiments, one or more client devices may use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a type of free and open-source software network application framework, such as one that can use a distributed file system for large-scale data processing (e.g., "big data").

[0233] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these various functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). If the connection to a user (e.g., a client device) is relatively close to an edge server, the core server can assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0234] Client devices may include those described in this article. Figure 4 The example computing device 400 described includes at least some components, features, and functions. By way of example and not limitation, the client device may be a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, aircraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming equipment or system, entertainment system, in-vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these described devices, or any other suitable device.

[0235] This disclosure can be described in the general context of machine-usable instructions or computer code, including computer-executable instructions such as program modules, which are executed by a computer or other machine such as a personal digital assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. This disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked via a communication network.

[0236] As used herein, the phrase “and / or” relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, “element A, element B, and / or element C” could include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, “at least one of element A or element B” could include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, “at least one of element A and element B” could include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Moreover, the use of the term “based on” should not be interpreted as “based on only” or “based on only”. Rather, a first element “based on” a second element includes instances where the first element is based on the second element but may also be based on one or more additional elements.

[0237] This document describes in detail the subject matter of this disclosure to satisfy legal requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have envisioned that the claimed subject matter may also be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be construed as implying any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

[0238] For example, the subject matter of this disclosure is illustrated with respect to various aspects described below. For convenience, various examples of aspects of this disclosure are described as numbered examples (1, 2, 3, etc.). These are provided by way of example only and do not limit this disclosure. Unless the context otherwise indicates, aspects of various implementations described herein may be omitted, replaced with aspects of other implementations, or combined with aspects of other implementations. For example, one or more aspects of Example 1 below may be omitted, replaced with one or more aspects of another example (e.g., Example 2) or more examples, or combined with aspects of another example. The following is a non-limiting overview of some example implementations presented herein.

[0239] Example 1: A system comprising:

[0240] A first memory is used to store test images for testing the computing system, the test images including multiple control packages;

[0241] A second memory, used to store at least:

[0242] A first test configuration, the first test configuration indicating a first execution order corresponding to a first set of control packets from the plurality of control packets; and

[0243] A second test configuration, the second test configuration indicating a second execution order corresponding to a second set of control packets from the plurality of control packets; and

[0244] A hardware controller communicatively coupled to the first memory and the second memory, the hardware controller being configured to guide the execution of at least one of a first test corresponding to the first test configuration or a second test corresponding to the second test configuration, based at least on the test image and one of the first test configuration or the second test configuration.

[0245] According to the system described in Example 1, the first test configuration instructs the execution of the first control package set in a first execution order, which is different from the default execution order of the first control package set as indicated by the test image.

[0246] According to the system described in Example 1, the execution of the first test includes: the hardware controller, in response to the first control package set not including one or more of the plurality of control packages, directing the execution of the one or more control packages based at least on the default execution order of the one or more control packages as indicated by the test image.

[0247] According to the system described in Example 1, wherein:

[0248] The storage location of the second memory includes a first field and a second field;

[0249] The first field includes a first entry of the first test configuration, the first entry indicating a first control package in the first control package set;

[0250] The second field includes a second entry of the first test configuration, the second entry indicating a second control package in the first control package set; and

[0251] The execution of the first test includes: the hardware controller instructing the second control package to be executed immediately after the execution of the first control package, based at least on the first entry and the second entry.

[0252] According to the system described in Example 1, the second memory includes a register group containing one or more registers.

[0253] According to the system described in Example 1, one or more of the first test or the second test correspond only to a subset of the plurality of control packages.

[0254] According to the system described in Example 1, one or more of the first test or the second test correspond to all of the plurality of control packages.

[0255] According to the system described in Example 1, one or more of the first control package set or the second control package set respectively include all the control packages executed during the first test or the second test.

[0256] According to the system described in Example 1, one or more of the first control packet set or the second control packet set are subsets of the control packets executed during the first test or the second test.

[0257] According to the system described in Example 1, the system is included in at least one of the following:

[0258] Control systems for autonomous or semi-autonomous machines;

[0259] Sensing systems for autonomous or semi-autonomous machines;

[0260] A system used to perform simulation operations;

[0261] Systems used to perform digital twin operations;

[0262] A system for performing optical transmission simulation;

[0263] A system for performing collaborative content creation for 3D assets;

[0264] A system used to perform deep learning operations;

[0265] A system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;

[0266] A system used to host one or more real-time streaming applications;

[0267] Systems implemented using edge devices;

[0268] Systems implemented using robots;

[0269] A system for performing conversational AI operations;

[0270] A system for performing one or more generative AI operations;

[0271] A system that implements one or more large-scale language model LLMs;

[0272] A system that implements one or more Visual Language Models (VLMs);

[0273] A system that implements one or more multimodal language models;

[0274] A system for generating synthetic data;

[0275] A system containing one or more virtual machines (VMs);

[0276] Systems that are at least partially implemented in data centers; or

[0277] A system that uses cloud computing resources at least in part.

[0278] Example 2: A system comprising:

[0279] A memory for storing test images for testing a computing system, the test images comprising a plurality of sequentially ordered control packets for execution in a linked list;

[0280] A register group, comprising one or more registers, the register group being used for:

[0281] The storage corresponds to a first test of the computing system and a first test configuration corresponding to a first control package set among the plurality of control packages. The first test configuration indicates the execution of the first control package set in a first execution order, which differs from the order of the first control package sets in the linked list.

[0282] A hardware controller, communicatively coupled to the memory and the register set, is used to guide the execution of the first test based at least on the test image and the first test configuration.

[0283] According to the system described in Example 2, wherein:

[0284] The register group is also used for:

[0285] The system stores a second test configuration corresponding to a second test of the computing system and to a second set of control packages among the plurality of control packages. This second test configuration instructs the execution of the second set of control packages in a second execution order, which differs from the sequential order of the second set of control packages in the linked list.

[0286] The hardware controller is also used to guide the execution of the second test based at least on the test image and the second test configuration.

[0287] According to the system described in Example 2, the first test corresponds to:

[0288] Only a subset of the multiple control packets; or

[0289] All of the plurality of control packages.

[0290] According to the system described in Example 2, wherein:

[0291] The first control package set includes all the control packages executed during the first test; or

[0292] The first set of control packets is a subset of the control packets executed during the first test.

[0293] According to the system described in Example 2, the execution of the first test includes: the hardware controller, in response to the first control package set not including one or more of the plurality of control packages, directing the execution of the one or more control packages, at least based on the linked list.

[0294] According to the system described in Example 2, wherein:

[0295] The registers in the register group include a first field and a second field;

[0296] The first field includes a first entry of the first test configuration, the first entry indicating a first control package in the first control package set;

[0297] The second field includes a second entry of the first test configuration, the second entry indicating a second control package in the first control package set; and

[0298] The execution of the first test includes: the hardware controller instructing the second control package to be executed immediately after the execution of the first control package, based at least on the first entry and the second entry.

[0299] Example 3: A method that includes:

[0300] Access a test image used to test the computing system, the test image indicating the default execution order of multiple control packages when testing the computing system;

[0301] Access a test configuration corresponding to a test of the computing system and to a set of control packages among the plurality of control packages, the test configuration instructing the control package set to be executed in an order different from the default execution order; and

[0302] The test of the computing system is performed at least based on the test image and the test configuration.

[0303] According to the method described in Example 3, the default execution order is based at least on the sequential order of the plurality of control packages as indicated by the linked list included in the test image.

[0304] According to the method described in Example 3, the test configuration is accessed from a register group that includes one or more registers and on which the test configuration is loaded.

[0305] According to the method described in Example 3, the test image is accessed from a memory on which the test image is loaded and which is separate from the register set.

Claims

1. A system comprising: a first memory to store a test image for testing a computing system, the test image including a plurality of control packets; a second memory to store at least: a first test configuration, the first test configuration indicating a first execution order corresponding to a first set of control packets from the plurality of control packets; and a second test configuration, the second test configuration indicating a second execution order corresponding to a second set of control packets from the plurality of control packets; and a hardware controller communicatively coupled to the first memory and the second memory, the hardware controller to direct execution of at least one of a first test corresponding to the first test configuration or a second test corresponding to the second test configuration based at least on the test image and one of the first test configuration or the second test configuration.

2. The system of claim 1, wherein, the first test configuration indicates execution of the first set of control packets in a first execution order that is different from a default execution order of the first set of control packets as indicated by the test image.

3. The system of claim 1, wherein, the execution of the first test includes the hardware controller directing execution of one or more control packets of the plurality of control packets based at least on a default execution order of the one or more control packets as indicated by the test image in response to the first set of control packets not including the one or more control packets.

4. The system of claim 1, wherein: a storage location of the second memory includes a first field and a second field; the first field includes a first entry of the first test configuration, the first entry indicating a first control packet of the first set of control packets; the second field includes a second entry of the first test configuration, the second entry indicating a second control packet of the first set of control packets; and the execution of the first test includes the hardware controller directing the second control packet to be executed immediately after execution of the first control packet based at least on the first entry and the second entry.

5. The system of claim 1, wherein, the second memory includes a register bank including one or more registers.

6. The system of claim 1, wherein, one or more of the first test or the second test correspond to only a subset of the plurality of control packets.

7. The system of claim 1, wherein, one or more of the first test or the second test correspond to all of the control packets of the plurality of control packets.

8. The system of claim 1, wherein, one or more of the first set of control packets or the second set of control packets respectively include all of the control packets executed during the first test or the second test.

9. The system of claim 1, wherein, one or more of the first set of control packets or the second set of control packets respectively are a subset of the control packets executed during the first test or the second test.

10. The system of claim 1, wherein, the system is included in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system to perform simulation operations; a system to perform digital twin operations; a system to perform optical transport simulation; a system to perform collaborative content creation for 3D assets; a system to perform deep learning operations; a system to present at least one of augmented reality content, virtual reality content, or mixed reality content; a system to host one or more live streaming applications; Systems implemented using edge devices; Systems implemented using robots; Systems for performing conversational AI operations; Systems for performing one or more generative AI operations; Systems implementing one or more large language models (LLMs); Systems implementing one or more visual language models (VLMs); Systems implementing one or more multi-modal language models; Systems for generating synthetic data; Systems including one or more virtual machines (VMs); Systems implemented at least in part in a data center; or Systems implemented at least in part using cloud computing resources.

11. A system comprising: a memory to store a test image for testing a computing system, the test image including a plurality of control packets that are sequentially ordered for execution in a linked list; a register bank including one or more registers, the register bank to: store a first test configuration corresponding to a first test of the computing system and corresponding to a first set of control packets of the plurality of control packets, the first test configuration to indicate execution of the first set of control packets in a first execution order that is different from a sequential order of the first set of control packets in the linked list; and a hardware controller communicatively coupled to the memory and the register bank and to direct execution of the first test based at least on the test image and the first test configuration.

12. The system of claim 11, wherein: the register bank is further to: store a second test configuration corresponding to a second test of the computing system and corresponding to a second set of control packets of the plurality of control packets, the second test configuration to indicate execution of the second set of control packets in a second execution order that is different from a sequential order of the second set of control packets in the linked list; and the hardware controller is further to direct execution of the second test based at least on the test image and the second test configuration.

13. The system of claim 11, wherein the first test corresponds to: only a subset of the plurality of control packets; or all of the control packets of the plurality of control packets.

14. The system of claim 11, wherein: the first set of control packets includes all of the control packets executed during the first test; or the first set of control packets is a subset of the control packets executed during the first test. the execution of the first test includes the hardware controller directing execution of one or more control packets of the plurality of control packets based at least on the linked list in response to the first set of control packets not including the one or more control packets.

15. The system of claim 11, wherein, 16. The system of claim 11, wherein: a register of the register bank includes a first field and a second field; the first field includes a first entry of the first test configuration, the first entry to indicate a first control packet of the first set of control packets; the second field includes a second entry of the first test configuration, the second entry to indicate a second control packet of the first set of control packets; and the first entry is different from the second entry. ​ The performance of the first test includes the hardware controller directing that the second control packet be executed immediately after execution of the first control packet based at least on the first entry and the second entry.

17. A method comprising: accessing a test image for testing a computing system, the test image indicating a default execution order for a plurality of control packets when testing the computing system; accessing a test configuration corresponding to testing of the computing system and corresponding to a set of control packets of the plurality of control packets, the test configuration indicating execution of the set of control packets in an execution order different from the default execution order; and executing the test of the computing system based at least on the test image and the test configuration.

18. The method of claim 17, wherein, The default execution order is based at least on an order of the plurality of control packets indicated by a linked list included in the test image.

19. The method of claim 17, wherein, The test configuration is accessed from a register bank including one or more registers and loaded with the test configuration.

20. The method of claim 19, wherein, The test image is accessed from a memory loaded with the test image and separate from the register bank.

Images